Downloads & Free Reading Options - Results

Folder: BIOPHYSICAL CHEMIST DEVELOPS THEORY OF BIOINFORMATION PROCESS; STAR GATE was an umbrella term for the Intelligence Community effort that used remote viewers who claimed to use clairvoyance, precognition, or telepathy to acquire and describe information about targets that were blocked from ordinary perception. The records include documentation of remote viewing sessions, training, internal memoranda, foreign assessments, and program reviews. The STAR GATE program was also called SCANATE, GONDOLA WISH, DRAGOON ABSORB, GRILL FLAME, CENTER LANE, SUN STREAK. Files were released through CREST and obtained as TIF files by the Black Vault and converted to PDF by That 1 Archive.

During the past year these investigators: (1) Illustrated application of computer science to neuroscience at three levels: measuring, modeling, and understanding the computational function of the columnar pattern of ocular dominance in primate visual cortex; (2) Demonstrated an algorithm for modeling polymap architectures of the cerebral neocortex, where the term 'polymap' emphasizes the joint occurrence of topographic mapping of multiple sub- modalities, interlaced in the form of macroscopic patches ('columns') into a single cortical lamina; (3) Considered a space-variant sensor design based on the conformal mapping of the halp disk, w=log (z+a), a 0, which characterizes the anatomical structure of the primate and human visual systems. (4) Showed that the best algorithm for fusing multiple space-variant fixations of the same scene show, under certain assumptions of pixel distribution, is indeed optimal in a least-squared-error sense; (5) Analyzed the characteristics of a synthetic sensor comparable, with respect to field width and resolution, to the primate visual system; (6) Showed a quantitative measurement of the macaque ocular dominance column pattern, based on measurement of local power spectral densities of a computer reconstruction and numerical flattening of VI.

The presentations given at this seminar, held in February 1988 in London, UK are reviewed in depth. Topics range from neural systems and models through languages and architectures to the respective European and American perspectives on neurocomputing. Contents: Introduction; Neural Systems and Models; Connectionist Models: Background and Emergent Properties; Historical Perspective; Programing Languages for Neurocomputers; Associative Memories and Representations of Knowledge as Internal States in Distributed Systems; Parallel Architecture for Neurocomputers; Combinatorial Optimization on a Boltzmann Machine; Neural Networks: A European Perspective; Neurocomputing Applications: A United States Perspective.

The presentations given at this seminar, held in February 1988 in London, UK are reviewed in depth. Topics range from neural systems and models through languages and architectures to the respective European and American perspectives on neurocomputing. Contents: Introduction; Neural Systems and Models; Connectionist Models: Background and Emergent Properties; Historical Perspective; Programing Languages for Neurocomputers; Associative Memories and Representations of Knowledge as Internal States in Distributed Systems; Parallel Architecture for Neurocomputers; Combinatorial Optimization on a Boltzmann Machine; Neural Networks: A European Perspective; Neurocomputing Applications: A United States Perspective.

The optical implementation of weighted interconnections is investigated and basic relationship are derived between the number of neurons, the number of connections and methods for selecting the positions of the neurons to achieve the maximum density of independent connections are presented. The connectivity of a neural network (number of synapses per neuron) is related to the complexity of the problems it can handle. For a network that learns a problem from examples using a local learning rule, it is proved that the entropy of the problem becomes a lower bound for the connectivity of the network.

The optical implementation of weighted interconnections is investigated and basic relationship are derived between the number of neurons, the number of connections and methods for selecting the positions of the neurons to achieve the maximum density of independent connections are presented. The connectivity of a neural network (number of synapses per neuron) is related to the complexity of the problems it can handle. For a network that learns a problem from examples using a local learning rule, it is proved that the entropy of the problem becomes a lower bound for the connectivity of the network.

This report describes the application of a neural computing approach for automated initial identification of seismic phases (P or S) recorded by 3- component stations. We use a 3-layer back-propagation neural network to identify phases on the basis of their polarization attributes. This approach is much easier to develop than a more traditional rule-based system because of the high-dimensionality of the input (8-10 polarization attributes), and because the data are station-dependent. The neural network approach also performs 3-7% better than a linear multivariate method. Most of the gain is for signals with low signal-to-noise ratio since the non-linear neural network classifier is less sensitive to outliers (or noisy data) than the linear multivariate method. Another advantage of the neural network approach is that it is easily adapted to data recorded by new stations. For example, we find that we achieve 75-80% identification accuracy for a new station without system retraining (e.g., using a network derived from data from a different station). The data required for retraining can be accumulated in about two weeks of continuous operation of the new station, and training takes less than one hour on a Sun4 Sparc station. After this retraining, the identification accuracy increases to 90%. We have recently added context (e.g., the number of arrivals before and after the arrival under consideration) to the input of the neural network, and we have found that this further improves the identification accuracy by 3-5%. This neural network approach performs better than competing technologies for automated initial phase identification, and it is amenable to machine-learning techniques to automate the process of acquiring new knowledge.

Granular computing is a computational paradigm that mimics human cognition in terms of grouping similar information together. Compatibility operators such as cardinality, orientation, density, and multidimensional length act on both in raw data and information granules which are formed from raw data providing a framework for human-like information processing where information granulation is intrinsic.

We quantify a source of ineffectual computations when processing the multiplications of the convolutional layers in Deep Neural Networks (DNNs) and propose Pragmatic (PRA), an architecture that exploits it improving performance and energy efficiency. The source of these ineffectual computations is best understood in the context of conventional multipliers which generate internally multiple terms, that is, products of the multiplicand and powers of two, which added together produce the final product [1]. At runtime, many of these terms are zero as they are generated when the multiplicand is combined with the zero-bits of the multiplicator. While conventional bit-parallel multipliers calculate all terms in parallel to reduce individual product latency, PRA calculates only the non-zero terms using a) on-the-fly conversion of the multiplicator representation into an explicit list of powers of two, and b) hybrid bit-parallel multplicand/bit-serial multiplicator processing units. PRA exploits two sources of ineffectual computations: 1) the aforementioned zero product terms which are the result of the lack of explicitness in the multiplicator representation, and 2) the excess in the representation precision used for both multiplicants and multiplicators, e.g., [2]. Measurements demonstrate that for the convolutional layers, a straightforward variant of PRA improves performance by 2.6x over the DaDiaNao (DaDN) accelerator [3] and by 1.4x over STR [4]. Similarly, PRA improves energy efficiency by 28% and 10% on average compared to DaDN and STR. An improved cross lane synchronication scheme boosts performance improvements to 3.1x over DaDN. Finally, Pragmatic benefits persist even with an 8-bit quantized representation [5].

We quantify a source of ineffectual computations when processing the multiplications of the convolutional layers in Deep Neural Networks (DNNs) and propose Pragmatic (PRA), an architecture that exploits it improving performance and energy efficiency. The source of these ineffectual computations is best understood in the context of conventional multipliers which generate internally multiple terms, that is, products of the multiplicand and powers of two, which added together produce the final product [1]. At runtime, many of these terms are zero as they are generated when the multiplicand is combined with the zero-bits of the multiplicator. While conventional bit-parallel multipliers calculate all terms in parallel to reduce individual product latency, PRA calculates only the non-zero terms using a) on-the-fly conversion of the multiplicator representation into an explicit list of powers of two, and b) hybrid bit-parallel multplicand/bit-serial multiplicator processing units. PRA exploits two sources of ineffectual computations: 1) the aforementioned zero product terms which are the result of the lack of explicitness in the multiplicator representation, and 2) the excess in the representation precision used for both multiplicants and multiplicators, e.g., [2]. Measurements demonstrate that for the convolutional layers, a straightforward variant of PRA improves performance by 2.6x over the DaDiaNao (DaDN) accelerator [3] and by 1.4x over STR [4]. Similarly, PRA improves energy efficiency by 28% and 10% on average compared to DaDN and STR. An improved cross lane synchronication scheme boosts performance improvements to 3.1x over DaDN. Finally, Pragmatic benefits persist even with an 8-bit quantized representation [5].

Recently, Deep Convolutional Neural Networks (DCNNs) have made unprecedented progress, achieving the accuracy close to, or even better than human-level perception in various tasks. There is a timely need to map the latest software DCNNs to application-specific hardware, in order to achieve orders of magnitude improvement in performance, energy efficiency and compactness. Stochastic Computing (SC), as a low-cost alternative to the conventional binary computing paradigm, has the potential to enable massively parallel and highly scalable hardware implementation of DCNNs. One major challenge in SC based DCNNs is designing accurate nonlinear activation functions, which have a significant impact on the network-level accuracy but cannot be implemented accurately by existing SC computing blocks. In this paper, we design and optimize SC based neurons, and we propose highly accurate activation designs for the three most frequently used activation functions in software DCNNs, i.e, hyperbolic tangent, logistic, and rectified linear units. Experimental results on LeNet-5 using MNIST dataset demonstrate that compared with a binary ASIC hardware DCNN, the DCNN with the proposed SC neurons can achieve up to 61X, 151X, and 2X improvement in terms of area, power, and energy, respectively, at the cost of small precision degradation.In addition, the SC approach achieves up to 21X and 41X of the area, 41X and 72X of the power, and 198200X and 96443X of the energy, compared with CPU and GPU approaches, respectively, while the error is increased by less than 3.07%. ReLU activation is suggested for future SC based DCNNs considering its superior performance under a small bit stream length.

Recently, Deep Convolutional Neural Networks (DCNNs) have made unprecedented progress, achieving the accuracy close to, or even better than human-level perception in various tasks. There is a timely need to map the latest software DCNNs to application-specific hardware, in order to achieve orders of magnitude improvement in performance, energy efficiency and compactness. Stochastic Computing (SC), as a low-cost alternative to the conventional binary computing paradigm, has the potential to enable massively parallel and highly scalable hardware implementation of DCNNs. One major challenge in SC based DCNNs is designing accurate nonlinear activation functions, which have a significant impact on the network-level accuracy but cannot be implemented accurately by existing SC computing blocks. In this paper, we design and optimize SC based neurons, and we propose highly accurate activation designs for the three most frequently used activation functions in software DCNNs, i.e, hyperbolic tangent, logistic, and rectified linear units. Experimental results on LeNet-5 using MNIST dataset demonstrate that compared with a binary ASIC hardware DCNN, the DCNN with the proposed SC neurons can achieve up to 61X, 151X, and 2X improvement in terms of area, power, and energy, respectively, at the cost of small precision degradation.In addition, the SC approach achieves up to 21X and 41X of the area, 41X and 72X of the power, and 198200X and 96443X of the energy, compared with CPU and GPU approaches, respectively, while the error is increased by less than 3.07%. ReLU activation is suggested for future SC based DCNNs considering its superior performance under a small bit stream length.

With the increasing complexity of computing systems, complete hardware reliability can no longer be guaranteed. We need, however, to ensure overall system reliability. One of the most important features of artificial neural networks is their intrinsic fault-tolerance. The aim of this work is to investigate whether such networks have features that can be applied to wider computational systems. This paper presents an analysis, in both the learning and operational phases, of a distributed feed-forward neural network with decentralised event-driven time management, which is insensitive to intermittent faults caused by unreliable communication or faulty hardware components. The learning rules used in the model are local in space and time, which allows efficient scalable distributed implementation. We investigate the overhead caused by injected faults and analyse the sensitivity to limited failures in the computational hardware in different areas of the network.

With the increasing complexity of computing systems, complete hardware reliability can no longer be guaranteed. We need, however, to ensure overall system reliability. One of the most important features of artificial neural networks is their intrinsic fault-tolerance. The aim of this work is to investigate whether such networks have features that can be applied to wider computational systems. This paper presents an analysis, in both the learning and operational phases, of a distributed feed-forward neural network with decentralised event-driven time management, which is insensitive to intermittent faults caused by unreliable communication or faulty hardware components. The learning rules used in the model are local in space and time, which allows efficient scalable distributed implementation. We investigate the overhead caused by injected faults and analyse the sensitivity to limited failures in the computational hardware in different areas of the network.

Analysis of motion contributes to the image understanding tasks by disambiguating scene information whenever, the observer and/or objects in the scene are in motion. This proposal is focused on research and development of algorithms for automatic recalibration from sensory to egocentric coordinates during egomotion.

Carbon nanotubes are often seen as the only alternative technology to silicon transistors. While they are the most likely short-term one, other longer-term alternatives should be studied as well. While contemplating biological neurons as an alternative component may seem preposterous at first sight, significant recent progress in CMOS-neuron interface suggests this direction may not be unrealistic; moreover, biological neurons are known to self-assemble into very large networks capable of complex information processing tasks, something that has yet to be achieved with other emerging technologies. The first step to designing computing systems on top of biological neurons is to build an abstract model of self-assembled biological neural networks, much like computer architects manipulate abstract models of transistors and circuits. In this article, we propose a first model of the structure of biological neural networks. We provide empirical evidence that this model matches the biological neural networks found in living organisms, and exhibits the small-world graph structure properties commonly found in many large and self-organized systems, including biological neural networks. More importantly, we extract the simple local rules and characteristics governing the growth of such networks, enabling the development of potentially large but realistic biological neural networks, as would be needed for complex information processing/computing tasks. Based on this model, future work will be targeted to understanding the evolution and learning properties of such networks, and how they can be used to build computing systems.

This was an instrumentation grant to purchase equipment of support of research in neural networks, information science, artificial intelligence, and applied mathematics. Computer lab equipment, motor control and robotics lab equipment, speech analysis equipment and computational vision equipment were purchased.

This was an instrumentation grant to purchase equipment of support of research in neural networks, information science, artificial intelligence, and applied mathematics. Computer lab equipment, motor control and robotics lab equipment, speech analysis equipment and computational vision equipment were purchased.

The superior colliculus is organized topographically as a neural map. The deep layers of the colliculus detect and localize targets in the environment by integrating input from multiple sensory systems. Some deep colliculus neurons receive input of only one sensor modality (unimodal) while others receive input of multiple modalities. Multimodal deep SC neurons exhibit multisensory enhancement, in which the response to input of one modality is augmented by input of another modality. Multisensory enhancement is magnitude dependent in that combinations of smaller single-modality responses produce larger amounts of enhancement. These findings are consistent with the hypothesis that deep colliculus neurons use sensory input to compute the probability that a target has appeared at their corresponding location in the environment. Multisensor enhancement and its magnitude dependence can be simulated using a model in which sensory inputs are random variables and target probability is computed using Hayes' Rule. Informational analysis of the model indicates that input of another modality can indeed increase the amount of target information received by a multimodal neuron, but only if input of the initial modality is ambiguous. Unimodal deep colliculus neurons may receive unambiguous input of one modality and have no need of input of another modality.

It is difficult to find good mathematical models for many natural problems such as pattern recognition. Not only does this difficulty preclude finding good solutions for these problems, but it also precludes estimating their complexity using the standard tools of the theory oc computational complexity (Traub, 1985). Part of the difficulty can be traced to symptoms such as ill-definition, fuzziness, and inexactness. However, the difficulty of modeling these problems may be inherent in some cases. Keywords: Photorefractive crystals; Adaptive optical networks; Connectivity; Entropy; Holograms.

Membrane bioreactor employs an efficient filtration technology for solid and liquid separation in wastewater treatment process. Development of membrane filtration model is significant as this model can be used to predict filtration dynamic which is later utilized in control development. Most of the available models only suitable for monitoring purpose, which are too complex, required many variables and not suitable for control system design. This work focusing on the simple time seris model for membrane filtration process using neural network technique. In this paper, submerged membrane filtration model developed using recurrent neural network (RNN) train using genetic algorithm (GA), inertia weight particle swarm optimization (IW-PSO) and gravitational search algorithm (GSA). These optimization algorithms are compared in term of its accuracy and convergent speed in updating the weights and biases of the RNN for optimal filtration model. The evaluation of the models is measured using three performance evaluations, which are mean square error (MSE), mean absolute deviation (MAD) and coefficient of determination (R2). From the results obtained, all methods yield satisfactory result for the model, with the best results given by IW-PSO. 

This effort focuses on building an intelligent, energy-efficient cloud architecture to improve cloud computing infrastructures. The rate at which cloud data gets modernized has increased, leading to more reviews comparing the various modernization methodologies and models. Several wealthy nations, including Turkey, have upgraded to more complicated and energy-efficient cloud infrastructure. We designed a Python application that uses an AI framework to maximize cloud computing's usage of computing resources and clean, renewable energy. This plan outlines concepts for a future neural network-trained digital ecosystem (ANN). The ANN model details energy forecasting tasks within a constrained system. Prominent corporations use AI to design policies to secure their cloud infrastructures and digital assets. Cloud computing systems were modernized by acquiring, normalizing, and transforming their file formats. Most cloud-based infrastructures were updated successfully. This was expected, given digital implementations dominate these systems. We'll investigate the energy consumption of AWS, AZURE, GCP, and Digital Ocean. Since most files were still on paper in 2015, the number of upgrades was modest. By 2020, a large part of cloud computing systems will be converted to digital format, with 98.68% accuracy for all cloud computing systems when trained on 80% of the data and evaluated on 20% of the data. Smart energy-efficient cloud solutions are replacing traditional data centers year by year. Smart energy-efficient cloud systems help preserve cloud computing systems and understand how cloud platforms are modernized and perform in energy prediction.

Talk by Ross Gayler for the Redwood Center for Theoretical Neuroscience at UC Berkeley. ABSTRACT. This talk is about computing with discrete compositional data structures in analog computers. A core issue for both computer science and cognitive neuroscience is the degree of match between a class of computer designs and a class of computations. In cognitive science, it is manifested in the apparent mismatch between the neural network hardware of the brain (essentially, a massively parallel analog computer) and the computational requirements of higher cognition (statistical constraint processing with compositional discrete data structures to implement facilities such as language and analogical reasoning). Historically, analog computers have been considered ill-suited for implementing computational processes on discrete compositional data structures. Neural networks can be construed as analog computers -- a class of computer design with a long history, but now relatively unknown. Historically, analog computation had advantages over digital computation in speed and parallelism. Computational problems were cast as dynamical systems and modelled by differential equations, which was relatively straightforward for models of physical problems such as flight dynamics. However, it was far less clear how to translate computations on discrete compositional data structures such as trees and graphs into dynamical systems. This is especially true for problems where the data structures evolve over time, implying the need to rewire the analog computer on the fly. This is particularly relevant to cognitive science because new concepts and relations can be created on the fly, and under the standard conception of neural networks this implies that neurons and connections would be created impossibly rapidly. In this talk I describe Vector Symbolic Architectures, a family of mathematical techniques for analog computation in hyperdimensional vector spaces that map naturally onto neural network implementations. VSAs naturally support computation on discrete compositional data structures and a form of virtualisation that breaks the nexus between the items to be represented and the hardware that supports the representation. This means that computations on evolving data structures do not require physical rewiring of the implementing hardware. I illustrate this approach with a VSA system that finds isomorphisms between graphs and where different problems to be solved are represented by different initial states of the fixed hardware rather than by rewiring the hardware. Graph isomorphism is at the heart of the standard model of analogical reasoning and motivates this example, although that aspect is not explored in this talk. BIO. Ross Gayler has PhD in psychology from University of Queensland, with a long and far-reaching interest in the mysteries of cognitive computing. His day job is as a statistician in the finance industry.

Talk by Ross Gayler for the Redwood Center for Theoretical Neuroscience at UC Berkeley. ABSTRACT. This talk is about computing with discrete compositional data structures in analog computers. A core issue for both computer science and cognitive neuroscience is the degree of match between a class of computer designs and a class of computations. In cognitive science, it is manifested in the apparent mismatch between the neural network hardware of the brain (essentially, a massively parallel analog computer) and the computational requirements of higher cognition (statistical constraint processing with compositional discrete data structures to implement facilities such as language and analogical reasoning). Historically, analog computers have been considered ill-suited for implementing computational processes on discrete compositional data structures. Neural networks can be construed as analog computers -- a class of computer design with a long history, but now relatively unknown. Historically, analog computation had advantages over digital computation in speed and parallelism. Computational problems were cast as dynamical systems and modelled by differential equations, which was relatively straightforward for models of physical problems such as flight dynamics. However, it was far less clear how to translate computations on discrete compositional data structures such as trees and graphs into dynamical systems. This is especially true for problems where the data structures evolve over time, implying the need to rewire the analog computer on the fly. This is particularly relevant to cognitive science because new concepts and relations can be created on the fly, and under the standard conception of neural networks this implies that neurons and connections would be created impossibly rapidly. In this talk I describe Vector Symbolic Architectures, a family of mathematical techniques for analog computation in hyperdimensional vector spaces that map naturally onto neural network implementations. VSAs naturally support computation on discrete compositional data structures and a form of virtualisation that breaks the nexus between the items to be represented and the hardware that supports the representation. This means that computations on evolving data structures do not require physical rewiring of the implementing hardware. I illustrate this approach with a VSA system that finds isomorphisms between graphs and where different problems to be solved are represented by different initial states of the fixed hardware rather than by rewiring the hardware. Graph isomorphism is at the heart of the standard model of analogical reasoning and motivates this example, although that aspect is not explored in this talk. BIO. Ross Gayler has PhD in psychology from University of Queensland, with a long and far-reaching interest in the mysteries of cognitive computing. His day job is as a statistician in the finance industry.

Talk by Ross Gayler for the Redwood Center for Theoretical Neuroscience at UC Berkeley. ABSTRACT. This talk is about computing with discrete compositional data structures in analog computers. A core issue for both computer science and cognitive neuroscience is the degree of match between a class of computer designs and a class of computations. In cognitive science, it is manifested in the apparent mismatch between the neural network hardware of the brain (essentially, a massively parallel analog computer) and the computational requirements of higher cognition (statistical constraint processing with compositional discrete data structures to implement facilities such as language and analogical reasoning). Historically, analog computers have been considered ill-suited for implementing computational processes on discrete compositional data structures. Neural networks can be construed as analog computers -- a class of computer design with a long history, but now relatively unknown. Historically, analog computation had advantages over digital computation in speed and parallelism. Computational problems were cast as dynamical systems and modelled by differential equations, which was relatively straightforward for models of physical problems such as flight dynamics. However, it was far less clear how to translate computations on discrete compositional data structures such as trees and graphs into dynamical systems. This is especially true for problems where the data structures evolve over time, implying the need to rewire the analog computer on the fly. This is particularly relevant to cognitive science because new concepts and relations can be created on the fly, and under the standard conception of neural networks this implies that neurons and connections would be created impossibly rapidly. In this talk I describe Vector Symbolic Architectures, a family of mathematical techniques for analog computation in hyperdimensional vector spaces that map naturally onto neural network implementations. VSAs naturally support computation on discrete compositional data structures and a form of virtualisation that breaks the nexus between the items to be represented and the hardware that supports the representation. This means that computations on evolving data structures do not require physical rewiring of the implementing hardware. I illustrate this approach with a VSA system that finds isomorphisms between graphs and where different problems to be solved are represented by different initial states of the fixed hardware rather than by rewiring the hardware. Graph isomorphism is at the heart of the standard model of analogical reasoning and motivates this example, although that aspect is not explored in this talk. BIO. Ross Gayler has PhD in psychology from University of Queensland, with a long and far-reaching interest in the mysteries of cognitive computing. His day job is as a statistician in the finance industry.

In this time period, previous work on the construction of an oscillating neural network 'computer' that could recognize sequences of characters of a grammar was extended to employ selective 'attentional' control of synchronization to direct the flow of communication and computation within the architecture. This selective control of synchronization was used to solve a more difficult grammatical inference problem than we had previously attempted. Further performance improvement was demonstrated by the use of a temporal context hierarchy in the hidden and context units of the architecture. These form a temporal counting hierarchy which allows representations of the input variations to form at different temporal scales for learning sequences with with long temporal dependencies. We further explored the analog system identification capabilities of these systems where the output modules take on analog values. We were able to learn a mapping from the acoustic cepstral values of speech to articulatory parameters such as jaw and lip movement. This is a model speech processing problem which allows us to test the usefulness of our systems for speech recognition preprocessing.

Cloud services are among the technologies that are developing the fastest. Additionally, it is acknowledged that load balancing poses a major obstacle to reaching energy efficiency. Distributing the load among several resources in order to provide the best possible services is the main purpose of load balancing. The network's accessibility and dependability are increased through the usage of fault tolerance. An approach for hybrid deep learning (DL)-based load balancing is proposed in this paper. Tasks are first distributed in a round-robin fashion to every virtual machine (VM). When assessing whether a VM is overloaded or underloaded, the deep embedding cluster (DEC) also considers the central processing unit (CPU), bandwidth, memory, processing elements, and frequency scaling factors. For cloud load balancing, the tasks completed on the overloaded VM are assigned to the underloaded VM based on their value. To balance the load depending on many aspects like supply, demand, capacity, load, resource utilization, and fault tolerance, the deep Q recurrent neural network (DQRNN) is also suggested. Additionally, load, capacity, resource consumption, and success rate are used to evaluate the efficacy of this approach; optimum values of 0.147, 0.726, 0.527, and 0.895 are attained.

This article aims to achieve two goals: to show that probability is not the only way of dealing with uncertainty (and even more, that there are kinds of uncertainty which are for principled reasons not addressable with probabilistic means); and to provide evidence that logic-based methods can well support reasoning with uncertainty. For the latter claim, two paradigmatic examples are presented: Logic Programming with Kleene semantics for modelling reasoning from information in a discourse, to an interpretation of the state of affairs of the intended model, and a neural-symbolic implementation of Input/Output logic for dealing with uncertainty in dynamic normative contexts.

This work presents a new algorithm called evolutionary exploration of augmenting convolutional topologies (EXACT), which is capable of evolving the structure of convolutional neural networks (CNNs). EXACT is in part modeled after the neuroevolution of augmenting topologies (NEAT) algorithm, with notable exceptions to allow it to scale to large scale distributed computing environments and evolve networks with convolutional filters. In addition to multithreaded and MPI versions, EXACT has been implemented as part of a BOINC volunteer computing project, allowing large scale evolution. During a period of two months, over 4,500 volunteered computers on the Citizen Science Grid trained over 120,000 CNNs and evolved networks reaching 98.32% test data accuracy on the MNIST handwritten digits dataset. These results are even stronger as the backpropagation strategy used to train the CNNs was fairly rudimentary (ReLU units, L2 regularization and Nesterov momentum) and these were initial test runs done without refinement of the backpropagation hyperparameters. Further, the EXACT evolutionary strategy is independent of the method used to train the CNNs, so they could be further improved by advanced techniques like elastic distortions, pretraining and dropout. The evolved networks are also quite interesting, showing "organic" structures and significant differences from standard human designed architectures.

This work presents a new algorithm called evolutionary exploration of augmenting convolutional topologies (EXACT), which is capable of evolving the structure of convolutional neural networks (CNNs). EXACT is in part modeled after the neuroevolution of augmenting topologies (NEAT) algorithm, with notable exceptions to allow it to scale to large scale distributed computing environments and evolve networks with convolutional filters. In addition to multithreaded and MPI versions, EXACT has been implemented as part of a BOINC volunteer computing project, allowing large scale evolution. During a period of two months, over 4,500 volunteered computers on the Citizen Science Grid trained over 120,000 CNNs and evolved networks reaching 98.32% test data accuracy on the MNIST handwritten digits dataset. These results are even stronger as the backpropagation strategy used to train the CNNs was fairly rudimentary (ReLU units, L2 regularization and Nesterov momentum) and these were initial test runs done without refinement of the backpropagation hyperparameters. Further, the EXACT evolutionary strategy is independent of the method used to train the CNNs, so they could be further improved by advanced techniques like elastic distortions, pretraining and dropout. The evolved networks are also quite interesting, showing "organic" structures and significant differences from standard human designed architectures.

Granular computing is a computational paradigm that mimics human cognition in terms of grouping similar information together. Compatibility operators such as cardinality, orientation, density, and multidimensional length act on both in raw data and information granules which are formed from raw data providing a framework for human-like information processing where information granulation is intrinsic. Granular computing, as a computational concept, is not new, however itis only relatively recent when this concept has been formalised computationally via the use of Computational Intelligence methods such as Fuzzy Logic and Rough Sets. Neutrosophy is a unifying field in logics that extents the concept of fuzzy sets into a three-valued logic that uses an indeterminacy value, and it is the basis of neutrosophic logic, neutrosophic probability, neutrosophic statistics and interval valued neutrosophic theory. In this paper we present a new framework for creating Granular Computing Neural-Fuzzy modelling structures via the use of Neutrosophic Logic to address the issue of uncertainty during the data granulation process. The theoretical and computational aspects of the approach are presented and discussed in this paper, as well as a case study using real industrial data. The case study under investigation is the predictive modelling of the Charpy Toughness of heat-treated steel; a process that exhibits very high uncertainty in the measurements due to the thermomechanical complexity of the Charpy test itself. The results show that the proposed approach leads to more meaningful and simpler granular models, with a better generalisation performance as compared to other recent modelling attempts on the same data set.

In this paper, a key based soft computing transmission of intraoral gingivitis image has been proposed without the exchange of common key in between the nodes. Gingivitis has been a type of periodontal disease caused due to bacterial colonization inside the mouth, having the early signs of gum bleeding and inflammations in human beings. In E-health care strata, online transmission of such intraoral images with secured encryption technique is needed. Session key based neural soft computing transmission by the dentists has been proposed in this paper with an eye to preserve patients’ confidentiality factor. To resist the data distortion by the eavesdroppers while on the transmission path, secured transmission in a group of tree parity machines was carried out. Topologically same tree parity machines with equal seed values were used by all users of that specified group. A common session key synchronization method was applied in that group. Intraoral image has been encrypted to generate multiple secret shares. Multiple secrets were transmitted to individual nodes in that group. The original gingivitis image can only be reconstructed upon the merging of threshold number of shares. Regression statistics along with ANOVA analysis were carried out on the result set obtained from the proposed technique. The outcomes of such tests were satisfactory for acceptance. 

Models for visual attention, based on the representation of an attentional space as a two dimensional map have led to a model of visual attention which has been successfully used in the application of a space-variant active vision system, described below. Also, it has been demonstrated that stereo fusion limits, such as Panum's fusional area, scale in a manner which is determined by the size of a cortical hypercolumn, and the local value of cortical magnification factor. This in turn supports the notion that stereo disparity is computed by a local correlational operator defined on the span of a single pair of ocular dominance columns A generalized image warp technique has been developed, which we term the 'protocolumn algorithm', which provides image level models of the mapping of ocular dominance and orientation column systems at the level of primary visual cortex. Finally, many of the ideas developed in this project have reached fruition in the construction of a space-variant active vision system. An initial prototype system has been constructed under hardware support from DARPA, and a number of difficult algorithmic problems in motor control, attention, space-variant image processing, and space-variant pattern classification, have begun to studied.... Visual cortex, Vision, Pattern recognition, Active vision.

Models for visual attention, based on the representation of an attentional space as a two dimensional map have led to a model of visual attention which has been successfully used in the application of a space-variant active vision system, described below. Also, it has been demonstrated that stereo fusion limits, such as Panum's fusional area, scale in a manner which is determined by the size of a cortical hypercolumn, and the local value of cortical magnification factor. This in turn supports the notion that stereo disparity is computed by a local correlational operator defined on the span of a single pair of ocular dominance columns A generalized image warp technique has been developed, which we term the 'protocolumn algorithm', which provides image level models of the mapping of ocular dominance and orientation column systems at the level of primary visual cortex. Finally, many of the ideas developed in this project have reached fruition in the construction of a space-variant active vision system. An initial prototype system has been constructed under hardware support from DARPA, and a number of difficult algorithmic problems in motor control, attention, space-variant image processing, and space-variant pattern classification, have begun to studied.... Visual cortex, Vision, Pattern recognition, Active vision.

During the past year, we have completed two important steps in our program for understanding the biological and computational significance of patterns of spatial mapping in the brain. First, we have found a simple algorithm which is capable of describing and synthesizing the patterns of ocular dominance columns and orientation columns in the cat and monkey. This algorithm is controlled by a small number of parameters, and we show that it produces patterns which are simular to those in our lab, and elsewhere, obtained from animal experimentation. Moreover, we show that a number of previously published algorithms for simular purposes can be shown to be equivalent to our algorithm. The significance of this work is that we can now describe and synthesize some of the major architectural features of cat and monkey sensory cortex with high accuracy. In addition, we have obtained some insight into the essential simplicity of these patterns. This work is currently in press in Biological Cybernetics. In addition, we have developed an algorithm for pattern recognition based on the multiple, parallel two dimensional mapping of the input data. We view this as an important step in our goal of developing insight into the use of multiple, parallel sensory mappings in the brain. We believe that this algorithm is the first pattern recognition algorithm to make explicit use of the kind of data format which is characteristic of the brain.

During the past year, we have completed two important steps in our program for understanding the biological and computational significance of patterns of spatial mapping in the brain. First, we have found a simple algorithm which is capable of describing and synthesizing the patterns of ocular dominance columns and orientation columns in the cat and monkey. This algorithm is controlled by a small number of parameters, and we show that it produces patterns which are simular to those in our lab, and elsewhere, obtained from animal experimentation. Moreover, we show that a number of previously published algorithms for simular purposes can be shown to be equivalent to our algorithm. The significance of this work is that we can now describe and synthesize some of the major architectural features of cat and monkey sensory cortex with high accuracy. In addition, we have obtained some insight into the essential simplicity of these patterns. This work is currently in press in Biological Cybernetics. In addition, we have developed an algorithm for pattern recognition based on the multiple, parallel two dimensional mapping of the input data. We view this as an important step in our goal of developing insight into the use of multiple, parallel sensory mappings in the brain. We believe that this algorithm is the first pattern recognition algorithm to make explicit use of the kind of data format which is characteristic of the brain.

This project aims to explore how social influence affects individuals' prosocial decision-making under time pressure, as well as the computational and neural mechanisms underlying this process, through two studies.

This project aims to explore how social influence affects individuals' prosocial decision-making under time pressure, as well as the computational and neural mechanisms underlying this process, through two studies.

One of the defining properties of deep learning is that models are chosen to have many more parameters than available training data. In light of this capacity for overfitting, it is remarkable that simple algorithms like SGD reliably return solutions with low test error. One roadblock to explaining these phenomena in terms of implicit regularization, structural properties of the solution, and/or easiness of the data is that many learning bounds are quantitatively vacuous in this "deep learning" regime. In order to explain generalization, we need nonvacuous bounds. We return to an idea by Langford and Caruana (2001), who used PAC-Bayes bounds to compute nonvacuous numerical bounds on generalization error for stochastic two-layer two-hidden-unit neural networks via a sensitivity analysis. By optimizing the PAC-Bayes bound directly, we are able to extend their approach and obtain nonvacuous generalization bounds for deep stochastic neural network classifiers with millions of parameters trained on only tens of thousands of examples. We connect our findings to recent and old work on flat minima and MDL-based explanations of generalization.

Deep learning using convolutional neural networks (CNNs) is quickly becoming the state-of-the-art for challenging computer vision applications. However, deep learning's power consumption and bandwidth requirements currently limit its application in embedded and mobile systems with tight energy budgets. In this paper, we explore the energy savings of optically computing the first layer of CNNs. To do so, we utilize bio-inspired Angle Sensitive Pixels (ASPs), custom CMOS diffractive image sensors which act similar to Gabor filter banks in the V1 layer of the human visual cortex. ASPs replace both image sensing and the first layer of a conventional CNN by directly performing optical edge filtering, saving sensing energy, data bandwidth, and CNN FLOPS to compute. Our experimental results (both on synthetic data and a hardware prototype) for a variety of vision tasks such as digit recognition, object recognition, and face identification demonstrate using ASPs while achieving similar performance compared to traditional deep learning pipelines.

With recent advancing of Internet of Things (IoTs), it becomes very attractive to implement the deep convolutional neural networks (DCNNs) onto embedded/portable systems. Presently, executing the software-based DCNNs requires high-performance server clusters in practice, restricting their widespread deployment on the mobile devices. To overcome this issue, considerable research efforts have been conducted in the context of developing highly-parallel and specific DCNN hardware, utilizing GPGPUs, FPGAs, and ASICs. Stochastic Computing (SC), which uses bit-stream to represent a number within [-1, 1] by counting the number of ones in the bit-stream, has a high potential for implementing DCNNs with high scalability and ultra-low hardware footprint. Since multiplications and additions can be calculated using AND gates and multiplexers in SC, significant reductions in power/energy and hardware footprint can be achieved compared to the conventional binary arithmetic implementations. The tremendous savings in power (energy) and hardware resources bring about immense design space for enhancing scalability and robustness for hardware DCNNs. This paper presents the first comprehensive design and optimization framework of SC-based DCNNs (SC-DCNNs). We first present the optimal designs of function blocks that perform the basic operations, i.e., inner product, pooling, and activation function. Then we propose the optimal design of four types of combinations of basic function blocks, named feature extraction blocks, which are in charge of extracting features from input feature maps. Besides, weight storage methods are investigated to reduce the area and power/energy consumption for storing weights. Finally, the whole SC-DCNN implementation is optimized, with feature extraction blocks carefully selected, to minimize area and power/energy consumption while maintaining a high network accuracy level.

Experiment 1: The Impact of Action Information on the Accumulation Speed and Threshold of Badminton Decision-Making: Based on Behavioral Models Purpose: 1. To construct a drift diffusion model to clarify the differences in the speed of evidence accumulation and thresholds between experts and novices. 2. To determine whether the greatest difference in the processing of action information occurs 50ms before the stroke between experts and novices. Method: A mixed design with two factors: 2 (Sports Experience: Expert vs. Novice) * 3 (Information Quantity: 950ms vs. 1000ms vs. 1050ms). Experiment 2: The Influence of Sports Experience on the Accumulation Speed and Threshold of Badminton Decision-Making 1. To clarify indicators related to the speed of information accumulation and thresholds in the decision-making process. 2. To integrate electroencephalogram (EEG) indicators into a neural computational model of action information processing. Method: A single-factor between-subjects design: 2 (Sports Experience: Expert vs. Novice). Neural modeling. Experiment 3: The Impact of Prior Information Consistency on Badminton Decision-Making Purpose: 1. To clarify how static situational information (prior information) facilitates decision-making by accelerating the speed of evidence accumulation and reducing the decision-making threshold. Method: A mixed design with two factors: 2 (Sports Experience: Expert vs. Novice) * 2 (Consistency: Consistent vs. Inconsistent vs. Neutral). Experiment 4: The Influence of Prior Information on Badminton Decision-Making: The Moderating Role of Time Pressure Purpose: 1. To clarify the moderating role of dynamic situational information (time pressure) in the influence of prior information on badminton decision-making. Method: A two-factor within-subjects design: 2 (Consistency: Consistent vs. Inconsistent) * 2 (Time Pressure: Long Response Window vs. Short Response Window).

In this paper we use artificial neural networks to predict and help compute the values of certain knot invariants. In particular, we show that neural networks are able to predict when a knot is quasipositive with a high degree of accuracy. Given a knot with unknown quasipositivity we use these predictions to identify braid representatives that are likely to be quasipositive, which we then subject to further testing to verify. Using these techniques we identify 84 new quasipositive 11 and 12-crossing knots. Furthermore, we show that neural networks are also able to predict and help compute the slice genus and Ozsv\'{a}th-Szab\'{o} $\tau$-invariant of knots.

In this paper we use artificial neural networks to predict and help compute the values of certain knot invariants. In particular, we show that neural networks are able to predict when a knot is quasipositive with a high degree of accuracy. Given a knot with unknown quasipositivity we use these predictions to identify braid representatives that are likely to be quasipositive, which we then subject to further testing to verify. Using these techniques we identify 84 new quasipositive 11 and 12-crossing knots. Furthermore, we show that neural networks are also able to predict and help compute the slice genus and Ozsv\'{a}th-Szab\'{o} $\tau$-invariant of knots.

In this paper we use artificial neural networks to predict and help compute the values of certain knot invariants. In particular, we show that neural networks are able to predict when a knot is quasipositive with a high degree of accuracy. Given a knot with unknown quasipositivity we use these predictions to identify braid representatives that are likely to be quasipositive, which we then subject to further testing to verify. Using these techniques we identify 84 new quasipositive 11 and 12-crossing knots. Furthermore, we show that neural networks are also able to predict and help compute the slice genus and Ozsv\'{a}th-Szab\'{o} $\tau$-invariant of knots.

The hardware implementation of deep neural networks (DNNs) has recently received tremendous attention: many applications in fact require high-speed operations that suit a hardware implementation. However, numerous elements and complex interconnections are usually required, leading to a large area occupation and copious power consumption. Stochastic computing has shown promising results for low-power area-efficient hardware implementations, even though existing stochastic algorithms require long streams that cause long latencies. In this paper, we propose an integer form of stochastic computation and introduce some elementary circuits. We then propose an efficient implementation of a DNN based on integral stochastic computing. The proposed architecture has been implemented on a Virtex7 FPGA, resulting in 45% and 62% average reductions in area and latency compared to the best reported architecture in literature. We also synthesize the circuits in a 65 nm CMOS technology and we show that the proposed integral stochastic architecture results in up to 21% reduction in energy consumption compared to the binary radix implementation at the same misclassification rate. Due to fault-tolerant nature of stochastic architectures, we also consider a quasi-synchronous implementation which yields 33% reduction in energy consumption w.r.t. the binary radix implementation without any compromise on performance.

Associative memories are one of the most interesting applications of neural networks. In general, an associative memory stores a set of information, called memories. The information is stored in a format such that when an external stimulus is presented into the system, the system evolves to a stable state that is closest to the input data. We can view this process as a content- addressable memory since the stored memory is retrieved by the contents of the input and not by the specific address. In other words, the memory can recognize distorted inputs as long as the input provides sufficient information. Later in this report we will show the characteristics of the associative memory by presenting distorted versions of the stored images, e.g., rotated, scaled, shifted ones, etc. to the system and see how it converges.

Associative memories are one of the most interesting applications of neural networks. In general, an associative memory stores a set of information, called memories. The information is stored in a format such that when an external stimulus is presented into the system, the system evolves to a stable state that is closest to the input data. We can view this process as a content- addressable memory since the stored memory is retrieved by the contents of the input and not by the specific address. In other words, the memory can recognize distorted inputs as long as the input provides sufficient information. Later in this report we will show the characteristics of the associative memory by presenting distorted versions of the stored images, e.g., rotated, scaled, shifted ones, etc. to the system and see how it converges.

We present a construction, rendered in Quipper, of a quantum algorithm which probabilistically computes a classical function from n bits to n bits. The construction is intended to be of interest primarily for the features of Quipper it highlights. However, intrigued by the utility of quantum information processing in the context of neural networks, we present the algorithm as a simplest example of a particular quantum neural network which we first define. As the definition is inspired by recent work of Fisher concerning possible quantum substrates to cognition, we precede it with a short description of that work.