Simulation of biological neural nets

The strength of interconnections in a network of neurons determines how the network will respond as a whole to a particular input; "the pattern of connection strengths represents what the network knows." (Ferry, 1987, p55) The connections are bidirectional allowing for feedback circuits.

It now seems generally accepted that the brain's power arises spontaneously from huge numbers of neurons highly interconnected and processing information in parallel. These neuronal assemblies are defined as groups of 'neurons that are simultaneously active in response to a certain input'. But it is incredibly difficult to study these assemblies physiologically, getting enough electrodes into a small enough space to study enough neurons is currently out of the question, so finding an assembly and then showing its synchronised operation and its spread is incredibly difficult.

The neural net approach developed by McCulloch and Pitts was a hardware approach or was carried out on paper in mathematical and logical procedures. For a long time this work received very little attention and only in the last 15 years has a simulation approach emerged which models in the computer the interconnections and the interactions and simulates the activity of the nerve 'assembly' or the neural network. Given the massive numbers of synapses onto it that any one nerve might have, some of which are excitatory and some of which are inhibitory, and given the modulations of neurotransmitters across those synapses, the triggering of that nerve depends on the summation of all those inputs exceeding the threshold of that nerve. A cell's response might be a graded response according to the strength of the overall inputs or it might be an all-or-nothing response based on a threshold. All the outputs might be feedforward type processes or some might feedback into layers of cells preceeding the layer of the cell being considered, thus controlling its behaviour.

These systems of simulated neural nets can exhibit learning when based on D.O.Hebb's rule for learning developed in 1949, such that:

"the connections between cells that are active at the same time will be strengthened, increasing the probability that the first cell will excite the second cell in the future. Connections between cells whose activity is not synchronised will be weakened. Synchronised patterns of firing that occur repeatedly will eventually become stable representations (or memories) of the inputs that give rise to them, and can be reactivated by only partial inputs." [Ferry, 1987, p56].

Other aspects of brain function modelling being explored include how can one maximise the number of representations that a given network can hold? Obviously, maximising the number of connections, it may be that every neuron in the brain is only a very few neurons away from every other, (in the realm of 5 neurons distant). But then how do these overlapping assemblies operate without interfering with each other? Possibly some form of negative feedback is involved in preventing an active assembly from becoming to large.

In a neural network simulation system each neuron takes account of the weightings of the outputs of its upstream neighbours to determine its own state. Of course, it is not the neuron somehow deciding of itself that it should look at what its neighbours are doing, this connectivity is already in place and is, so to speak, woken up by use. Each neuron exists in a net of similar neurons and they are wired up, axon to dendrite, synaptically connected, forming self-programming processing subsystems. These networks are inherently robust - answering von Neumann's earlier call for reliability in artificial computing systems: if some neurons malfunction the overall function of the network is not affected.

Information is encoded in neural connections rather than separate memory elements, as unique patterns of interconnections. Also the system learns 'spontaneously' because it alters the strength of particular interconnections according to the repitition of use of those interconnections and the array of possible experiences provided and possible solutions arrived at, i.e. training. In computer terms it might be thought of as 'self-programming'.

This training process: the alteration of weightings on particular synapses in the network; can also be acheived by a recurrent or feedback network in which an error weighting is generated by comparing the actual output to the desired output and then fedback into the weightings of the synapses of the processing layer of neurons.

Summary

To rehash the neural net idea I want to quote from Tank and Hopfield's article in the December 1987 issue of Scientific American.

"A biological neuron receives information from other neurons through synaptic connections and passes on signals to as many as a thousand other neurons. The synapse, or connection between neurons, mediates the 'strength' with which a signal crosses from one neuron to another. Both the simplified biological model and the artificial network share a common mathematical formulation as a dynamical system - a system of several interacting parts whose state evolves continuously with time. Computational behaviour is a collective property that results from having many computing elements act on one another in a richly interconnected system. The overall progress of the computation is determined not by step-by-step instructions but by the rich structure of connections between computing devices. Instead of advancing and then restoring the computational path at discrete intervals, the circuit channels or focuses it in one continuous process." [Tank & Hopfield, 1987, pp62-63]

One might liken this activity to the human process of consensus decision making, where a problem is discussed until everyone involved knows enough about it for a decision to evolve from the range of opinions held by individual members of the group. The 'computational surface' of the nodes in a neural network shifts according to the weightings of each node. The weightings alter with training, i.e. through exposure to examples of the kinds of problems to be encountered by the particular network, and the solutions develop in the form of a kind of best-fit. "The network carries out the computation by following a trajectory down the computational surface. In the final configuration the circuit usually settles in the deepest valley" to find the best solution. [Tank & Hopfield, 1987, p67]. This approach is good for perceptual problems and for modelling associative memory. Using Hebbian synapses we can develop a model for learning: "synapses linking pairs of neurons that are simultaneously active become stronger, thereby reinforcing those pathways in the brain that are excited by specific experiences. As in our associative-memory model, this involves local instead of global changes in the connections.

Modelling real networks

Computational neuroscience is one of the major areas of investigation into what it is that brings about consciousness in what we know to be the extraordinarily complex but highly organised networks of neurons in the human brain. At the Tucson II conference Paul Churchland, of the University of California at San Diego, asked:

Churchland took us briefly through (a "cartoon version" of) the visual system.

A pattern of light "... a representation on your retina is transformed by going through a trillion little synapses into a new pattern at the LGN that is projected forward to V1 (the primary visual cortex), (where it is) transformed by another population of synaptic connections in to a third pattern. It rather looks like the basic mode of representation in the brain is in patterns of activation across populations of neurons, and the basic mode of computation is transfomation from one pattern to another pattern, to another pattern. Transformations which co-opt relevant kinds of information. Information that is relevant to its day-to-day behavior." [from Churchland's presentation at Tucson II]

This is very similar to the structure of a neural net (of course !, given that they were designed from actual neurons).

One of the primary problems being used in neural network development is that of face recognition, i.e. attaching a name to the face. Churchland presented work done by Garrison Cotrell's group at the University of California at San Diego, using a feedforward neural network havng 80 cells in the inner layer, which did a pretty good job of the basic face recognition task. He then mentioned some of the ways in which it failed and how one might deal with these failures using recurrent or feedback weighting of the connections in the network and discussed how this relates to some aspects of consciousness such as short-term memory.

Churchland's description of the face-recognition network: The Input layer is made up as 64 x 64 neurons (4096 neurons) consisting in photocells having a photograph of a face projected onto it ("being stimulated to an appropriate level of brightness"). The midddle layer consisted in eighty cells. The output layer had eight cells which give 8 bits to identify: face/non-face; male; female; "name" (5 bits); The network was trained up on about 11 faces and a small number of non-faces, with a number of examples of each face, and it did very well on the three kinds of distinguishing it had to do. When shown a test set of novel faces it did about 10-15% less well than it did on the learned set. Still a remarkable performance and not far down on our own sort of performance.

But this network cannot discriminate ambiguous images (like the duck-rabbit illusion). To paraphrase Churchland: What a feedforward neural network does is embody an input/output function, with a unique output for every different input. To achieve something like the handling of ambiguity we need something more than feedforward networks. So he introduces "recurrent pathways" which bring contextual information from the rest of the system of the brain and feed it back into the network. This allows the network to "modulate its own responses to perceptual input" These recurrent pathways are the channels for the feedback information which we have discussed above. For example there are a very large number of descending pathways from the visual cortex back to the LGN, more than there are projecting from the LGN up to the visual cortex.

Recurrent pathways were originally introduced into neural nets as a form of short-term memory. They also provide a level of directability and handling of amibiguity as well as answering some of the other desiderata for a theory of Consciousness. In the brain, the best candidate for a neural correlate of consciousness is the thalamo-cortical system which is a massive recurrent network centering on the thalamus (see Newman and Baars on the thalamo-cortical system)