On Animation: The Illusion of Life

In the effort to understand living things the Greeks, particularly Aristotle, postulated three souls: the vegetal soul, the basis of plant life, which in animals and humans was to be found in the liver; the animal soul, seated in the heart and, coupled with the vegetal soul, the basis of life; and the rational soul, housed in the brain and that which distinguished humans from animals. 

Several spirits provided nourishment for and produced movement in these souls. It was the function of the lungs to gather pneuma from the air, whence it was delivered to the brain and mixed with the vital spirit from the heart to form 'animal spirit' for redistribution through hollow nerves to 'animate' the body. Galen wrote this theory up and his writings held sway until in the 17th century much of his ideas provided the framework within which new anatomical knowledge was interpreted. For example, Descartes wrote: 

"The cavities of the brain are central reservoirs ... animal spirits enter these cavities. They pass into the pores of its substance and from these pores into the nerves. The nerves may be compared to the tubes of a waterworks; breathing or other actions depend on the flow of animal spirits into the nerves. The rational soul takes place of the engineer, living in that part of of the reservoir that connects all of the various tubes. These spirits are like the wind. When they flow into a muscle they cause it to become stiff and harden, just as the air in a balloon makes it hard."

Descartes used an analogy based on the technologically most advanced concepts of the day. Within the world of public entertainment at the time the latest rage were the "automata", animated mechanical models which emulated the activity of people and animals by hydraulic means. And it was this mechanical "animation" which gave the appearance of life. 

Descartes split mind and consciousness from the body because he could not rely on the information of the senses and his devotion to the Roman church would not allow him to reduce his soul (or mind) to a material thing, a 'res existans'. The soul was connected into the body through the pineal gland, a singular organ at the base of the brain. Singular in the sense that unlike the rest of the organs of the brain it is not twinned. 

As regards sensations, in this case those of the eye, Descartes suggested that all the nerves of the eye went to the pineal body at the base of the brain to be relayed to the soul. But he denies that what we see are tiny pictures formed within our head. 

"(For) we have to consider that thought may be induced by many things besides pictures - e.g. by signs and words, which in no way represent the things signified." [Dioptrics, Discourse IV].

The neural cells of the retina consist in rods and cones overlayed by a middle layer of interneurons and an outer layer of ganglion cells whose axons make up the fibres of the optic nerve. The rods detect changes in brightness and the cones detect different wavelengths of light, i.e. colour. The concentration of cones to rods is much greater at the fovea (which is the centre of focus) and decreases towards the edges of the retinal field. The rods are able to detect down to the single photon (or packet of light energy). 

Once an array of retinal cells has been stimulated by photons, those neural cells so stimulated propagate an 'action potential', as the signal is called, to the layers of interneurons which then feed to the retinal ganglion cells. Some of these interneurons do motion detection using the propagation delays through the nerve-cell axons. The ganglion cells then gather up all the light pattern information processed in the retina and send it through the optic nerve to the visual cortex. In the optic nerve right and left sides of the retinal data are mapped together such that the right side of both retinas is sent, via the lateral geniculate nuclei deep in the centre of the brain to the left visual cortex and the left side of both retinas is sent (also via the lateral geniculate nuclei) to the right visual cortex. In the lateral geniculate nuclei the processing pathways separate into a colour sensitive pathway and a brightness contrast sensitive pathway. 

In the visual cortex three kinds of processing seem to take place, each in several stages. Certain cells are selective for colour or brightness. Other cells are selective for orientation. And in other cells various aspects of feature detection take place. Starting with low level edge detection performed by cells sensitive to bright-to-dark transitions of particular orientations, these are then integrated into more complex features through connections into further layers of cells. So we have here a hierarchy of detected features. 

Projections from the visual cortex also go to medial temporal regions which appear to be involved in detection of movement and in stereopsis (which is one aspect of depth perception) possibly correlating or integrating that information with other information about movement derived from other sensory pathways. 

The brain cannot be carrying images as representation but can only be carrying coded data about images for analysis by the various detection and recognition processors in it. This is an ongoing processing, never finished, always becoming, acquiring more examples, the current frame overlaying the previous as they process through the brain via different pathways experiencing different propagation delays providing the means for comparison, the current frame with the previous. In this we find various tie-ins with memory processing. 

Of course the brain doesn't deal with coded image data in frames of film or video as we talk about it, unless it is being presented with film or video. Nevertheless, at the finest level, the quantum mechanical, all things which might cause sensation come in discrete packets, or quanta, so that we might say that at the finest level the universe is digital. It is the processes of sensation that give the illusion of an analogue world. It takes time for things to happen in the brain, so the differences in propagation delay through different processing pathways smooth out the gaps between quantum events as sensed. 

All the pathways in the brain are massively interconnected, not just hierarchically as levels of integration, but also horizontally. It has been suggested that every neuron in the brain, all 10 billion or so, may only be half a dozen neurons away from every other neuron, in interconnection terms. The combination of highly organisd, highly complex processing systems and subsystems, with this massive interconnectedness is probably enough to in-and-of-itself generate consciousness. And even more especially so when coupled to the social/linguistic processes of the multiplicity of generative entities providing the apparent environment which each individual encounters. Within an individual system of sensations and their reverberations, consciousness is generated through the whorls of feedforward and the regulation of feedback, and within a social system, language and its reverberations as culture and so on all feedback into the apparent individual elements of the system intertwining them and interconnecting them, creating continuously a single unitary social fabric which directs, and is a consequence of, the becoming of consciousness. If we can only sense, can we know anything else? Language is the means solely of propagating sensations through groups allowing and governing the interpretation of sensation and shaping the consciousness. There can be nothing which cannot be talked or written about. There can only be that which is sensed and which might be talked or written about. 

Our knowledge of the world is entirely a knowledge of sensation, of the encoded signs of sensory data established at the sensor be it eye or ear or skin. The processing systems of the sensors and the brain, the feature detectors and the integrators, the comparators and the models for comparison all derive their capacities from the exercise of the sensory system in the first place and its moulding into sociality during the whole process of development and maturation. 

What I think is going on is something like this. All these highly organised, highly interconnected pathways of neural tissue form into networks which are then trained by the data flowing through them so that they become pathways of interpretation. All this data is patterned and as more and more patterned data flows through the networks they develop form around the information. The different pathways and splittings-off of this patterned data carry out particular processing transforms on the data, all the time preserving the relational consistency of the pattern. Data flows through various processing elements. In the visual system data is split off into colour information pathways, brightness and contrast information, edge orientation and depth detection pathways, and motion detection pathways using both positional and temporal differentiation. The resulting patterns having been analysed for these data are then sent to higher interpretive processing and image recognition. Somewhere along the way they are integrated with auditory and other sense data and the unitary conscious knowing of the world and ourselves ensues. 

All the organisation, interconnectedness, time-delayed reverberation and, probably, re-integration provide this continuous but always regenerating and generative system of perception and ideas which by its own process either is or generates consciousness. We are talking about highly complexly organised systems in continuous flux which show a centripetally organised structure feeding into itself almost as an oscillatory system which by its own activity may be said to go 'live'. Very many complexly interlinked feedback loops which put the system in oscillation (or on the very edge of chaos) which is then the liveness, the presence in the continual present, of consciousness. 

Indivduality is a function of our language and the framework of our culture and its in-forming of ourselves. It is an illusion because the sensation and in-form-ation forms part of the processor (software builds hardware, so to speak). Though it is regarded within our language system as being outside us it is really part of us. We are open systems without real boundaries between us and the world. A unitary entity of information and flesh, ecology and relations between things and the sensing in-formed thing (almost a processing element in the mind of Gaia, though gone mad with the psychosis of vastly conflicting desires) folding back onto itself generating controlled far-from-equilibrium fluid structures always in becoming. 

The system is generative partly because it is an open system and partly because it is so complexly interconnected with all sorts of levels of feedback relations, that we can't help but always see things in the light of things known and seen previously, and vice-versa, see what is current in the light of what is new. It is a natural function of the brain to be generative. Language is generative, creativity is a generative process. Animation is produced from generative processes both of concepts and visual manifestations. Culture is a construction made out of individuals, history and ideas.