Interacting
with your
TV?

by Stephen Jones.

When you stand in front of a painting there is an interaction, but only between yourself and the history of art. The object is completely static, you do all the work in the extraction of meaning. When you sit at home in front of the tv you don't have to do a thing, just let it wash over you. Even when an important topic is being presented the retention of the ideas for most viewers is of a very low level. Why? The lack a interaction, the lack of active participation means that information/meaning is not being extracted (the picture and sound Just wash over you), the brain is not being put into gear.

This is a highly political matter, this process of informing (the forming of ideas within) people, developing the structures for understanding what is going on around us. We will come back to this below.

Now, video is not television, though it may well take on many aspects of the appearance of television, and it is not in itself a simply visual art except in the sense that film or photography may be said to be. It stands independently from the traditional visual arts (painting, sculpture, etc.) and should be seen within the framework of its own characteristics, not viewed as some inadequate version of painting.

Video requires electro-technological support for its manifestation. lt is really only stored as instructions to a replay system about the image to manifest by modulating the electron beam scanning the monitor screen. It is a continuingly regenerated/repainted image, always in a condition of becoming. Philosophically it is this always becoming' condition which distinguishes video from painting. A painting is finished, thus in itself having being, whereas the video image is only ever becoming, fading away again to be replaced by the next frame.

This brings in another aspect of video, the temporal/durational. Most video work is a fixed series of images stored on tape, coming one after the other on to the screen, the result of some sort of compilation or editing process. The images are changing in time, the record, usually, of motion of some sort; new sequences following one after the other. Obviously, it is this temporal aspect which distinguishes video from painting or photography, and makes it closer to music as a form. Though it is only recently that the questions of musical attributes, such as rhythm, are being brought into video. For example, part of the basis of my editing and processing sequences in Severed Heads work is rhythm and counterpoint.

The frame is video's characteristic in common with painting, photography, film; but the role of the frame is also extended. In painting and photography the frame is a fixed surface (ie. 2D) boundary, whereas in film and in video the frame is also a boundary in time indicating the registration of one image on top of the previous continuously for the duration of the work. In film, the information contained in the frame is a concrete representation of the image, but in video the frame and the information are coded as instructions for the monitor to create the image directly onto its 'painting' surface, the phosphor target of the screen.

The instructions for generating video images are stored on magnetic tape, and in the current realm of the technology are analog (ie. continuously variable) encodings of the image information picked up by the camera. Video will soon assume the character of computer graphics by being encoded digitally. In this approach the screen area is divided into a grid of 'pixels' and each pixel is given values of red, green and blue light depending upon the image. Computer graphics are video's nearest relation, to the extent now that the camera is substantially displaced by the computer as a means of creating images.

So we have here some of the elements of the technological arts: Images in becoming, having duration and change over time, encoded and needing a technical decoding medium for their manifestation.

Now, video's historical parent is television, which usually could hardly be considered art, but is a highly political medium having the role these days of being the sole supplier of information to the average person. It is a one-way outpouring of predigested ideas and images of the world which 'informs' the viewer, but leaves them no means of denying or attacking the information other than in themselves. And of course if that was the only thing you'd ever seen/heard on the subject then there would be no basis for an opposed viewpoint. Where TV is the only source of information then the view of the world that is built up will be solely based on the view of the world painted by the producers of TV. There is no means of interacting with television, except occasional takeovers of newsrooms.

So the question of setting up means of interacting with the medium arises. When video first appeared it was touted as the solution par excellence for redressing this lack of capacity for interaction and feedback from the people to the media. Thus was Community Access media embraced. But it really didn't work because there was just nowhere to show the product of the process. The best that could be said was that some people learnt a bit more about how the media worked to control and direct our thinking. What we find in video art, and more particularly video installation, is an entirely different class of interactivity. Interaction in the sense of working (or playing) with the system rather than in the sense of debating the information flow.

Video art thus becomes a matter of exploring the things which render it different from the other forms with which it is linked. Obviously making pictures is primary, but they are pictures which move over time, are not necessarily generated with a camera, displayed on systems which may allow interaction, usually coupled with auditory information. What video art didn't do, despite the polemics of many artists who used it, was to actually come to grips with the problem of setting up means of interacting with the media. In fact the matter just did not arise in the kind of video art we are discussing.

We also have to deal with the extraction of meaning from the information presented. The personality is an active meaning extractor, using, along with the material being viewed, contextual data, the result of the history of the person and the linguistic world in which that person lives, ie. the consensus that we call the 'culture'. This process is an interaction, but in information terms not a conversation, that takes two active and responsive entities. It is one side of a conversation and as such is very much the relationship of the viewer to the artwork or to the TV as usually recognised.

It is one of the great possibilities of much time-based art that it can be engaged with in a more truly conversational sense if the presentation system has some means of sensing and responding to the activity of the viewer. This is the cybernetic view. Feeding back to the viewer some modification of the presented material, dependent upon the viewers actions in regard to this material.

There is another sense of the idea of interaction with the artwork and that is in the sense that the work is viewed as an instrument which can be played much like a musical instrument. Only, usually, the means of playing the instrument are removed from direct contact with, say, the keyboard, and made remote. The audience (or in performance, the performer) is removed from direct contact and the instrument senses where the person is and reacts in some, at least partially pre-programmed way.

Of course systems of hardware which sense the presence of a person in some particular area are fairly common now. Two major uses of this approach spring to mind. One is the standard burglar alarm which by interference in an ultrasonic or infra-red field in a room detects any movement in that room and triggers an alarm. The other is the basic door opening system used in most public buildings. This uses a little radar device to sense your proximity and open the door for you. The kind of information derived from these sensors is quite minimal, usually only 'switch the alarm on' or 'open the door'. For a really interactive system which engages with the audience, we need to derive much more information from the sensor. This usually comes in the form of a voltage difference when compared with some neutral condition, or, in fully computer-based systems, in the form of numbers relating to some differential established by the designer of the system. These voltages or numbers are then linked to the control of the output devices which form the presentation system.

This sort of interactive art has been around for almost two decades now. The original experiments used proximity sensors based on the inherent static electric charge carried by a person. A long wire or a metal plate was used as the capacitive element in an oscillator which was tuned to a neutral (non-oscillating) state, when no one was near the wire. Not always an easy thing to do. Then, when someone came near the wire their charge would place a different capacitance on the oscillator, de-tuning it and making it oscillate. This was the Theremin, and tended to make a range of wailing sounds the pitch of which depended on the relative positions and movements of the people interacting with the piece.

The next approach to detection was the placing of switches in strategic locations in floors and doors and especially daises so that people moving over the dais would trigger the switch and thus trigger some event to occur, eg. the starting of a motor. This approach, being essentially on or off, was elaborated for more interactivity by the use of pressure sensors. Again we see the need for a range of differentials which yields a greater range of changes possible in the system being controlled by the sensors.

In Australia this kind of work was being explored in the the old Central Street Gallery by Phillippa Cullen, a dancer, and a small group of electronics and music people, and on a somewhat grander scale by a similar group of people at Australia '75 in Canberra.

With the appearance of low cost microwave and ultrasonic doppler-shift radar detectors a new level of sensor was available which allowed a large range of position variability to be translated into voltages for use with voltage control synthesisers. Theoretically, you could place two of these devices (tuned to different frequencies so they didn't interfere with each other) on adjacent walls of a room at right angles to each other, so that the exact position of a person in the room could be tracked. I say theoretically, because, although I mapped out and published ideas for this kind of interactive system in 1979, [see my "Notes on the Cybernetics of Language and Video" self-published in 1979] I never managed to implement it and I have never seen anyone else implement anything similar.

The next step in sensor systems is the use of video cameras and scanning systems to analyse the ongoing condition of the video information and report on any changes in brightness levels in the camera field. It is in this approach that the first successfully controllable interactive video and musical installations have appeared. For example Warren Burt, a Melbourne composer, has been using this in working with a dancer, setting up the performance space so that the dancer's movements in that space allow her to virtually compose the music as she goes. Severed Heads are using the same system to allow audience interaction with the work.

In 'Chasing Skirt', our work in the Biennale '88, we have created a system where both the visual and the musical elements are a function of interactive behaviour. The sensing system in our work and in Warren Burt's work is the Perceptive Systems Pty Ltd '3DIS' system. Three-Dimensional Interactive Stage. This is a computer-based system which uses a camera to view the audience area. A number of regions in the camera field are demarcated so that the computer knows what parts to look at for changes. The system is initialised with no one in the camera field. Then when running, if a person enters any of the demarcated areas in the viewing field the relative brightness differential is detected and used to determine a particular change in the music or the video-mix. The computer is programmed to patch a particular change in musical instrument control information (ie. MIDI) to the music synthesisers, or a particular configuration of video signals to the video mixer and thence to the monitors.

The result in 'Chasing Skirt' is a continually changing version of the video and musical ideas established by Tom Ellard and myself, presented to the audience in a manner controlled by the audience. So, as the audience plays with the system more and more they can gain some kind of active control over how the music sounds or how the video mix is working.

It is really in the capacity for these systems to allow the audience or the performer to interact with the medium that we find a new level of art experience. As Tom Ellard has said in the Biennale '88 catalog entry:

. "...the idea is to build a system that is an audience to its audience. It will observe the activities of the people observing its activities. While you are entitled to your opinion of the worth of the work, remember that the work is entitled to its opinion of you. You might be entertaining but are you art?"

It is not so much a system being critical of itself, (an entirely different kind of feedback), as a system being opened up so that it is no longer simply a one way presentation of predigested TV pap.