Exchange Fields: Embodied Positioning as Interface Strategy

Bill Seaman 2001

This text is also available in its original format as a printer-friendly, PDF file. You must have the Adobe Acrobat Reader to view it. Click here to download it for free.


The work Exchange Fields was commissioned by the exhibition Vision Ruhr, held in Dortmund Germany last year. The work was a collaboration between the Dutch dancer/choreographer Regina van Berkel and myself. The programmer Gideon May also became involved in the project. The central question dealt with the generation of a new kind of interface - how might an embodied experience of interface be layered into the content of an interactive media/dance comprised of video, text, a sculptural installation and music? Exchange Fields sought to develop a novel interface strategy by eliciting culturally determined environmental 'behavior in relation to objects' as a grammar of gesture that could be used as input to the reacting system. The physical interface system functioned as an embodied intuition. The work sought to tap into pre-linguistic environmental knowledge related to the use of particular varieties of objects. A series of furniture/sculptures were developed. Each furniture/sculpture was designed with a unique implied "suggestion" of how the body might be positioned in relation to that object. This suggestion was non-logo-centric. It was embodied in the form of the physical interface itself and reinforced through linguistic captioning affixed near the work.

This interface design method is not intended to be a generalized input strategy. A dynamic relation is experienced by the participant that is brought about through their embodied physical positioning. This "gesture" functions as an input into a computerized system that dynamically links output consisting of pre-recorded performance/dance images (video), text and sound. These have been choreographed in relation to the particularity of that embodied position. For each unique furniture/sculpture a set of related dances was recorded. It is the physical engagement of the participant relative to the visual and audible output that gives the work its artistic experiential content and power.

This project does not seek universality in its application as an interface strategy but by its strategy of eliciting an 'action in relation' to evoke a universality of experience in interface (input) activity to system (output) activity. The work penetrates through some cultural barriers because it is pre-linguistic. It does not solely rely on textual language and symbolic systems but leverages kinesthetic knowledge that grounds the interpretations of the dancers movements that are observed in relation to the vuser's (viewer/user's - pronounced viewser's) personal experience. The furniture sculpture draws on the vuser's memory of past environmental/object relations as a layer of content. The poetic text functions as another layer of resonant content, pointing at the same complexities of human/machine relations that are made operative through the experiential nature of the work.

The highly skilled Dutch programmer Gideon May was called upon to work on the code and the physical interface connectivity. Initially, a video sensing system working with gesture recognition was considered. This use of a video camera as an input device to responsive, interactive systems was influenced by earlier work such as David Rokeby's Very Nervous System [1] as well as Myron Krueger's [2] explorations.

Cameras are extremely good at bringing certain kinds of data into a system. The sensate body as a model implies a multi-sensory approach to responsive environments. The ability to program computers to parse a series of differing sensual inputs meaningfully will become pivotal to future intelligent sensing systems working functionally in responsive environments. The difficult task is to author a programming environment that can provide the meaningful interpolation of the differing fields of input signals gleaned from the environment, as well as to provide the appropriate machinic responses back out into that environment based on the intelligent interpretation of those fields.

In this project video sensing implied a series of issues that needed to be addressed in the design of the system. Issues of design included the selection or designation of the gestures that were to be recognized by the system, the way in which these gestures were to enable seamless intuitive interaction, methods of communicating to the user the proper gestures to use, and the design of the space in which this could best be accomplished. To these were coupled technical problems such as the camera positions needed to respond to multiple participants and the ability on the part of the system to discriminate and respond to individual gestures in the context of multiple users and alternative movements.

Initially a multiple camera system was considered. By having two streams of data, one could position different participants in space. Yet the difficulty of addressing the issues noted above led to the abandonment of the use of the camera as an input device. Although computer-based sensing is a dynamic solution to many forms of interactivity, it is not the only solution. Similarly, developing a computer program that makes intelligent decisions is not the only means of embodying human intelligence in an environment.

Human intelligence embodied in the design of objects and environments can also enable intelligent responsiveness. As humans, we are constantly involved in processes of 'summing up' sensual perceptions of chosen environments. We draw on both the history of our sensual experience (learning/memory) and the real time embodied experience provided by our senses. Lacking the time to develop such a programming environment or a sensing machine (neither is a small task), I began to look for a simplified approach to the problem set.

Instead of training participants to gesture so that they might intelligently communicate with this responsive poetic system, I began to think that the physical environment itself might suggest particular gestures or body positions. How could I mold the environment to suggest particular positioning? How could I develop a series of physical interfaces that would be highly suggestive of their own use, substituting intuitive use for the notion of training the participant to use a system-related positioning language? Could this interface strategy also provide a layer in the artistic content of this participatory work. Instead of gesture recognition made by the machine, I embodied part of the intelligence of system, in the design of a series of physical interfaces. We learn through embodied experience, why not imply a series of embodied interactive experiences through the form of the interface itself? Here, the formal characteristics of interface suggest interactions with the environment. Certainly others have been motivated to design physical interfaces. Hiroshi Ishi and his Tangible Media Group in the Media Lab at MIT [3] have been researching environmental physical interfaces for some time. In terms of artworks, Jeffrey Shaw's Legible City [4] comes to mind, where a physical engagement with an exercise bike enables the navigation of a virtual environment. Jeffrey has also used furniture as an interface in his Virtual Museum, where movement in a chair was used for navigation. Detailed descriptions of Shaw's work can be found in the beautiful compendium of his work entitled . Jeffrey Shaw:A User's Manual: from Expanded Cinema to Virtual Reality.4

The focus of Exchange Fields is to develop a means of intelligent feedback that would enfold different layers of artistic content, with embodied experience. One of these layers is composed of text relating to human-machine interaction. The work thus functions as a meta-machinic assemblage (to borrow and extend the term machinic assemblage from Deleuse and Guattari) [5] In How We Became Post-Human [6] N. Katherine Hayles points at a fascinating set of perspectives related to cybernetic systems and the humanist subject. I began to think about the creation of different forms of furniture/sculpture as a means of extending embodied poetic interactivity. The language of 'poetic constraint' (pun intended) was environmentally extended into a series of suggestive interfaces. As this environment was being developed, potential layers of content were being considered. Another set of questions opened up:


1) How could the music be developed so that an emergent musical structure might arise through the use of the environment?

2) What technology should be used to enable the media-experience?

3) What kinds of aesthetic decisions inform this media-experience?

4) How might the environment function in an organism-line manner to achieve seamless intelligent responsiveness?

5) How should the dance be shot and edited so that the time-based modules could be called up in a seamless manner?

6) How could the furniture/sculpture function as an associative element of content while simultaneously suggesting their use?

7) How should the use of the furniture/sculpture 'handshake' with the computer to trigger appropriate code responses?

8) How should the image be shown to enable one to have a direct feedback experience, via images of the dance?

9) How should the dance be lit to enhance this experience?

10) What should the movement itself be?

11) How might the combinatorics of these modules function so that multiple users could use the system simultaneously?

12) How could the simultaneous use by a set of interactants function to generate another layer of content in the work?

13) How could the video be handled so that it might be able to be layered, or multiple images shown simultaneously?

The music for the work had three different streams: a linear spoken/sung text, a techno-ambient soundtrack, and modular musical drones that were triggered through interaction. These drones were initially performed by David Beaudry, UCLA artist and Ph.D. candidate in the department of music. Beaudry was instructed to play notes that arose out of silence and then returned to silence. They added an interactive tonal environment to the spoken/sung text and techno-ambient rhythms. The music was contemplative. Along with the music, the emotive nature of the dance, qualities of light, both recorded and environmental, additional video of energy related processes, and the suggestive shape of the furniture/sculpture, all functioned together as a set of evocative fields contributing to the overall emotional tone of the work. I was exploring what I call Recombinant Music as derived through participant interaction, merging with 30 minute cycles of continuously looped linear music played within the space of the exhibit. The use of each piece of furniture/sculpture, triggered both a segment of the dance and a related tonal audio layer.

As noted above, a series of problems surround the parsing of complex sense related data to facilitate computer interpretation. In Exchange Fields, I sought a clear solution. [7] In this work simple infra red beams were broken as participants interacted with the suggestive sculptural/furniture. A signal would be sent to the computer to play a particular segment of video and piece of music that was specifically tied to the chosen physical interface. The system accommodated simultaneous users by mixing layers of video together .

A series of modular segments of video, coming out of black and then returning back into black, were recorded for each furniture/sculpture. A year of discussion and collaboration between Seaman and Van Berkel led to the development of the dance and aesthetics of the work. The ZKM, the Center for Art and Media in Karlsruhe, Germany, provided both the environment for shooting and editing the dance. [8]

A high-contrast lighting scheme enabled Van Berkel to play with the edges of a central beam of light. Van Berkel moved in and out of this beam. This use of light both functioned as a unifying scheme for the combinatorial environment as well as a layer of aesthetic content. An intimate shooting strategy was used exploring extreme close-ups that heightened the potential relation between vuser and dancer.

As one explores the work, a unique, uncanny felling is generated through the pointed feedback mechanism of the video segments. As the body of the participant triggers dance segments directly related to the body, an oscillation between a media mirroring of behavior and the subtle and sometimes marked difference between dancer and participant is observed. This experience is sensually charged through the aesthetics of the image

and eroticism of the dance. The video footage explores the close observation of the body. The positioning of the participants also has sensual implications. The participant's body becomes slightly constrained when exploring the furniture/sculpture. This "poetic constraint" [9] helps to focus the participant in relation to their own lived physicality. The relative stillness of this constrained position is contrasted with the related movement of the dancer. Again, an uncanny paradoxical stillness/movement state is generated. It must be noted that even when the "wrong" position is taken by the vuser, a charged body/machine relation is still triggered by the system.

When multiple users become involved, up to four images of the dance are shown on the screen, mixed together in a layered video space. The text concerns the circulation of energy states (see below) between human and machine. As the images become layered through the exploration of multiple users, a tertiary relation between those multiple participants is also generated. The once precise feedback image of the dancer becomes a mixed field of pure energy and movement. Differing states of human/machine exchanges are thus explored. Along with the dance, two side screens provide very slowly moving images of different energy-related processes such as the lighting of a flame or the dispersion of smoke. These images qualify the dance and also help to set the emotional tone of the work.

The key to the construction of intelligent environments is an ability to think across a series of different domains related to form, content, software, hardware, interface and environment. These domains can be seen to function in a time-based continuum as potentially inter-operative self-organizing, organic constructions. Thus the techno/biological paradigm of a new, organically responsive design paradigm comes to the fore. The seeds of this goes back to the 60s with Gordon Pask's writing about the Architectural Relevance of Cybernetics [10], and Ascott (see below) in terms of cybernetic art forms. Krueger [11] states that Pask was important because he "posited that the domain of architectural design was not the determination of the form of the building but the structuring of the social context in which humans interacted with their environment and with each other." Roy Ascott early on saw the potentials of cybernetic behavioural relations in terms of works of art. In the paper, Behaviourist Art and the Cybernetic Vision, Ascott [12] presented the following concept:

Behaviourist Art constitutes, as we have seen, a retroactive process of human involvement, in which the artefact functions as both matrix and catalyst. As matrix, it is the substance between two sets of behaviours; it neither exists for itself nor by itself. As a catalyst, it triggers changes in the spectator's total behaviour. Its structure must be adaptive implicitly or physically, to accommodate the spectator's responses, in order that the creative evolution of form and idea may take place. The basic principle is feedback. The system Artefact/Observer furnishes its own controlling energy; a function of an output variable (observer response) is to act as an input variable, which introduces more variety into the system and leads to more variety in the output (observer's experience). This rich interplay derives from what is a self-organising in which there are two controlling factors; one, the spectator is a self-organising subsystem; the other, the art work is not usually at present homeostatic...

There is no prior reason why the artefact should not be a self-organising system; an organism, as it were, which derives its initial programme or code from the artists creative activity and then evolves in specific artistic identity and function in response to the environment which it encounters.

As computer-based systems and technological sensory extensions change our relation to nature, our constructed environment and language, we need to create mechanisms that function at the highest possible level of human/machine interaction, to best reflect upon this complicated rhizome of emergent relations.

Each object in Exchange Fields suggests the relative positioning of the participant. There is a cybernetic relation between object, participant, and media. I am not trying to condition the participant to act in a certain way in an effort to control their behavior. I am seeking to make an associative environmental form of art that elicits particular behaviors that function within a spatial range. These objects are suggestive of their use and thus catalytic of certain ranges of behavior through the active association of the interactant. This relation between viewer/user, object and media, has the multiple functions of being suggestive of participatory behavior as well as being conceptually associative, adding a layer of content into the work. Because the text in the work is about human/machine relations, the content becomes meta-behavioral and meta-positional. Each furniture/sculpture presents a field of potential readings and behavioral evocations. When the user sees that the image they trigger is related to a specific part of the body, the system becomes self-reinforcing. This becomes a positive, intuitive, embodied approach to interactivity. While there is a fine line between eliciting behavior and controlling behavior, the forms I have created are embodied intuitions about each furniture/sculpture's potential use. For example, if one approaches a door and sees a handle, one reaches out to use the handle to open the door. I do not see this as being controlling, in fact it is directly supported by the history of related experiences. I seek to specifically engage associative processes as a layer of content.

In the acquisition of language, we come to understand a long history of embodied relations with differing objects and situations as they are encountered as environmental contexts. We also have a history of behavioral relations with environmental contexts through a form of spatial patterning. These patterns are derived from the personal experience of multiple uses of similar objects in a variety of situations. Each of the furniture/sculptures in this work elicits a rage of associations for particular bodily relations. Simple titles were placed adjacent to the furniture/sculptures that also suggest behavior through textual language. Because particular behaviors are suggested, a simple infra-red beam could be placed in the most likely zone of use to be triggered by the likely behaviors. This beam could be broken during different forms of participation The nature of the objects suggests but does not require particular uses. This enabled one to 'misuse them', to trigger media through activity that is different from the anticipated interaction and still generate a relevant media-experience. The intelligence of the system does not rely on a computer's code-based reading of behavior (the gesture recognition approach). It instead relies on an intuitive understanding of the probable behavior of interactants as projected by the artist into the design of the computer interface. The intelligence of the designer is re-embodied in the system through the design of the objects, the simplicity of textual labels placed adjacent to the furniture/sculptures and the placement of simple sensors. The images triggered by use of the system as stated above, also reinforce the intuitions surrounding use. This implies that the intelligent design of the environment and the suggestiveness of objects and media encountered within that environment, can be as important as an intelligent program used to facilitate a range of cybernetic feedback relations.

As interfaces and interactive environments become increasing complex, instructions of a textual or visual nature can augment the intuitive associative suggestiveness of the interface. Sometimes it is even necessary to have a person help instruct in the use of a complex system. In some cases, the traditional idea of an interface is entirely removed in works that are triggered by voice or gesture recognition. Here the question surrounds the development of a vocabulary of words or gestures that can enable intelligent feedback by the system to a wide variety of input signals. The salient question becomes how can we minimize the need for additional instructional information? In Exchange Fields, the suggestive nature of the furniture/sculpture pointed toward the specific positional body-language needed to trigger relevant media-responses from the system.

New forms of interface may be invisible, physical, virtual (image based) or some combination of the above. As the complexity of these systems is increased, often the time that it takes for a user to learn how to use the system also increases. The potential of the system, its ability to meaningfully engage the participant, is often directly related to its complexity. One endeavors to author systems that enable the vuser to access resonant content in the most intuitive, engaging manner. The overarching question here is whether the experience of the system is worth the investment of time required to learn its use. I must admit that I have opted in the past for more complex interfaces to empower the participant and to increase their level of interactivity. In my work with the programmer Gideon May, The World Generator / The Engine of Desire. [13] The user is empowered to construct complex virtual worlds in real time. For this work the interface has an unusual depth of complexity, yet the outcome it achieves, the construction of virtual worlds in real time, is worth the investment of time it takes to learn how to use the system. In contrast, Exchange Fields sought to minimize the user learning curve and enable multiple participants to access the work simultaneously. It explores related but quite different subject matter. When I think of the works of art that have most engaged me over time, they have almost always included some aspect of my personal investment of energy. I am perhaps authoring for an ideal user that enjoys an extended level of engagement.

Interface design is context dependent. Artists also author for different contexts. Works for the internet must have a different interface strategy than works that can explore physical interfaces other than the mouse and keyboard. For example, when living in Australia, it didn't make sense for me to make works with a physical interface. I needed to make engaging works which could easily be transported and stored. Emphasis was put on the visual interface and readymade interface solutions were employed. This context dependence suggests that the particular interface developed for Exchange Fields is not to be implemented in other contexts, however, the strategy used and the realizations developed may have a wider range of interest.

The potential for the construction of intelligent environments involves the inter-functionality of participant, hardware, software, media and environment. The intelligence of the system is not derived by any one of these elements but arises through their inter-functionality. New approaches can be taken to each of the research domains that inform poetic cybernetic systems. One potential is to develop software tool-kits to aid in the construction of future intelligent environments involving sensing and its relation to differing human behaviors and language recognition. Tool-kits related to media-behaviors as triggered in response to environmental sensing are also a key area of research. Because the internet provides for vast distributed environments, the very definition of environment needs to be considered in relation to distributed access. Along with software, the evocative nature of physical interface construction can also become an active component in the suggestive functionality of an interactive environment. Physical space functions in a continuum with the media environment in which the vuser is intimately implicated through a circulation of energy states. The authorship of the media component, be it video, virtual space or some other yet to be defined form, also potentially becomes inter-functional in intelligent environments. The content of this media adds to the evocative resonance of the integrated environment of interaction. Any component can be used to layer in additional content into the experience until the entire system functions in a self-organizing, organic manner.

[1] David Rokeby(2000: http://www.interlog.com/~drokeby/home.html)3/9/01

[2] Myron Krueger: see Sylvia M. MŸller for aec (1996: http://www.aec.at/prix/kunstler/Emkrueger.html) 3/9/01
Hiroshi Ishi and his Tangible Media Group in the Media Lab at MIT,

[3] Tangible Media Group(2000: http://tangible.media.mit.edu/) 3/9/01

[4] 1997. Jeffrey Shaw _ A User's Manual: from Expanded Cinema to Virtual Reality. Ostfildern-Ruit: Cantz Verlag.

[5] DELEUZE, G. and GUATTARI, F. 1987. A Thousand Plateaus: Capitalism and Schizophrenia. vol.2. Trans. by Brian Massumi. Minneapolis: University of Minnesota Press. P.145

[6] Hayles, N. K. (1999) How We Became Post Human. Chicago and London: University of chicago Press.

[7] In the long run more complex environmental feedback relations must be researched. Ted Krueger in his paper "Interfaces to Non-Symbolic Media" discusses the following problem set in terms of research into intelligent architecture; he states: 'Biological material systems have many material properties that are inherent to their nature that may also be desirable in synthetic systems. These include [among other foci] self reproduction, self-repair, learning, interactivity, and autonomous diagnosis.' (Krueger, for publication in 2001) Certainly many of these ambitious foci could also be applied to symbolic media, as well as to systems that integrate symbolic and non-symbolic media. Krueger, T. (to be published in 2001) Interfaces to Non-symbolic Media. Interfaces et sensorialite, Louise Poissant, Editor, Groupe de recherche en arts mediatiquesPresses de l'Universite du Quebec

[8] Special thanks goes to Jeffrey Shaw who facilitated this aspect of production.

[9] For further reading about combinatorics and "Poetic constraints" see MATHEWS, H. AND BROTCHIE, A. 1998. Oulipo Compendium. London: Atlas Press. See also MOTTE, W. 1998. Oulipo, A Primer of Potential Literature. Normal, Illinois: Dalkey Press

[10] Pask, G. 1969. "The Architectural Relevance of Cybernetics". Architectural Design, September 1969

[11] Krueger, T. 2000. The Architecture of Extreme Environments. "Space Architecture" issue of Architectural Design. Dr. Rachel Armstrong, ed.

[12] ASCOTT, R. 1966. Behaviourist Art and the Cybernetic Vision. Cybernetica, International Association for Cybernetics, Namur, IX, pp.247-264.

[13] Information about The World Generator / The Engine of Desire can be found at the following sites: 235 Media (1997: http://www.235media.com/media_art/install/worls_generator.html) 3/9/01

See also i3net (1997: http://www.i3net.org/ser_pub/services/magazine/november97/seaman.html) 3/9/01

See Also : Temaceleste.com (1997: http://www.temaceleste.com/eng/showpicture.asp?sez=features&ID=232) 3/9/01

Bill Seaman, The World Generator / The Engine of Desire, 1997 (Photo by C. Hierholzer).

See also my thesis from The Centre For Advanced Inquiry into Interactive Art, University of Wales entitled: "Recombinant Poetics: Emergent Meaning as Examined and Explored Within a Specific Virtual Environment." 1999.

See also CaiiA(2000: http://caiia-star.newport.plymouth.ac.uk/) 3/9/01

Exchange Fields

Seaman 1999

thought becomes force, the circulatory fold of abstract physics

energy circulates becoming our exchange fields

the alchemical metaphor becomes code, the codes become enfolded

motion becomes thought, the flow of behavioral signals

light becomes information, response fields activated

informations becomes light, paths and returns

light becomes thought, trajectories and relays

thought becomes action, enfolded in response

thought becomes positional force, transference states, the body articulated

energy becomes action, circulation informs the fields

thought becomes energy, a return that inter-folds

proximity becomes a trigger, the physics of emotion dispersion

a body becomes entangled with an abstraction embodied

bodies become arousal, circulating emotives

the body becomes articulation, layered across the states

abstractions become one, the plural articulation

the drone lines becomes circulatory, angular musculature of the body-song

thought merges with the machinic flow, grasping for a balance

sexuality becomes a linguistics, the clock of abstract physics

dance becomes a return that folds, the abstraction of states of exchange

trajectories becomes gestural, reacting with precision

gesture becomes light flows, hand measures, trigger fields

motion becomes erotic, the touch field distances of abstract states

the boundary becomes a bridge, the physics of pulse edges circulating

a container becomes activated, and motivates the chain reaction across differing states

multiple vessels becomes engaged, inter-fields of energy

containers become saturated, held in suspension

a motion becomes ecstatic, a hovering balance of circulating abstractions

energy becomes transgressive enfolded in the circulation

the constraints becomes costly, no sliding off of meaning

touch becomes arousal, the tensile surface organization

emotion becomes confused, complex dimensions of release

vapor becomes a thought body, the woven arms inverted, inserted

breathing becomes energy, the voice of exchange

the body becomes a battery, the musculature of containment

a battery becomes a litany, alternate embodiment, a transference of the states

a process becomes a physicality, light flows into larger bodies, the summing of momentum

a word becomes a plurality, folding across the intervals

a plurality becomes a oneness, the plural musculatures of encoded articulations

a boundary becomes amorphous, this alternate perspective, the fine proximity

a site becomes the body cathartic, the body of information

mining becomes the metaphor of articulation, the exchange fields

a gesture becomes informed, grasping at the balance of abstractions intermingling

motion becomes poignant, grafting emotives, the life of emergence

an angle or positioning becomes a path to navigate, a circulation trigger

motion becomes you, emergent of flows