/ ____ / / /\ / /-- /__\ /______/____ / \ ________________________________________________________________ Leonardo Electronic Almanac Volume 13, Number 11, November 2005 http://lea.mit.edu ________________________________________________________________ ISSN #1071-4391 ____________ | | | CONTENTS | |____________| ________________________________________________________________ INTRODUCTION ------------ EDITORIAL --------- FEATURES -------- < Virtual Theatre - One Step Beyond Machinima, by Joe Geigel > < Neil and Iona: A Case Study in Cybernetic Performance Art, by Jason Van Anden and Lauri Goldkind > < Patchwork in motion: A practice-led project investigating the shifting relationships and processes associated with the performing body in interactive and non-interactive visual environments, by Maria Adriana L. Verdaasdonk > < VR as a Performance for an Audience, by Paul Hertz > LEONARDO ABSTRACTS SERVICE (LABS) -------------------------------- Recent top-rated abstracts from the English-language LABS database: < It's in Your Eyes - Gaze Based Image Retrieval in Context > By Lasse Scherffig < Operating In Media Reality - For A Critical Practice in Interactive Images > By Samuel Bianchini < Improvisatory Music and Painting Interface > By Hugo Sol’s Garc’a < Doctor Photo: The Cultural Roots Between Portrait Photography as Medecine in Nineteenth Century America > By Tanya Sheehan LEONARDO REVIEWS ---------------- < Eating Architecture, reviewed by Stefaan Van Ryssen > < The Test Drive, reviewed by Michael R. Mosher > < Image Retrieval: Theory and Research, reviewed by Pia Tikka > LEONARDO -------- < Contents: Leonardo Vol. 39, No. 1 (2006) > LEONARDO NETWORK NEWS --------------------- < Common Systems: The Invisible Dynamics of the Pacific Rim and the Bay Area > < Design as Seeing as Thinking Workshop, Istanbul, Turkey > < swissnex Joins Leonardo Organizational Membership Program > BYTES ----- < Director, Institute for e-Learning and Research in the Arts and Design College of Arts and Architecture, The Pennsylvania State University > < Director, School of Film and Digital Media, University of Central Florida > < INTERNATIONAL CONFERENCE: New Constellations: Art, Science and Society > < NEW MEDIA/INTERMEDIA: Entry level Assistant Professor position, Purdue University > < artBridge: A Matrix of Inspiration > ________________________________________________________________ INTRODUCTION ________________________________________________________________ This month in LEA, guest editors Annette Barbier and Marla Schweppe look at multimedia performances through four refreshing and different essays that explore different aspects of the topic. We begin with Joe Geigel's *Virtual Theatre - One Step Beyond Machinima*, which introduces a technical framework for defining and performing a theatrical work in a virtual space. As proof of concept for this framework, a real time, distributed improvisation is showcased. In *Cybernetic Performance Art; The Trouble with Blurring the Distinction Between Art and Life*, Jason Van Anden and Lauri Goldkind look at developing technology to make artworks improvisationally simulate emotional behavior in real time and space, and discover how a boundary was crossed between the disciplines of static sculpture and live performance. Following that, we embark on a *Patchwork in motion: A practice-led project investigating the shifting relationships and processes associated with the performing body in interactive and non-interactive visual environments* with Maria Adriana L. Verdaasdonk. Finally, Paul Hertz deals with *VR as a Performance for an Audience*, which contemplates the possibility of creating VR performances in a traditional musical performance or theatrical situation, with an audience, as a hybrid or intermedia art form. Amnon Wolman's desktop performance unfolds differently, in real time, each time it is played. It addresses not only our ever-varying sense of time, but also the intimate space of the desktop in creating a unique, individualized performance for every listener. Accompanying the issue is a specially curated gallery. Jack Ox's networked performance proposes multiple points of entry as well as of reception. While creating a live, real-time event, she also incorporates static images, visualizations of musical sequences. Beno”t Maubrey incorporates sound and video "accompaniment" into the body of the moving performer. Christina Ray and Glowlab challenge our notion of performance by using cell phones to transmit the spectacle of everyday life observed. Bob Gluck invites viewers to enter into the use of ritual objects, which respond to and amplify their actions, making them participants in a dialog with the work. Pedro Rebelo creates musical "prostheses" which extend the acoustic into the electronic realm. Bob Ostertag examines the relationships between our bodies, our society's detritus, and the machines that interpret our actions, our computers. The special issue is available at: http://mitpress2.mit.edu/e-journals/LEA/archive.html and the gallery: http://mitpress2.mit.edu/e-journals/LEA/GALLERY/mmperf ________________________________________________________________ EDITORIAL ________________________________________________________________ by Annette Barbier and Marla Schweppe Annette Barbier Chair, Interactive Arts and Media Columbia College Chicago 624 S. Michigan Avenue Chicago, IL 60605 U.S.A. abarbier [@] colum [dot] edu Marla Schweppe Rochester Institute of Technology 73 Lomb Memorial Drive Rochester, NY 14623 U.S.A. Marla [dot] Schweppe [@] rit [dot] edu In the modern era, technology has made important contributions to, and it might be said even partnered with, live performance. One of the most memorable early innovators was Winsor McCay, better known as creator of the comic strip Little Nemo and the animated dinosaur Gertie than for stage performance. In addition to his better-known activities, McCay was also one of the "chalk talk" performers who entertained while drawing, and in fact moved this genre forward enormously by introducing animated film projected onto his sketchpad on stage. Modern audiences are more familiar with a version of the Gertie cartoon that incorporates a bookend story of a gentleman's bet made over dinner that McCay could bring a dinosaur to life. However, the meat of the story, McCay's interaction with Gertie, was actually performed live on stage, with McCay timing his performance to the film, and giving Gertie directions, feeding her, and chastising her. [1] Decades later, pioneer television producer Ernie Kovacs also interacted live with technology, incorporating it into his performance, which was in fact completely reliant upon various television techniques. Working without canned laughter or a studio audience, Kovacs created performances that could only be seen on TV, using, for example, a split screen to match the bottom half of his face with the top half of his guest's face in a hilarious interview. [2] In the realm of high art, perhaps the best-known early innovator is Merce Cunningham, whose 1965 work with John Cage, David Tudor, Stan Vanderbeek, and Billy Klźver (Variations V) used antennae and photocells to allow the movement of dancers to control tape recorders and live short wave radios. [3] This issue of the Leonardo Electronic Almanac looks at a wide array of contemporary works incorporating technology into performance, inheritors of the legacy of these early experiments. Like their predecessors, these works, through incorporating technology, challenge the limits of performance, even redefine it. Contributors examine work that ranges from the mapping of space to the examination of time, from the centrality of the human body to its replacement by the avatar or robotic/sculptural body. Some works unfold in real time, while others feature frozen moments; some happen on a network, others on the desktop; some embrace virtual space, others are firmly rooted in "real" space. Both papers and gallery statements in this issue represent some of the most innovative efforts by contemporary creator/performers in grappling with technology. LEA Multimedia Performance Special: http://mitpress2.mit.edu/e-journals/LEA/archive.html LEA Multimedia Performance Gallery: http://mitpress2.mit.edu/e-journals/LEA/GALLERY/mmperf/ _____________________________ REFERENCES 1. http://www.bpib.com/illustrat/mccay.htm 2. http://www.museum.tv/archives/etv/K/htmlK/kovaksernie/kovacsernie.htm 3. http://www.medienkunstnetz.de/works/variations-v/ AUTHOR BIOGRAPHIES ANNETTE BARBIER is an artist whose work began in sculpture and moved through video to new technologies including computer animation, virtual reality, and net art. Her work addresses home, domesticity, and the ways in which identity is bound with one's environment. It has moved from an emphasis on the personal to a consideration of the global, looking at ways in which the home has come to be defined more broadly as populations shift, and as our interdependence becomes increasingly clear. Barbier graduated from the School of the Art Institute of Chicago with an MFA. She dropped out of college to spend a year in France, which was formative in making issues of home, culture and identity central to her work. Years later, a Fulbright lectureship in India with her 3-year-old daughter confirmed the importance of travel in questioning one's conceptions about the world, and resulted in a travel diary tape. More recent work is growing from a profoundly moving trip to Vietnam in 2003. Barbier chairs the Interactive Arts and Media Department at Columbia College, Chicago, where she teaches interactive installation, animation, and new media. MARLA SCHWEPPE is a full professor in the Interactive Media Design and Imaging Department at the Rochester Institute of Technology. She serves as head of the Digital Studio and Director of Visualization. Early in her career she worked in theater, television and movies in New York City. She has been teaching computer graphics and animation for more than two decades. She developed the graduate and undergraduate programs in computer animation at RIT. She is currently developing curriculum in game art and design in coordination with a game programming degree. Her creative work includes the incorporation of her theatrical background into interactive digital performances and experiences, animatronics, explorations in physical interaction, and collaborative support for visualization and simulation of information and ideas. ________________________________________________________________ FEATURES ________________________________________________________________ VIRTUAL THEATER - ONE STEP BEYOND MACHINIMA by Joe Geigel Department of Computer Science Rochester Institute of Technology 102 Lomb Memorial Drive Rochester, NY 14623 U.S.A. jmg [@] cs [dot] rit [dot] edu KEYWORDS virtual theater, distributed performance, storytelling, gaming, machinima ABSTRACT In this paper, we introduce a technical framework for defining and performing a theatrical work in a virtual space. The system, built on a 3D gaming engine, is defined for real time, interactive performance using a metaphor of traditional theatrical mechanisms and processes. We also give the details of "What's the Buzz?", a real time, distributed improvisation created in the spring of 2004 as a proof of concept for our framework. _____________________________ 1.0 INTRODUCTION Imagine, if you would, a theatrical performance: The stage is set, the actors are acting and singing, the lights are lit and the audience is engaged and enjoying the show. Now envision each one of these participants to be in different parts of the world, simultaneously sharing the same theatrical performance over a computer network. This is the goal of a Virtual Theatre system: To enable actors, crew, and audience to share and participate in a single theatrical performance using a shared and distributed virtual space. The idea of performing theatrical works in a virtual setting is certainly not unprecedented. One such example, which inspired us, is VRML Dream, a distributed version of Shakespeare's A Midsummer Night's Dream, which was streamed over the Internet in April 1998 [1]. Other works, such as those performed by Trokia Ranch [2, 3] or that employ the Interactive Virtual Environment (IVE) developed at MIT [4], combine both physical and virtual stage components in a single work. In our work, we attempt to generalize the components and processes of theatrical work performed in virtual reality with the goal of creating a technical infrastructure for the realizations of such performances. In searching for underlying technologies for building such a system, one need look no farther than modern 3D gaming engines through which players interact, in real time, in a distributed 3D virtual space. Such an approach is currently being taken in the emerging field of Machinima [5]. Defined as animated filmmaking within a real-time virtual 3D game environment [6], these virtual reality engines are adapted as soundstages for shooting an animated cinematic work. It is our intention to shift the storytelling perspective from that of a filmmaker to that of a stage director and investigate means by which these systems can be adapted for use in a theatrical rather than a cinematic context. 2.0 CINEMATIC VS THEATRICAL STORYTELLING Perhaps motivated by the increasing similarities between computer-animated films and 3D games, the choice of using a gaming engine for cinematic purposes is a natural one. A gaming engine does provide instant access to a customizable 3D world, and the increasing widespread availability and flexibility of the technology does make their use for filmmaking an attractive option. The same can also be said for other forms of multimedia performance. In fact, if one compares the mechanisms of cinematic storytelling with that of, say, theatre, we discover that gaming engines may be that much more naturally applicable to the latter rather than the former. For example, we can compare theater and cinema along the dimensions of spontaneity, time structure, focus, and interactivity. Spontaneity - Theater is live whereas cinema is pre-recorded. Each performance of a theatrical work will differ from previous performances. In cinema, the presentation is fixed, captured in time. Time Structure - Theater is experienced in real time without the aid of editing. Not only is film edited, but the proper editing of the final piece is essential to the telling of the story. Focus - In both film and theater, the goal of the storyteller is to grab the focus of the audience. In film, this is done irrevocably; the audience cannot view any part of the scene that the filmmaker does not include. In theatre, the storyteller must earn this attention since the audience has more control over where he or she looks. In short, with theater, the performance is experienced from the point of view of the audience whereas in film, it is from the point of view of the filmmaker. Interactivity - In theater, as opposed to film, the audience takes an active rather than passive role in the entertainment. The performance of a story within a theater is not just live, but it is also interactive. The audience is present to witness the performance, and audience reaction drives the actors. In considering the above dimensions with respect to games, we note that games are live, real time, experienced from the point of view of the player and are designed to be extremely interactive. Since the requirements for games closely match those of theater, it is a natural choice to use the engines on which games are built as the foundation for a virtual theatre system. In addition, as an added advantage, many games are designed to be played in a distributed environment, with players resident in different physical locations. As a result, networking support comes as an integral capability when using a real-time gaming engine. 3.0 BUILDING A VIRTUAL THEATER In defining the architecture and interfaces for our virtual theater system, it was an important goal that users interact with the system using a familiar theatrical metaphor. This will allow users' focus to remain on the storytelling process using the mechanisms of theater and not be burdened with the details of any particular gaming system. In order to keep the details of the underlying game engine transparent to the user, the architecture of our Virtual Theatre system uses a set of layered interfaces. At the top level is the Virtual Theatre Layer. Users of the system utilize the interfaces defined in this layer in defining the actions and interactions of a given performance. The Virtual Theatre layer sits on top of the gaming engine, which provides basic access to, and control of, a 3D virtual world. The gaming engine interfaces with low-level graphics hardware and systems in order to provide real time performance (Figure 1). For our implementation [7], we built on Multi-user Programming Pedagogy for Enhancing Traditional Study (MUPPETS), a collaborative virtual environment (CVE) originally designed for enhancing student education in the areas of programming and problem solving [8,9]. The core MUPPETS system is written in C++ and includes a complete and integrated Java programming environment. Whereas the MUPPETS core performs object rendering in C++ for efficiency, control of objects within the virtual space is implemented by writing simple programs in Java. The Virtual Theatre layer is defined by a number of Java classes, redefining the basic functionality of the underlying MUPPETS system allowing control of the 3D world to be defined in a theatrical context. Major classes defined by the Virtual Theatre Layer include: * Actor - Has methods for describing position and orientation within virtual space as well as a means to react with the stage environment * Director - Directs the behavior of an actor or actors * Stage Manager - Has the ability to define, set, and trigger lighting and staging cues * Stage Elements - Objects controllable by the stage manager. Can include lights, sets, props, etc * Audience - Has the ability to set their own view as well as respond to the performance. These base classes are extended using inheritance to define specific objects, behaviors and methods for a given performance. Additional classes are implemented as wrappers to any peripheral devices used during the performance. 4.0 IMPLEMENTATION AND PROOF OF CONCEPT A prototype implementation of the system was built and used as a proof of concept for the Spring 2004 production of "What's the Buzz?" "What's the Buzz?" is an interactive, distributed improvisation concerning a flock of bees in search of nectar. The story is told though a series of vignettes. In each vignette, the flock encounters a unique flower that proceeds to thwart the flock's efforts in obtaining its goal. The piece was performed and viewed entirely within the virtual space provided by MUPPETS. 3D models and animation sequences were created using Maya and imported into the system. Individual Java classes were designed and implemented for computational control of each of the virtual participants involved in the performance (bees, the flock, the flowers, the stage manager, audience members) extending the base classes provided by the Virtual Theatre Layer of the system. Human command of the objects on the virtual stage was achieved via manipulation of software object instantiations corresponding to these virtual participants using external peripheral devices (as described below) as an interface. The performance involved the interactive interplay between an ensemble of human "actors" controlling performers and other elements of the piece on the virtual stage. In realizing the performance, we had a limited number of VR peripherals at our disposal including a one node, 6DOF motion tracker and a data glove. Additional interaction was performed using traditional devices (e.g. keyboard and mouse). The motion of the flock was determined by a basic behavioral model [10] with the swarm following the path of a lead bee. The model also included an emotive component that drove the motion of the bees to convey one of a set of predefined emotions [11]. Both the path of the lead bee and the emotion to be expressed by the flock were controlled by a human participant equipped with the motion tracker and the data glove. Flower behavior was achieved through a set of pre-scripted animations triggered by another human participant. Each audience member viewed the performance on his/her own monitor attached to an individual workstation. Audience interaction was performed using the mouse. Mouse motions were used to guide the view of the audience member, whereas the mouse buttons were used for applause and/or booing. As in real theatre, each active audience member was assigned his or her own separate seat location within the virtual theater. Finally, a human Stage Manager controlled staging and lighting by triggering predefined cues. Although the stage manager shared the same virtual stage as the other participants, he had the option of viewing the environment from different perspectives. The stage manager also had a simple GUI superimposed on the view of the scene used for triggering cues. 5.0 CONCLUSIONS AND FUTURE WORK Working within a theatrical context is a time proven means to tell a story. This paper introduced a framework for enabling theatrical storytelling in a virtual space. We feel that the framework successfully enabled translation of the theatrical components and processes from a physical setting to a virtual space. Extending the example of Machinima, we chose to use a gaming engine as the technical basis for our system. Since gaming systems rely on the same performance dimensions as theater, using a gaming system as the foundation for a virtual theater system is a natural choice and worked well. It was, however, particularly important to redefine the interface of the gaming engine to work within a theatrical paradigm. This allows users to work with the system with the perspective of putting on a theatrical performance as opposed to playing a game. The Virtual Theatre Project has been funded through 2006. We are planning for at least two new productions. In these future productions, we plan on enhancing and expanding the current framework, thus enabling more challenging productions: incorporating more complex VR devices (e.g. full-body motion capture) and introducing a physical stage, thus moving the performance to a mixed reality setting. _____________________________ REFERENCES 1. S.N. Matsuba and B. Roehl, *Bottom, Thou Art Translated: The Making of VRML Dream*, IEEE Computer Graphics and Applications, 19 (2), March/April 1999, 45-51. 2. http://www.troikaranch.org/ 3. K. Farley, "Digital Dance Theatre: The Marriage of Computers, Choreography and Techno/Human Reactivity", *Body, Space and Technology*, 3(1), 2002, (http://www.brunel.ac.uk/depts/pfa/bstjournal/3no1/journal3_1.htm) 4. F. Sparacino, G. Davenport and A. Pentland, "Media in performance: Interactive spaces for dance, theater, circus, and museum exhibits", *IBM Systems Journal*, 39(3-4), 2000, 479-510. 5. http://www.machinima.com/ 6. P. Marino, *3D Game-Based Filmmaking: The Art of Machinima*, Paraglyph Publishing, 2004. 7. J. Geigel and M. Schweppe, *Theatrical storytelling in a virtual space*, Proceedings of the 1st ACM workshop on story representation, mechanism and context, New York, NY, 2004, 39-46. 8. A. Phelps and D. Parks, *Fun and Games: Multi-Language Development*, ACM Queue, 1 (10), February 2004, 46-56. 9. http://muppets.rit.edu 10. C. W. Reynolds, "Flocks, Herds, and Schools: A Distributed Behavioral Model", in Computer Graphics, 21(4) (SIGGRAPH '87 Conference Proceedings) 25-34. 11. K. Law, *Emotion in Flocking*, Masters Project, Rochester Institute of Technology, Department of Computer Science, June 2004. AUTHOR BIOGRAPHY Joe Geigel is an Assistant Professor in the Computer Science Department of the Rochester Institute of Technology. His research interests include computer graphics, multimedia systems, and applications of virtual reality to theater. Geigel earned his doctorate in computer science from the George Washington University. He's an active member of the ACM, IEEE, and ACM/SIGGRAPH. _____________________________ NEIL AND IONA: A CASE STUDY IN CYBERNETIC PERFORMANCE ART by Jason Van Anden jason [@] smileproject [dot] com and Lauri Goldkind, LCSW lgoldkind [@] gmail [dot] com 149 Prospect Park SW, #5 Brooklyn, NY 11218 U.S.A. http://www.smileproject.com behavior, cybernetics, interactive, Multimedia Performance Art, psychology, robotics, sculpture, software ABSTRACT In the course of developing technology to make artworks improvisationally simulate emotional behavior in real time and space, a boundary was crossed between the disciplines of static sculpture and live performance. This paper documents the basis of the Smile Project's creation, the resulting new technology and the questions that have arisen for the artist in his pursuit of the project's creation. INTRODUCTION Neil and Iona are a pair of human-scale robotic sculptures who interact with each other and their audience "emotionally". Each is capable of manipulating the other's feelings, as well as those of their audience, through facial expressions, sound and movement. In the course of their creation, a line was crossed between sculpture and performance. What I originally understood to be an ensemble of sculptures had become an infinite series of performance pieces differentiated by the material they were provided. I am still grappling with the challenges this paradigm shift has produced. New media technologies have made it very easy for artists to mix disciplines once separated by well-developed boundaries. This article discusses the concepts that inspired the creation of Neil and Iona, documents the technologies designed to realize them and presents questions raised by their exhibition. CONCEPT I am a sculptor and professional software architect. The creation of living artworks has been an opportunity to mix two areas of my passion and expertise. The idea was sparked by something that I most likely misunderstood while listening to a radio interview. What I thought the guest had explained was "how our first understanding of love comes from recognizing our mother's smile." I have since been corrected, but this snippet of cognitive science inspired a lasting interest in the emotional exchanges between people. In 1998, I started to create a series of primitive drawings representing visual/emotional connections as seen from an infant's vantage. At the same time I was also attending weekly group therapy sessions. In therapy I started to interpret the repeated patterns of behavior taking place between my fellow participants and me as feedback loops. Feedback loops are a central idea across many disciplines. F. Heylighen, C. Joslyn and V. Turchin (2005) define feedback loops as the flow of information back to its origin in a system. Furthermore, a feedback loop is a circular causal process in which information leaving a system is returned as inputs to the system, possibly involving other systems in the loop. An example of how a feedback loop might impact a group therapy session follows: Participant A brings up a problem that everyone but Participant B can relate to. As the rest of the group engages in a discussion about A's problem, B begins to feel bored. B's boredom mounts and turns to frustration and anger. B's anger builds until it takes the form of an outburst and B blames the group for being insensitive. The discussion turns to focus on B, which makes A feel start to feel disenfranchised and disconnected, which soon mounts to frustration and anger, and so on. I was very interested in what such a system might look like visually and developed software later in 1998 to represent this. This software took the form of a group of simultaneously running applications that could pass data representing human interaction between themselves. Upon receiving data, each would play out a behavior and then broadcast feedback to the other participating applications. The behavior and the feedback were determined by unique properties assigned to each application. More recently in 2004, in conversations with artist and California State University, Sacramento mathematics professor Gene Oldfield, I came to understand my software as a sort of finite state machine. Kam (1997) describes finite state machines as models of behavior composed of states, transitions and actions. A state stores information about the past and reflects the input changes from the system start to the present moment. Transitions indicate that a state has changed and are described by conditions that would need to be fulfilled to enable the transition. An action is a description of an activity that is to be performed at a given moment (Kam, 1997). Oldfield also suggested that I investigate cybernetics, a discipline that has been overshadowed by modes of science, but that sought to blend epistemology, scientific method and systems theory. Wiener (1948), a founding father of cybernetics, described it as: the study of control and communication in the animal and the machine. Werner and his contemporary Ludwig Bertalanffy, a founder of Systems Theory, were interested in how feedback acts to regulate a system and saw the mechanism of feedback as a universal construct affecting all systems, including those electronic, mechanical and emotional (Scrivener, 2005). While both Cybernetics and Systems Theories are not at the forefront of theoretical thinking in the sciences today, they are precursors to more modern communications theory, information systems theory and computer science. Although not its original intent, the desire to debug my group therapy experience had led me to create a finite state machine in the spirit of Cybernetics. I merged this technology with art, animating my primitive mouth drawings as a way to express how the program was feeling. This evolved into "The Smile Project," an installation of a family of five interactive human-scale robotic sculptures, each with its own personality, capable of emoting using sound, body language and facial expressions. Neil and Iona [fig 1] were created in 2003. They are the first two of the family of five animatronic sculptures that were originally envisioned. In the course of their development, it was vitally important to me that the viewer's aesthetic experience transcended the technologies driving them. I did not want the "gee-whiz" factor to overshadow the art. Their facial expressions are composed of hundreds of animation cells drawn in graphite on Mylar and their fleshy, bulbous bodies are carefully sculpted objects. I set a goal that they be visually compelling with or without electricity. By adding sound and gesture, I came to realize that I had crossed a boundary into performance. With this in mind, I refined my software to be based on a loose understanding of modern improvisational performance. SOFTWARE (THEEGO) The brains and spirit of Neil and Iona are a software package I invented called TheEgo. Referencing one of the most seminal psychoanalysts of the twentieth century, Sigmund Freud (1856-1939), TheEgo is a technological representation of my understanding of human personality and interaction. Heffner (2005) explains Freud's theory of personality development and his notion of the ego as the reality-based part of one's personality that moderates between one's impulses (the Id) and one's moral compass (the Super-Ego). TheEgo software is a personality model that takes into account the feelings of robotic others. Version 1.0 was written in Visual Basic 5.0 in 1999 and ran on a network of five used 486 computers running Windows 95. Each computer used sounds and animated facial expressions to express itself. The current version, TheEgo 2.0, uses the same architecture as version 1.0, with enhancements that enable motion control, input for sensors and a sophisticated graphical interface used to design personalities. This was written in Python 2.3 and is run on embedded systems inside each robot, currently Gentoo Linux. Each uses a CMUCam for vision, and ControlFreak stepper motor controllers for movement and gesture. TheEgo has two modes of operation, a runtime mode called Egotronic and a design mode called Super-Egotronic. EGOTRONIC Egotronic mode is run during exhibition. In this mode, the software decides how to act based upon a personality file defined in Super-Egotronic mode. The flow is straightforward; the robot receives input, chooses a behavior and then commands the appropriate devices accordingly, whereupon it loops back. The input takes two forms, sensory and emotional. Sensory feedback is received by hardware connected to the robot itself, currently the CMUCam and feedback from the ControlFreaks. Emotional feedback comes from data packets passed between robots that describes its current state of mind. The output of Egotronic mode comes in the form of the physical movement of the robot, the sounds it makes and the facial expressions output to the screen. Super-Egotronic can graphically simulate what will occur in Egotronic mode; therefore it is the best way to illustrate the output of Egotronic mode. Super-Egotronic Super-Egotronic mode is used to build sequences of activity called Behaviors that are assembled to create a robot's Personality. The interface is broken into three sections, the Unconscious, the Imaginary Order and the Mood Grid. 1. Conscious: Used to assemble sequences of activity that form a behavior. 2. Mood Grid: Three-dimensional universe where behavior bubbles are plotted to form a personality. 3. Subconscious: Palettes or repositories of images, sounds and movement commands used to define a behavior. Behaviors are made up of a series of elements called "beats". Each beat represents a moment in time, like a drum beat. Each beat defines actions that should take place when its turn comes up. These actions can take the form of an image, a sound, a command that describes how the robot's body should move or a combination of all three. A behavior requires at least one beat that is followed by as many as is needed; the number of beats determines the length of the behavior. Each behavior is assembled in the Conscious, the pane of the software (see figure 1) that serves as a workbench for creating behaviors. Freud believed that everything one is aware of is stored in the conscious, and this pane acts as the small portion of the software that forms behaviors. Freud also described the concept Subconscious, or the part of one's personality that is right below the surface of our everyday interactions. For TheEgo, stockpiles of information such as images, sounds and commands are stored in the Subconscious (Heffner, 2005). A Behavior is plotted in a multi-dimensional universe called the Mood Grid. The Mood Grid looks and functions like a vector-based 3D drawing program; the behaviors appear as colored circles on it. As in a drawing program, Behaviors can be inserted, moved, sized and deleted. Behaviors can also be grouped together, moved as a unit and assigned to a layer that can be colored, locked or made invisible. Each view can be zoomed into or out of and scrolled. The user can choose to see four views at a time or one large view. The user assigns which 2D plane is displayed in which window. The Mood Grid can be configured to use as many dimensions as is imagined; however, I find working in more than three becomes really hard to comprehend. The user can also configure the names of the axes. I work in three dimensions labeled as: * Euphoric - Depressed * Kind - Mean * Active - Passive Personality development flow: 1. A new Behavior is created and appears both as a circle on the Mood Grid and as a new sequence in the Imaginary Order. 2. Sequences of images, sounds and movement commands from the Unconscious are assembled into a behavior in the Imaginary Order. 3. Behaviors bubbles are arranged in the Mood Grid to define complex probabilistic behavior. 4. A simulation is run, and the results are reported in the Crystal Ball. 5. The personality file is saved, uploaded to the robot and run in Egotronic mode. BEHAVIORAL DEVELOPMENT The uninterrupted sequences of Behaviors are probabilistically determined by their placement in the Mood Grid, weighted by feedback represented by objects called Influence Points. The rules are pretty simple; a Behavior can only transition to another that overlaps it. The amount of the area of intersection and the current behavior's proximity to Influence Points determines the likelihood of it being chosen during a turn. Influence Points represent feedback activity such as sensors and the emotional state of the other entities; how close they are to the current Behavior determines the strength of its pull. Super-Egotronic mode can run as a graphical simulation that reports the traversal ratios over time. This feature can be very useful in understanding how a system works and is relied on to fine tune the Personalities. Identity Crisis Historically, there is a prerequisite period of creative confusion artists must endure if a new medium is to ripen into a new art form. Daniel Giordan describes this phenomenon in terms of how photography impacted painting: "Empowered with the ability to authentically capture images in a split second, the photographer became the official recorder of our life and times, a role that painting had held for hundreds of years. In 1839, faced with the invention of photography, Paul DeLaroche is supposed to have declared, 'From today, painting is dead.' Painting was not dead, it simply needed to assume a new role regarding the kind of art it would create." (Giordan, 2005). I believe that it would be na•ve of me to assert that emotive cybernetic improvisational performance will one day justify its own genre. Artists working with simulating life should be prepared to face a blurred boundary between forms of expression, regardless as to whether it blossoms into its own discipline. Neil was exhibited for the first time at *ArtBots: The Robot Talent Show*, organized by Douglas Repetto in 2003 at Eyebeam Atelier in New York City. This was my first experience with an audience that had no preconceived idea what Neil was. It was here that, because of viewers, Neil was transformed from a living sculpture into a robotic performance artist. The audience was nonplussed by the technology, taking for granted that he was robotically alive. By simply hiding the technology inside Neil's skin, I had successfully transcended the "gee-whiz" factor and ended up in a completely different medium. The audience saw Neil as an accessible art object, who was also supposed to perform in an engaging way; like a person not a thing; a *who* as opposed to a *what*. Viewers spoke to him, made faces at him, touched him, and hugged him. This was an outcome I had not anticipated from the vantage of my studio. In hindsight it makes sense that people would anthropomorphize an anthropomorphic robot. This experience has resonated with me since and completely changed how I see my own sculptures. As with working with an actor, what Neil communicates and how he communicates has resulted in completely different works of art. Actors are blank canvases that have the potential to become almost anyone depending on the material they are provided. If Neil makes infantile sounds, he is a baby to be taken care of, if he speaks with authority, people are inclined do as he commands, if he tells jokes and giggles he is understood as a clown. The open system I created to drive Neil and Iona's emotional behavior has left me with a tool I am just now learning how to use. The paradigm shift leaves me questioning the difference between sculpture and performance. I understand the former and am still quite perplexed by the latter. By creating a sculpture meant to simulate life, I have accidentally created a pair of living puppets or cybernetic performance artists. Now that the technology is resolved, I am left with a troubling question: What exactly should they become? _____________________________ REFERENCES 1. S.N. Matsuba and B. Roehl, *Bottom, Thou Art Translated: The Making of VRML Dream*, IEEE Computer Graphics and Applications, 19 (2), March/April 1999, 45-51. 2. http://www.troikaranch.org/ 3. K. Farley, "Digital Dance Theatre: The Marriage of Computers, Choreography and Techno/Human Reactivity", *Body, Space and Technology*, 3(1), 2002, (http://www.brunel.ac.uk/depts/pfa/bstjournal/3no1/journal3_1.htm) 4. F. Sparacino, G. Davenport and A. Pentland, "Media in performance: Interactive spaces for dance, theater, circus, and museum exhibits", *IBM Systems Journal*, 39(3-4), 2000, 479-510. 1. D. Giordan, *The Digital Aesthetic v.1* http://www.webreference.com/graphics/essay/essay2/. Retrieved June 17, 2005. 2. C.L. Heffner, *Psychology 101*, 2005, http://allpsych.com/psychology101/ego.html Retrieved 19 June 2005. 3. F. Heylighen, C. Joslyn and V. Turchin (editors), *Principia Cybernetica Web* (Principia Cybernetica, Brussels), URL: http://pespmc1.vub.ac.be/FEEDBACK.html. Retrieved 17 June 2005. 4. T. Kam, *Synthesis of Finite State Machines: Functional Optimization*, Kluwer Academic Publishers, Boston, MA, 1997. 5. A. Schrivner, *A Curriculum for Cybernetics and Systems Theory*, 2005 http://www.well.com/user/abs/curriculum.html#From. Retrieved 17 June 2005. AUTHOR BIOGRAPHY JASON VAN ANDEN'S cybernetic artworks have evolved through a mix of passionate art making and a compulsion to invent. His current focus is on developing graphical software-based systems that simulate human emotional interaction based upon what he perceives as behavioral mechanics. This investigation has resulted in a variety of artworks, including a pair of human scale emotive robots named Neil and Iona and a popular online videogame called *Farklempt!*. Van Anden holds a BFA in Sculpture from Syracuse University and attended the Skowhegan School of Painting and Sculpture. He is the President and Chief Executive Officer of Quadrant 2, Inc., a technology company he founded in 1996. LAURI GOLDKIND is a Ph.D. candidate in Social Work at the Wurzweiler School of Social Work at Yeshiva University in New York, NY and holds a Masters in Social Work from the State University of New York at Stony Brook, Stony Brook, NY. Her dissertation focuses on the intersection of the juvenile justice and public education systems. She has raised money and run programs for a variety of nonprofit agencies in New York City. _____________________________ PATCHWORK IN MOTION: A PRACTICE-LED PROJECT INVESTIGATING THE SHIFTING RELATIONSHIPS AND PROCESSES ASSOCIATED WITH THE PERFORMING BODY IN INTERACTIVE AND NON-INTERACTIVE VISUAL ENVIRONMENTS. by Maria Adriana L. Verdaasdonk Department of Dance Creative Industries Faculty Queensland University of Technology Victoria Park Road Kelvin Grove Qld 4059 Australia m [dot] verdaasdonk [@] student [dot] qut [dot] edu [dot] au http://www.vision.co.jp/66b KEYWORDS patchwork, interdependencies, salience, figure-ground, segmentation, performer-media relationships, conversations, active co-player, body texturizer ABSTRACT According to visual perception theory, humans have a tendency to view a scene in terms of "figure" as foreground of the scene, and "ground" as background of a scene. In performances using motion graphics and multiple visual projection screens, decoding the scene in this way may be more problematic. This research explores stage environments where performers and visual images become active co-players, in both interactive and non-interactive contexts. Here the concept "body texturizer" describes how the body inhabits and effects the visual environment. Interactive technology is perceived as a means to augment performer expression and theatrical illusion, rather than a demonstration of the technological devices themselves. INTRODUCTION As a departure point for exploring the issue of dominant feature or salience in a theatrical environment, I borrow from visual perception theory the view that humans have a tendency to view a scene in terms of "figure" as the salient feature or foreground, and "ground" as the less salient feature or background of a scene (for further explanations of salient feature and figure-ground, see appended glossary). However, in performances using visual projections and multiple screens, the visual images in motion often become more compelling than performers and their movements, the visual projections thus becoming the dominant feature. In many stage sets, performers are seen in motion against an enclosing, relatively immobile setting. In such settings, Arnheim refers to Duncker's rule that objects in the visual field are "seen in a hierarchical relationship of dependence. . . . The dancer is seen as part of the stage setting, not the stage setting the outer rim of the dancer" [1]. According to Arnheim, Duncker's rule infers that if the framework is immobile, in this case the stage setting, then the moving dancer or actor becomes the dependent object. In this way, the performer as the salient "figure" is in a relationship of dependence to the "ground." While such a rule may apply where theatrical sets are static, in performances where both performers and the stage set, i.e. the visual projection screens, are in motion, any decoding of scenes in terms of figure and ground becomes more problematic. INTERDEPENDENCIES To open up this issue, I am drawn to the idea of viewing relationships between performers and visual media in terms of interdependencies. Birringer highlights this potential when he states that "[if] technology and bodies are seen in terms of flows of energy or intensity or as a fluid dynamics, then there is grounds for collaboration" [2]. Dinkla corroborates this when she proposes that in interactive performances, performers act as part of a networked system: "The field is characterized by the inherent changeability as well as the impossibility of disentangling the interdependencies between the dancers, choreographer, musician, stage space and technological/digital systems, which are interconnected and subject to mutual influence" [3]. Thus along with the notion of a fluid, reciprocally beneficial relationship comes an inherent inextricability. Auslander also raises this point when he asks, in what he refers to a mediatized performance, is what we are seeing the "juxtaposition of the live and the digital, a shifting among realms? My feeling is that the answer is no, that we now experience such work as a fusion, not a con-fusion, of realms, a fusion that we see as taking place within a digital environment that incorporates live elements as part of its raw material. Rather than a conversation among distinct media, the production presents the assimilation of varied materials to the cultural dominant. In this sense, Dance + Virtual = Virtual" [4]. In the views of Auslander and Dinkla, there occurs a blurring of entities on the performance stage, while in Auslander's view per se any distinct physicality of the dancer is subsumed under the dominant (which in this particular research would be the visual) media. This becomes a challenge, in terms of the appeal to create a stage environment where there can be distinct "conversations" among the various entities, including performers, the visual and sound media, as well as the creative technical people responsible for creating and manipulating the media. As the area under investigation and practical performance outcome(s) of the Patchwork in Motion project involve collaborative teams comprising the above-mentioned performers and creative participants, I will mainly present views collectively in the first-person plural for the remainder of this paper. SCENES AS SITES: SEGMENTED FRAGMENTS AND "SEAMLESS" MEDIA To develop potentially distinctive conversations, we have engaged the metaphor of patchwork as a method for creating a segmented sequence of scenes. The scenes are like pieces of a patchwork quilt: The separate fragments bind together as subsystems to form an interconnected fabric or network. An open, mobile space, it can be added onto and/or reconfigured. While an actual patchwork quilt is a two-dimensional static form, in the context of a theatrical setting, two-dimensional screens encompass a three-dimensional stage space, with projected motion graphics and performer movement creating a kinetic, dynamic environment. The patchwork metaphor implies possibilities for exploring non-linear sequences of different contrasts and moods, an assemblage of pieces with perhaps no central, overriding motif. A work-in-progress, performers in the project to date are of various backgrounds, including ballet, contemporary dance, hip-hop, mime and butoh. Developing scenes as segments allows for an exploration of different theatrical moods, while each "patch" becomes a site for investigating clear themes in terms of the relationship between diverse body movements and visuals. This segmentation allows both performers and viewers to identify the sequence of events as they occur temporally. In terms of the rehearsal process too, the notion of segmented scenes allows for an immersive, reiterative process, particularly in scenes where performers need to accustom themselves to interactive devices such as motion or pressure sensors and camera tracking. From the technical perspective, however, the visual and sound artists and programmers suggest that while the term patchwork describes the multiple patching of cables in networking the hardware, programming involves a more seamless process. Although the performance outcomes appear as a temporal sequence of segments, the technical process involves multiple pathways between non-linear media segments. The comments of the audio-visual team highlight the matter of interconnection between the scenes themselves, for, from the viewpoint of live staging and artistic expression, there is also a need for smooth transitions rather than merely presenting a series of disjointed fragments. Co-ordination of the scenes thus takes on a cinematic editing approach where composition involves a series of edited "shots." Gradual transitions maximize continuity, while instantaneous cuts cause disruption yet may also result in unexpected moments that thwart viewer attention. This provides opportunities for visual dissonance, defined by Solso as the "tension caused when one experiences a disparity between what one expects to see and what one actually sees" [5]. Assembling the patchwork therefore calls for fluid sequencing to sustain the patchwork "in motion," yet it is important that the marks of construction - its patchwork structure - are not effaced in the process. DEFINING PERFORMER-MEDIA RELATIONSHIPS Returning to the subject of interdependencies, from a practical point of view it is necessary to try and define possible performer-media relationships. Saltz, writing on the role of interactive technology in theater, has compiled a useful taxonomy of performer-media relationships [6], which he himself states is neither exhaustive nor conclusive. I find several of his terms appropriate in seeking to contextualize our use of visual media. One category Saltz refers to as "virtual scenery" [7]. In Patchwork in Motion, this involves dynamic rather than static imagery, for example, animated landscapes, geometrical shapes or lattice structures that serve to highlight, integrate or immerse the performer. Our manipulation of this virtual scenery is either interactive, that is, the computers are able to sense the environment via sensors or cameras; or non-interactive - the visuals are pre-recorded or generated on the fly in real time and are not produced through any sensing of the environment. This virtual scenery does not merely form a backdrop however, for we are developing scenes in which a performer appears to 'inhabit' the projected virtual scenery or the visual images become an ally or adversary to the performer. Saltz defines a further relationship as "instrumental media," where "interactive technology is used to create new kinds of instruments" [9], and I find this term most aptly describes the way in which different images and/or sound effects may be triggered by a performer's body actions via sensors or camera tracking. Instrumental media can be multimodal; for example, body movement simultaneously links to both visuals and sound. A multimodal case in point is a scene entitled Purgatory. Here, a performer is seated on a box-shaped device called GelForce, currently being developed at the University of Tokyo [10]. This device contains a tactile sensor that measures the magnitude and direction of a force, in this case, the pressure of the performer's buttocks and fingertips, which is then measured optically by a camera inside the box. Prior to this staging, this device had never been used in a theatrical setting, and, while it had been used to achieve sophisticated visual effects, it had not yet been programmed to produce a sound outcome. The scene Purgatory thus became a site for a meeting of artistic and scientific research agendas, the end result being that the performer's movements were mapped to both visual and sound outcomes in the live performance. This experience was mutually valuable, for it led to an augmented performer expression concurrent with the development of the technology itself. Saltz identifies a further performer-media relationship as "virtual puppetry" [11], which he describes as blurring from and into instrumental media, the difference being that "while instrumental media is an extension of the performer, a kind of expressive prosthesis, a virtual puppet functions as the performer's double" [11]. Therefore, where instrumental media augment or complement a performer, a virtual puppet becomes a kind of doppelganger or a performer's mouthpiece. In addition, Saltz refers to "dramatic media" [12], where media interact with live performers, not simply as virtual puppets but in the role of characters in a narrative. Saltz highlights the theatrical potential of virtual puppetry and interactive media, and to this end a number of developers of interactive environments are creating systems that endow media with "real" agency, that is, with a capacity for adaptive and generative behavior [13]. However, while the idea of media endowed with simulated intelligence is fascinating, it is the interest - and therefore perhaps also the limits - of this project to explore visual environments that only at times become interactive, for it is not always the goal for the media to be responsive to performer movement. Our concern in this project lies rather with the visual and sound artists responsible for the live manipulation of the media as being "real" agents in the co-creation of the performance. However, what has emerged here is the notion of "active co-player," a term I have cobbled together to describe the way in which this particular investigation is examining interdependencies between performers and media. Yet it must be stated that in our case, this term implies no behavior-based ability on behalf of the media itself. SETTING UP CONVERSATIONS: THE NOTION OF ACTIVE CO-PLAYER The notion of active co-player is effective in setting up stage environments in which there can be dynamic conversations between performers and visual media. The projected imagery is seen as an active co-player in abstract, geometrical, figurative, anthropomorphic or non-anthropomorphic ways. The visuals become ally, adversary or playmate to the performers, as well as providing textural surfaces and lighting effects, and in this way, the media becomes an energy or force to challenge, combat, tease, play with or inhabit. The notion works well in both interactive and non-interactive contexts. In non-interactive scenes, active co-playing occurs through choreographed body movements and precise pre-recorded or real-time placement and layering of the projected visual images. Active co-playing also occurs through a kind of structured improvisation, that is, throughout a given scene there are predetermined points in terms of body positions or visual objects, with free interplay occurring between these points. In terms of the interactive scenes, there is an apparent causal relationship between performer movement and the visual and sound outcomes. In these scenes, as in the non-interactive scenes, visual media and performer movement can be preset, occur through structured improvisation or be a mixture of both. One concern in scenes involving interactive devices is that viewers are aware, at least at the beginning of a scene, of the cause-and-effect relationship between performer and visual media. Here, I apply Palindrome Inter-media Performance Group's notion of "gestural coherence," described as "the perceptual coherence between sound and the movement that generates it" [14]. Although they specify gestural and sonic coherence, I find this term equally points to coherence between body movement and visual media. EXPOSING AND SHIFTING THE SO-CALLED DOMINANT FRAMEWORK: SOME PRACTICAL SOLUTIONS Black and white imagery One solution to date, via a previous multimedia performance piece entitled "Test-patches," which informs the current project, has involved the use of black-and-white imagery to minimize visual information. As the natural world is generally perceived in color, we may have been able to suspend typical visual associations, while use of black and white itself evokes the atmosphere of a "living filmstrip" with its inherent references to media of the past. Here, geometric shapes, dreamy Chinese ink-like liquid effects and sharply defined areas of darkness and light facilitate both high-contrast and subtle visual environments that mask, highlight, integrate or immerse the performer. Motion sensors and tracking devices in several of the scenes enable the live synchronization of body movement to visuals. In a scene called Liquid Dream, two dancers wear motion sensors on their wrists with real-time liquid-ripple visual effects occurring in response to their hand movements. In another scene entitled Signal, a sensor is worn on the performer's back as lattice-like visuals create an immersive cubic structure that expands and contracts in sync with his breathing and movement. In a non-interactive scene entitled Barcode A, alternating black and white bars of varying widths create a moving venetian blind effect. The performer is partially viewed between the gratings while multiple cast shadows complicate the task of distinguishing the actual performer. In another non-interactive scene, a mime creates an imaginary ball and throws it into the air. It appears as if by magic on the screen as a virtual object that then humorously expands and multiplies. The scenes described here were shaped through active co-playing, involving visuals of black, white or gradations of grey and with performers costumed in either black or white, to generate diverse scenes based around a visual idea, a mood, a body movement or an interactive idea. Color and texture: The "body texturizer" In patchwork, each unit is block-like, the quilting process involving a singular repeating element. Yet each block or element need not be homogeneous. The piece-meal construction of Patchwork in Motion is comparable to the compositional method Deleuze describes "in 'crazy' patchwork, which fits together pieces of varying size, shape, and color, and plays on the *texture* of fabrics" [15]. Hence, color and texture can play a vital role, extending possibilities for variation and variegation. In the piece Test-patches, we could achieve high-contrast visual effects through light and shadow, with more blurred effects through shades of grey. However, in Patchwork in Motion, we have been looking at different ways to create contrast, placing emphasis on visual textures, including organically derived or inspired textures. I have coined the term "body texturizer" as a means to better understand and describe the way in which projected visual textures can appear to intensify or be modified by performer movement via interactive devices. As such, this term becomes a facet of the active co-player notion and indicates the potential for scenes where the stage setting, rather than being an enclosing space, can - in contrast to Duncker's rule cited in the introduction to this paper - actually appear to become the outer rim of the dancer. Texture, as applied here, refers to the way visual images can be used in combination - for example, spatially through multi-layering or montage of visual elements in a single scene, or temporally through juxtapositions of visual images from one scene to the next. Although these images are virtual and therefore not actually tactile, they "may evoke a sense of physicality" [16]. The body can appear to become a part of the visual texture, for example, as a fragmented part of the screen. To illustrate, in the case of frontal projection and a spinning performer costumed in white: the performer seems to become a mini projection surface as the outline of the body blurs and is discernible no longer as human but rather as a moving fragment of the screen. The performing body can interactively shape the textural layers either through motion or pressure sensors, or through camera tracking, thereby becoming a living "texturizer." In contrast to the body as a detached screen fragment, the performer may also be revealed as an entity in her/his own right, "inhabiting" and affecting the visual environment. We have recycled previous material - a specific patchwork method - by revisiting the scene mentioned earlier entitled Barcode A. In the new version, the performer becomes a kind of transparency gauge: the spinning black bars darken with movement, become lighter when movement slows and become transparent when the performer is motionless. The visual layers containing the spinning bars partially conceal a further colored background layer, revealed completely when the performer is still. What occurs here is that the geometric black and white stripes evoke the verticality of trees in a forest, while the revealed background layer assumes the texture and horizon of an undulating landscape. A further idea currently being explored is a scene entitled "Living Lens." Two performers move across the stage, their movement corresponding to virtual lenses that appear on the projection surface, creating a moving magnifying effect over the projected organic textures. Interdependency exists between the performers and the visual artists who coordinate lens motion in real time with the performers' movement. Here, there is a sense of both micro and macro levels, while a certain viscosity seems to emanate from both performer movement and the visual motion. These examples illustrate the ways in which the performing body potentially shapes and is shaped by the visual textures. It is important to note here, however, that the aim is not an exploration or demonstration of the visual media or interactive technology per se; rather, the interactive devices and visual imagery are perceived as a means to augment performer movement or enhance theatrical illusion. CONCLUSION In performances in which visual projections are a theatrical component, the images often become more compelling for viewers than performers and their movements. In such environments, however, it is possible to create a layering and juxtaposition of imagery, and in this way, possibilities emerge for collaboration between performers and visual media. Interactive devices facilitate opportunities where visual media become responsive to performers, thereby augmenting and extending expressive potential. Rather than perceiving the stage setting in terms of figure and ground dependencies, performers and visual media become active co-players through dynamic interchange: mutually interdependent yet where the essential nature of each may also emerge. GLOSSARY Salient feature Admittedly a slippery notion, this is used in theories of perception and cognition to refer to features or stimuli that are conspicuous, prominent, or highlighted compared to the rest of the visual field and thus draw the attention focus of an observer. In this way "image features that are interpreted perceptually as 'foreground' rather than 'background' are assigned the greatest weights" [17]. Figure-ground This is also a complex issue. To put it simply, Arnheim cites Rubin's rule, which states, "the enclosed surface tends to become figure whereas the enclosing one will be ground" [18]. In two-dimensional pictorial space, texture, proximity and figure-ground reversals may result in ambiguous figure-ground relationships. In a three-dimensional stage setting with a performer in motion, it may be a relatively easy task to distinguish the familiar human figure as the enclosed, conspicuous feature against the enclosing two-dimensional surfaces of the projection screens. Referring to sculpture, Arnheim notes the three-dimensional volume of space surrounding the object, the perception being that "the statue seems to monopolize all the figure qualities" [19]. Although the sculptured object in space is a static example, in dynamic stage settings, the three-dimensional volume of theatrical space can also be seen to isolate the figure qualities of the performer. However, in performances with performers seen against multiple projection screens, and where the projected imagery includes virtual objects and textures in motion, it may be more complicated to assign greatest weights in terms of the most prominent features. _____________________________ REFERENCES 1. R. Arnheim, *Art and Visual Perception: A Psychology of the Creative Eye*, p. 368, London: Faber and Faber (1972). 2. J. Birringer, *Media Performance: Along the Border*, p. 127, Baltimore and London: The Johns Hopkins University Press (1998). 3. S. Dinkla, "Towards a Rhetoric and Didactics of Digital Dance", in *Dance and Technology: Moving Towards Media Productions*, pp. 22, Berlin: Alexander Verlag (2002). 4. P. Auslander, *Liveness: Performance in a Mediatized Culture*, p. 38, London and New York: Routledge (1999). 5. R.L. Solso, *Cognition and the Visual Arts*, p. 122, Cambridge, Massachusetts; London: The MIT Press (1999). 6. D.Z. Saltz, Live Media: Interactive Technology and Theatre, *Theatre Topics*, Vol. 11:2, pp. 107-130 (2001). 7. Ibid, p. 124. 8. Ibid, p. 125. 9. Ibid, p. 126. 10. K. Vlack, K. Kamiyama, T. Mizota, H. Kajimoto, N. Kawakami, and S. Tachi, GelForce: A Traction Field Tactile Sensor for Rich Human-Computer Interaction, *IEEE Xplore*, retrieved June 26, 2005 from http://www.ieeexplore.ieee.org/iel5/9749/30762/01424969.pdf?arnumber=1424969 11. D.Z. Saltz, ibid, p. 127. 12. Ibid. 13. F. Sparacino, G. Davenport and A. Pentland, "Media in Performance: Interactive Spaces for Dance, Theater, Circus, and Museum Exhibits," *IBM Systems Journal*, Vol. 39, Numbers 3 & 4, retrieved February 5, 2004 from http://www.research.ibm.com/journal/sj/393/part1/sparacino.html 14. J.B. Rovan, R. Weschler, and F. Weiss, Seine hole Forme: Artistic Collaboration in an Interactive Dance and Music Performance Environment, *Crossings eJournal of Art and Technology*, 1(2), retrieved August 15, 2004 from http://crossings.tcd.ie/issues/1.2/Rovan/ 15. G. Deleuze, *A Thousand Plateaus: Capitalism and Schizophrenia*, p. 476, Minneapolis: University of Minnesota Press (c1987). 16. R. Vella, *Sounds in Space, Sounds in Time*, p. 144, London: Boosey & Hawkes Music Publishers (2003). 17. T.D. Albright, "More Than One Way to See It Move?" *Proceedings of the National Academy of the Sciences of the United States of America*, Vol. 96, p. 7611, July 1999. 18. R. Arnheim, ibid, p. 220. 19. Ibid, p. 223. BIBLIOGRAPHY Albright, T.D., More than one way to see it move? *Proceedings of the National Academy of the Sciences of the United States of America*, Vol. 96, pp. 7611-7613, July 1999. Retrieved July 1, 2005 from: http://www.vcl.salk.edu/Publications/PDF/Albright1999_PNAS.pdf Arnheim, R., *Art and Visual Perception: A Psychology of the Creative Eye*, London: Faber and Faber (1972). Auslander, P., *Liveness: Performance in a Mediatized Culture*, London and New York: Routledge (1999). Birringer, J., *Media & Performance: Along the Border*, Baltimore and London: The Johns Hopkins University Press (1998). Deleuze, G., *A Thousand Plateaus: Capitalism and Schizophrenia*, Minneapolis: University of Minnesota Press (c1987). Dinkla, S., "Towards a Rhetoric and Didactics of Digital Dance", in *Dance and Technology: Moving Towards Media Productions*, S. Dinkla and M. Leeker (eds.), pp. 14-28, Berlin: Alexander Verlag (2002). Rovan, J.B., Weschler, R., and Weiss, F., Seine hohle Forme: "Artistic Collaboration in an Interactive Dance and Music Performance Environment", *Crossings eJournal of Art and Technology*, 1(2), retrieved August 15, 2004 from http://crossings.tcd.ie/issues/1.2/Rovan/ Saltz, D.Z., "Live Media: Interactive Technology and Theatre", *Theatre Topics*, Vol. 11:2, pp. 107-130, fall 2001, retrieved August 14, 2004 from http://muse.uq.edu.au.gateway.library.qut.edu.au/journal/theatre_topics/v011/11.2saltz.html Solso, R.L., *Cognition and the Visual Arts*, Cambridge, Massachusetts; London, England: The MIT Press (1999). Sparacino, F., Davenport, G., and Pentland, A., "Media in Performance: Interactive Spaces for Dance, Theater, Circus, and Museum Exhibits", *IBM Systems Journal*, Vol. 39, Numbers 3 &4, 2000, retrieved February 8, 2005 from http://www.research.ibm.com/journal/sj/393/part1/sparacino.html Vella, R., *Sounds in Space, Sounds in Time*, London: Boosey & Hawkes Music Publishers (2003). Vlack, K., Kamiyama, K., Mizota, T., Kajimoto, H., Kawakami, N., and Tachi, S., "GelForce: A Traction Field Tactile Sensor for Rich Human-Computer Interaction", *IEEE Xplore*, retrieved June 26, 2005 from http://www.ieeexplore.ieee.org/iel5/9749/30762/01424969.pdf?arnumber=1424969 AUTHOR BIOGRAPHY Maria Adriana Verdaasdonk (Melbourne, Australia) is a theatre studies graduate who based herself in Tokyo in the early 1990s to study the martial art aikido and the Japanese dance form known as butoh. Working intensively with the dancer Akiko Motofuji, she also participated in several performance projects with the butoh dancer Kazuo Ohno under the direction of his son Yoshito Ohno. In 1994, she co-founded Tokyo-based multimedia performance unit 66b/cell with her partner and visual artist, Tetsutoshi Tavata. This media and performance collective presently works in theaters, warehouses and galleries and performs internationally in dance festivals and media arts festivals. Through a piece entitled Test-patches (2001-2003), 66b/cell have been exploring interactive possibilities of motion sensors and camera-tracking systems in collaboration with Junji Watanabe of the University of Tokyo. This performance piece was presented at the media arts festival Ars Electronica, in 2002. Based both in Tokyo, Japan, and Brisbane, Australia, Maria is currently a research candidate at Queensland University of Technology, where she is continuing to explore stage environments focusing on the performing body and the relationship with digital visual and sound media. She received the Royal Society Encouragement of the Arts Award for visual research on the Japanese art concept of *ma* in 1996, and the Peter Elkin Drama Prize for the Faust II Project in 1997. _____________________________ VR AS A PERFORMANCE FOR AN AUDIENCE by Paul Hertz 2215 W. Fletcher St. Chicago, IL 60618 U.S.A. paul-hertz [@] northwestern [dot] edu KEYWORDS virtual reality, performance, immersion, agency, intermedia, formal systems. ABSTRACT Virtual Reality (VR) systems that support multiple viewers in the same physical space, such as the CAVE, introduce an element of audience-directed performance into what is commonly described as a "user-centered" medium. This essay contemplates the possibility of creating VR performances in a traditional musical performance or theatrical situation, with an audience, as a hybrid or intermedia art form. It briefly develops some theoretical aspects of navigation, subjectivity, agency, and the immersed body, and proceeds to discuss a new collaborative performance work, *Fools' Paradise*, by the author and composer Stephen Dembski. _____________________________ NAVIGATION AND NARRATIVE DEVICES Virtual reality, or VR, has been rather loosely defined as a computer-human interface that creates the illusion of the immersion of the user in a computer-generated environment [1]. It generally achieves this illusion through sensory vividness and responsive interaction. "Virtual environments" predate computers: from the elaborate panoramas popular in the 19th century to Morton Heilig's Sensorama (1956), artists and engineers have long attempted to immerse audiences in a virtual world with available technologies. A more narrow definition of virtual reality, current in computer engineering and science, applies the term to interfaces that use 3D graphics to generate stereo images from the user's perspective on displays that fill most of the visual field, with real time navigation and interaction. The earliest such interfaces used booms or head-mounted displays. More current technologies use large stereo projection screens and lightweight LCD-shutter goggles or polarized glasses. The latest technologies can eliminate the goggles or glasses, and provide totally unencumbered stereo vision, with some limitations. Virtual reality is not just an interface. It can properly be regarded as a medium. From its inception, artists have been using it to construct virtual worlds for other persons. Virtual reality advocates routinely describe VR as a "user-centered" medium. This may hold true of head-mounted displays (already become relics), but it breaks down in systems that support multiple viewers, such as the CAVE and other large-scale, unencumbered systems. Unless every person observing a virtual world possesses independent vision, tracking and navigation, the split experience of performer and audience quickly seeps into the experience. In the typical CAVE scenario, a single user with head tracker and navigation wand negotiates a world while onlookers crowd around. Virtual reality developers sometimes play with this scenario. At one extreme, Hisham Bizri's *Las Meninas: The Articulation of Vision* [2] explicitly incorporates the privileged navigator as a guide in a "virtual museum." At the opposite extreme, Drew Browning's VR interfaces [3] provide immersive experiences from the viewpoint of a disabled person, and can only be fully appreciated by the navigator. For a variety of reasons, the dominant mode of design for VR experiences is one of architectural or "world" exploration. VR software bears the legacy of military use for simulations and scientific use for visualization. It is only an art medium by adoption. In most VR worlds, one navigates continuously through a virtual space as if in a real space. The occasional passage through a wall, gravity-defying flight, or "teleportation" to new space are the only concessions to other ways of articulating transitions from one setting to another. This is not to say that cinematic or theatrical transitions are impossible or even undesirable. The recent VR world *Special Treatment* [4], a recreation of Auschwitz, uses fade in/fade out transitions to provide a more "episodic" feel to the world, and on some occasions takes control of navigation - a use of cinematic passivity that seems peculiarly relevant to the subject matter. Navigation in VR combines a mode of travel with wayfinding [5]. As in architecture, the deployment of spaces creates diverse possibilities of wayfinding, which may be regarded as the creation and use of systems of information that enable purposeful travel. Thus, navigating architecture and navigating information bear a deep similarity [6]. When wayfinding is effective, the architecture/user interface tends to sink below the level of conscious perception. Most VR worlds are designed with various points of attraction, places intended to attract participation and interaction, points where decisions are made. Animation and sound cues are often used to flag such points. Much has been made of the opposition of narrative and interactive media. We sacrifice our agency to the passive pleasures of narrative, and in a darkened cinema even allow the screen to construct physical motion for us. Interactive technologies return agency to us: Even when media offer only fixed choices, at least we get to choose (or so the argument goes). Virtual reality supposedly provides unrestricted choice, six degrees of freedom that represent the ultimate return of agency to the participant. Yet the language of VR, however primitive compared to older technologies, nevertheless introduces articulated choices with its wayfinding devices. In the end, one can choose to follow the cues, or not. Expecting a reward, most people follow the cues. THE BODY AND ITS DOUBLE Bodies in virtual reality often seem to exist as a kind of spectral presence, a floating point of view that passes through walls and flies where it will. This is primarily the fault of the interface, both in software design and hardware design. Vision has been privileged over all other senses. Sound has generally been secondary, and made to conform to the virtual architecture through digital signal processing that supplies it with spatial qualities. Motion control is isolated to a handheld wand. Use of the whole body to navigate is comparatively rare, although there have been some notable uses of physical devices such as bicycles or surfboards to introduce a stronger physical component to VR navigation [7]. The ghostly presence of the subject in VR is akin to other virtual presences, e.g., the "place" where people meet in telephone conversations or in MUDs and MOOs and other electronic communication spaces. VR has the advantage over these other media of providing a degree of unparalleled immersion. Furthermore, the sense of immersion can be enhanced by interactivity of a very direct sort - physical movement of the head and body or gestures with the wand or other devices directly affect perspective and virtual objects. The body is not so much uninvolved in VR as it is reduced to its visual system. Reflecting on her collaborative work with dancer Yacov Sharir, *Dancing with the Virtual Dervish: Virtual Bodies*, Diane Gromala has compared subjectivity within VR to subjectivity in experiences of pain and imagining [8]. In Gromala's view, this binary signals opposing boundary states of intentionality. Nothing is more personal than pain, nor more communal than imagination, which can project us beyond ourselves, into the world and into other people. The immersed body shares aspects of both pain and imagination: It splits one's sense of reality from others even as it allows one to participate in a shared reality. She compares this state of mind to shamanistic practices of self-inflicted pain, or mythopoetic marks on the body that promote or record a transition of consciousness. With some irony, she suggests that sim-sickness, the nausea associated with sensory dissonance in VR and other simulation engines, is at once a revelation of the problematic nature of attempts to split meat from mind and a mark of the insistent mass of the body, cognizant of its existence both within and without the virtual space. Elsewhere I have suggested that sensations of immersion and sensory fusion facilitate the inscription of desires for plenitude into virtual spaces by serving as symbolic equivalents for visionary states of mind [9]. Frustration, like sim-sickness, ensues as the mark of our inevitable failure to morph transcendent symbol into immanent experience. The psychological reality sketched out by Gromala may not be susceptible to empirical proof, but clearly it can function as an elaboration of the linkages between body, symbolic reasoning, and immersive spaces. In performances of *Dancing with the Virtual Dervish*, Sharir danced fully immersed with a tethered HMD. His visual field, an interior architecture constructed from scans of Gromala's body and internal organs, was projected for the audience, whose members at determined points could also suit up and navigate the virtual world. He describes his sensations while performing as follows: "The sensation of disembodiment cannot be disconnected from the sensation of embodiment; that is, I feel the physicality, the groundedness of gravity simultaneously with the sense of immersion and altered abilities, such as the ability to 'fly' through the simulation" [10]. AGENCY AND ILLUSION The foregoing discussion suggests that navigational freedom is as problematic in VR as in constructed hypermedia, and that rhetorical devices of narrative inevitably creep in. It also attempts to provide the bare scaffolding of a theory of the immersed body as having a double presence, simultaneously real and imaginary. Illusory freedom and doubled bodily presence bring on another fraught topic, agency. If freedom of navigational choice is the mental sign of empowered agency, bodily freedom is its physical counterpart. Although VR could hardly live up to its techno-utopian hype, the capacity of the immersed subject to make choices in interactive domains such as VR does imply a radical demarcation from media that do not support such choices, such as cinema or written narratives. Such media have highly elaborated languages and immersive strategies but cannot detect or respond to the user, whose point of view is embedded in the narrative medium. Real-time interaction endows the user with agency and even constructs an emergent agency from the latent contingencies of program code when an immersive environment responds with sufficiently complex behaviors of its own. In most VR environments, the user's freedom of choice is not reflected in a similar freedom of movement of the body. By and large, tracking and control devices are not designed to encourage full body movement. For artists using VR either for performance or for user participation, devices that allow whole body movement are a necessity. One could argue that taking VR in the direction of performed action for an audience runs counter to the prevailing tendency, moving away from real-time interaction back towards narrative, away from collective authorial experiment back to passive reception. If this represents a sort of retrocession, the proper analogy is not with cinema or text, but with improvisational music. The virtual world, explored by expert navigators for an audience immersed, if not in a virtual world, at least in the bracketed time of a multisensory performance, behaves as an instrument, not as a deterministic architecture. Every performance is different - and should be. Teasing out the hidden possibilities of VR in performance will expand the vocabulary by which agency is constructed in real-time interaction, while also reaching a wider audience than fits within the confines of a CAVE. Virtual worlds, which are sufficiently vivid to sustain an expert performance, whether through-composed or improvised, for an audience, can very probably be reworked as participatory installations that are also vivid. That is more than can be said for string quartets or puppet theaters. FOOLS' PARADISE In recent work, I have attempted to "turn the tables" on the user-centered illusion of VR, to treat VR as a composed performance medium for an audience. *Fools Paradise*, a VR work created in collaboration with composer Stephen Dembski, is one result of my research. Software for controlling audio signal processing via a VR interface is another. Although the software provides "pretty good" audio spatialization, it is designed above all to use VR data to control any desired aspect of audio signal processing. In effect, a VR environment can become an interface for real-time sound synthesis and processing. I would like to see a similar degree of freedom arrive in the visual aspects of VR - a departure from the strict maintenance of the illusion of 3D architecture towards an exploration of the full visual capabilities of the system. At times this can be achieved by as simple a device as having the performer "explode" the 3D model by leaning too close to the screen, suddenly revealing the artificiality of the virtual world. Such a move is unspeakably gauche in proper VR, of course - let us daintily avoid the boundaries of the system. As artists, we should find out where they are trying to work them into our vocabulary. *Fools Paradise* functions as both a VR installation and a performance. It is based on the *Proverbs of Hell* by visionary poet and artist William Blake. Dembski scored forty-eight selected proverbs for soprano, cello, and flute. Hertz created a virtual world where each song is associated with a large, colorful "jewel" embedded in a tableland on an island floating in a sea of stars. Seen from afar, the island is a tongue and the tableland is a book with a stream of words running down its middle crease to cascade over the edge of a cliff onto the tip of the tongue and into the sea. The person who navigates the virtual world sees a mask floating in front of her. The mask's position and orientation are controlled by the wand. When the navigator enters a jewel, the jewel dissolves, a new mask appears, and the musical setting of one of the proverbs is performed. The navigator continues her journey with the new mask, but its image is also left behind at the location of the jewel. As the journey progresses, the tableland becomes filled with masks, all of which turn to face the navigator. In the performance version of *Fools' Paradise*, a dancer navigated the virtual environment in front of a rear projection screen set off to one side. The audience saw the virtual world projected on large wall in front of it. Musicians received cues from the dancer's interaction with the environment, which also triggered electronic audio events. The installation version uses recordings of the music to construct an aural landscape where the navigator's position and motion interact with audio signal processes. Visual and audio wayfinding devices that were not needed by the dancer provide cues to non-expert navigators. Dembski and I have discussed the possibility of producing yet a third work from the installation, a DVD that would be an "expert" navigation of the installation, in which audio in particular is very different from the first performed version. As a composition, *Fools Paradise* is grounded in abstract notational structures that help to determine its visual architecture, musical material, and event-flow. In other words, it is an intermedia artwork, where events in one sensory modality may be mapped onto events in another modality. At the same time, different media are left free to display their own unique qualities, without strict mapping. The degree of mapping at any given point is part of the compositional process. This approach to composition is peculiarly apt for digital technology, which allows compositional structures to operate at all levels of granularity and with a degree of abstraction that supports the construction of cross-modal mappings. In responsive environments, performances can attain an unprecedented degree of precision and synchronicity of events. Intermedia composition with a formal system of this type also permits collaborative artists to develop a pool of common elements, even when they work in different sensory modalities. Dembski used as part of his compositional material sets of pitch-classes derived from a graph from which the forty-eight jewels were also derived. The jewels mapped the different pitch-classes to their colors, but the graph was not used in the virtual world: it served as compositional scaffolding. Such correspondences are not meant to be directly perceived by the audience, but they do enable the compositional process and can provide a formal unity that contributes to the overall effect of a multi-modal artwork. Although Steve Dembski and I embraced technology to realize our work, we were acutely aware of the twin necessities of approaching technology from a critical point of view and of engaging culture as a historically rooted enterprise. Blake's *Marriage of Heaven and Hell* presents a critique of the tyranny of religious institutions and a plea for personal freedom of belief that resonates today. We honor Blake as a multimedia artist *avant la lettre*, who melded poetic language with printed and painted images in his books. The explicit use of an expert performer, the blending of live acoustic music with electronics, and the use of hand-painted texture maps in the VR landscape all were meant to point to the continuity of human presence at the core of culture. As artists, first of all we create experiences - not precious objects or reified cultural values. We place ourselves within tradition not to ensure approval of our productions but to challenge our present moment as Blake and other artists challenged their historical moments. IN CLOSING, AN APERTURE The rationales for virtual reality as a performance medium and for expanding its sensory modalities and repertory of responsiveness well beyond the simulation of 3D environments are not unique to the medium. In all electronic communications media, "[our] primary goal should be to prevent closure from occurring too quickly, before the possible has an opportunity to deploy the variety of its richness" [11]. In media where the boundary between reader and author has dissolved or assumed a fractal complexity, closure often means a return to one-sided communication and models of cultural dominance. This is particularly the case with broadband technologies on the Internet, heavily weighted to push information rather than pull it. It may seem a forlorn and absurd gesture to open up the artistic possibilities of a medium in the face of the massive systems of fear and domination that loom in our possible futures, but the sum of such gestures can assume a profile of hope, an embarkation to other destinies. _____________________________ REFERENCES 1. Steve Bryson et al., *Implementing Virtual Reality*, SIGGRAPH Course Notes (1993), Introduction, p 1.1.1. 2. Hisham Bizri et al., 1996-97. http://www.evl.uic.edu/art/art_project.php3?indi=32 3. Drew Browning, 1997. http://www.evl.uic.edu/art/art_project.php3?indi=139; http://bucky.aa.uic.edu/DVL/drew/isea/ 4. Geoffrey Baum et al., 2005. http://meta.appliedinteractives.com/stp/ 5. William R. Sherman and Alan B. Craig, *Understanding Virtual Reality*, San Francisco, CA; Morgan Kaufmann Publishers, 2003: pp. 332-362. 6. Romedi Passini, "Sign-Posting Information," in *Information Design*, Robert Jacobsen, ed. Cambridge, MA, The MIT Press, 2000. 7. Cf. *Legible City* (Jeffrey Shaw, 1990) and similar projects documented in Stephen Wilson, *Information Arts*, Cambridge, MA, The MIT Press, 2002: chapter 7.4. 8. Diane Gromala, "Pain and Subjectivity in Virtual Reality," in *Clicking In*, Lynn Hershman Leeson, ed. Seattle, WA, Bay Press, 1996. 9. Paul Hertz, "Form, Substance, Correspondence: Intersensory Composition in Digital Media," Siggraph 2001 *Electronic Art and Animation Catalog*, full text online at: http://collaboratory.nunet.net/phertz/portfoli/pdf/formsub.pdf 10. Yacov Sharir, http://www.art.net/~dtz/sharir1.html 11. Pierre LŽvy, *Collective Intelligence: Mankind's Emerging World in Cyberspace*, Robert Bononno, trans., New York and London, Plenum Press, 1997: p. 122. RELATED URLS http://collaboratory.nunet.net/phertz/portfoli/ http://collaboratory.nunet.net/phertz/portfoli/pdf/ignotheo.pdf http://collaboratory.nunet.net/phertz/paradise/ http://collaboratory.nunet.net/phertz/paradise/foolprod.html *Fools' Paradise* is also documented in the Rhizome Artbase, http://www.rhizome.org. Fools' Paradise was first performed May 1, 2004, with Juliet Petrus, soprano; Mackenzie Danner, flute; Danah Albaum, cello; Jessie Seetoo, VR Performer; and Matt Cozza, Stage Manager. AUTHOR BIOGRAPHY Paul Hertz is an intermedia artist working in digital and traditional media. Multimedia applications designer, Collaboratory Project, Northwestern University, USA. Independent curator, currently associated with Block Museum, Northwestern University. Co-Director, Center for Art and Technology (CAT), Northwestern University, 2003-04. Visiting Artist, CAT, 2001 and 2002, teaching virtual reality technology for artists. Work exhibited in numerous new media festivals: Siggraph, ArCade, ISEA, Biennial de Buenos Aires. Early work while residing in Spain (1971-1983) includes intermedia collaborations with musicians and theatrical performers. LEONARDO ABSTRACTS SERVICE DATABASES (LABS) Leonardo is pleased to announce the most recent top-rated thesis abstracts from the English-language LABS database, hosted by Pomona College, CA, USA. The top-rated authors are chosen by a panel of peer-reviewers under the direction of Sheila Pinkel. More about the English language LABS database and how to submit material can be found at: http://leonardolabs.pomona.edu < It's in Your Eyes - Gaze Based Image Retrieval in Context > By Lasse Scherffig ABSTRACT: The interface through which visual media are accessed are the eyes of the observer. In turn, the interface through which a medium may access its observer are her eyes, too. EyeVisionBot uses this input/output duality of vision by incorporating gaze into a computer interface. Its aim is allowing users to search for images by looking at images. EyeVisionBot has been build as an interactive art installation for the media museum of the ZKM | Center for Art and Media. The base on which the system is build comprises eye tracking technology, content based image retrieval and interaction design. The thesis "It's In Your Eyes", however, does not solely cover the system in terms of hardware, software and design but pays much attention to both its cultural context and impact. To do so, it is organized in three chapters carrying the titles "art", "science" and "interface". Each chapter contains a section on the historic role of eye tracking in the particular field and two more sections discussing relevant ideas and developments from those fields that touch and are touched by EyeVisionBot. Among these are discussions of the role of interactivity in media art, of closed circuit art, of the idea of objective information aesthetics, of current trends in information retrieval and visualization and finally a broad discussion of the interface concept. The latter first is defined recurring on several relevant authors (such as Donald Norman, Brenda Laurel, Terry Winograd and Matthew Fuller) paying special attention to the differences between their respective theoretical conceptualizations of the interface. This analysis discloses a trajectory reaching from seeing interfaces as a point of psychophysical interaction towards a designed surface, a space for communication and finally interface as everything. Subsequently, Michel Foucault's book "Discipline and Punish" serves as the theoretical background on which Human Computer Interaction is redefined and criticized. The computer science practices of creating scenarios and use cases as well as the method of user modeling, it is argued, model users in the sense of defining their behavior. Interfaces, in fact, generally do so and generally may be identified with Foucaultian cases of discipline and punishment. Systems of algorithmic surveillance in this course are put into relation to interfaces in general and EyeVisionBot in particular. In spite of the structural division into three main topics numerous links between distinct parts of the thesis are drawn. Closed circuit art (by Bruce Nauman and Peter Weibel), for instance, serves as the link between the scientific ideas of traditional cybernetics, media art, surveillance and Human Computer Interaction. Date Submitted to Database: 21 July 2005 _______________ < Operating In Media Reality - For A Critical Practice in Interactive Images > By Samuel Bianchini ABSTRACT: How do individual and collective interactive imaging practices entail new methods of representation, production, and use, and does it change our relationship to a reality that is increasingly mediated by technology? With interactivity, we become the "users" of media, in particular of images. Activating the image and becoming active in front of the image: representation no longer occurs just on the screen, but rather it is also played out before it and beyond, in relation to the images and in the relationships that they mediate. Coupling operation with representation and physical reality with symbolic reality, at the confluence of devices and uses, interactive images, whether local or in a network, are caught up in an array of activities and operations of different nature: human, both individual and collective; and technological, software and hardware. As representation, the image is a tenant-lieu [it takes the place of something]; when interactive, it is also a donnant-lieu [it makes something take place]. Modes of operation and representation engage a work shared between the initial author and the spectator, between "formalizing" and "performing," between the "information-image" and the "experience-image." On the basis of our artistic research and those of other artists, two forms of practice are analyzed with a view to better understand and situate them, to generate new ones, and to learn to lay down conditions that are propitious to a critical practice in interactive images. Far from simplistic communicational or cybernetic models and reassessing the status of the recorded sequence of images, the interactive image as a plan of action, of representation, and of reflection can entail exercising consciousness in accordance with a time and representation depth, a "bathmology." A node of varying activities, this image configures an environment and stages power struggles, affective and effective participation, and tactics and strategy. In sum, it forms a theater of operations, which, once situated, localized, interconnected, shared, distributed, and disseminated, defines a mediated reality to be grasped and invested. Date Submitted to Database: 25 July 2005 _______________ < Improvisatory Music and Painting Interface > By Hugo Sol’s Garc’a ABSTRACT: Shaping collective free improvisations in order to obtain solid and succinct works with surprising and synchronized events is not an easy task. This thesis is a proposal towards that goal. It presents the theoretical, philosophical and technical framework of the Improvisatory Music and Painting Interface (IMPI) system: a new computer program for the creation of audiovisual improvisations performed in real time by ensembles of acoustic musicians. The coordination of these improvisations is obtained using a graphical language. This language is employed by one "conductor" in order to generate musical scores and abstract visual animations in real time. Doodling on a digital tablet following the syntax of the language allows both the creation of musical material with different levels of improvisatory participation from the ensemble and also the manipulation of the projected graphics in coordination with the music. The generated musical information is displayed in several formats on multiple computer screens that members of the ensemble play from. The digital graphics are also projected on a screen to be seen by an audience. This system is intended for a non-tonal, non-rhythmic, and texture-oriented musical style, which means that strong emphasis is put on the control of timbral qualities and continuum transitions. One of the main goals of the system is the translation of planned compositional elements (such as precise structure and synchronization between instruments) into the improvisatory domain. The graphics that IMPI generates are organic, fluid, vivid, dynamic, and unified with the music. The concept of controlled improvisation as well as the paradigm of the relationships between acoustic and visual material are both analyzed from an aesthetic point of view. The theoretical section is accompanied by descriptions of historic and contemporary works that have influenced IMPI. Date Submitted to Database: 30 August 2005 _______________ < Doctor Photo: The Cultural Roots Between Portrait Photography as Medicine in Nineteenth Century America > By Tanya Sheehan ABSTRACT: This dissertation investigates the cultural roots of an analogical relationship between commercial portrait photography and medicine in nineteenth-century America. Using the methodological tools of visual, cultural, and science studies, I examine professional photographers' reliance upon medical models to describe their operations in the portrait studio. Through a series of historical case studies, I show that photographers appropriated medical discourse in an effort to strengthen their professional legitimacy. What was at stake in this effort, however, far exceeded the establishment of commercial photography as one respected profession among many. Representations of photography as medicine, I argue, shaped the institutional and epistemological character of portrait photography by defining photographers' field of operations as the physical and social health of the middle class. In a period when the social body appeared particularly vulnerable, commercial photographers attempted to construct a cultural belief in photography's rehabilitative powers and authority over the body. My study of photography as medicine focuses on the cultural context of Philadelphia, a city which witnessed the professional advancement of both technologies in the nineteenth century. I begin by discussing the kind of social status and institutional structures which the photographic profession sought to borrow from medical models. Chapter 2 examines representations of portrait photography as a diagnostic way of seeing and a normalization of the body analogous to operative medicine. I go on to consider in chapter 3 how photographic literature ironically transformed the serious health risks associated with photographic chemistry into an occasion for praising the unparalleled healing powers of photographers, their materials, and the chemical environment of the photographic laboratory. In chapter 4 I show that portrait photography's reliance on light lent the medium further potential as a panacea. Like popular phototherapies, photography promised to rehabilitate the social identities of studio patrons by removing bodily deviations from an ideal of whiteness and bourgeois respectability. I conclude with a brief discussion of how my historical project intersects with debates concerning body-imaging techniques in contemporary American medicine and popular digital culture. Date Submitted to Database: 19 September 2005 ______________________________________________________ LEONARDO REVIEWS October 2005 ______________________________________________________ This month in Leonardo Electronic Almanac we are featuring Stefaan Van Ryssen's review of Eating Architecture. As Ryssen says on reflection, food and architecture do go together quite well in fact it my view that art and food appear to be almost inseparable in today's culture of curation as knowledge management since it provides an obvious opportunity for further entrepreneurial intervention. Food and art, and now architecture are inseparable as the cooked and the raw or so it seems. As a companion to Ryssen's breezy informative review we have featured Pia Tikka's reaction to Image Retrieval: Theory and Research by Corinne Jšrgensen. Her review is a thorough and equally thought provoking reflection on a manual which in her words is an "excellent introductory text to the field of image processing." Finally in our featured selection Mike Mosher elegantly juggles the rather dense Indian clubs that comprise Avital Ronell's new book, The Test Drive. In Mosher's view it is not a book for the beach, and it seems from his review, that it sits somewhere between the insightful speculations of Eating Architecture and the solid user's manual that Tikka has chosen to review. Fortunately we have their overviews on line at Leonardo Reviews to help us decide how much more attention we need to pay to these new books. As usual all the new reviews posted this month along with the archive are available at http://leonardoreviews.mit.edu Michael Punt Editor-in-Chief Leonardo Reviews ________________________________ REVIEWS POSTED NOVEMBER 2005 9th RAI Festival of Ethnographic Film Reviewed by: Aparna Sharma Aki Peltonen by Radio Banana Reviewed by Michael R. (Mike) Mosher Computers and Art by Stuart Mealing, Ed. Reviewed by John Knight Don't Make Me Think: A Common Sense Approach to Web Usability by Steve Krug Reviewed by Kasey Rios Asberry Eating Architecture by Jamie Horwitz and Paulette Singley, Eds Reviewed by Stefaan Van Ryssen Histories of the Future Daniel Rosenberg and Susan Harding, Eds. Reviewed by Kathleen Quillian Image Retrieval: Theory and Research by Corinne Jšrgensen Reviewed by Pia Tikka Lara Croft: Cyber Heroine by Astrid Deuber-Mankowsky. Trans. by Dominic J. Bonfiglio Reviewed by Dene Grigar The Test Drive by Avital Ronell Reviewed by Michael R. (Mike) Mosher Eating Architecture by Jamie Horwitz and Paulette Singley, Eds The MIT Press, Cambridge, 2004 385 pp., illus. 97 b/w. Trade, $39.95 ISBN: 0-262-08322-1. Reviewed by Stefaan Van Ryssen Hogeschool Gent Jan Delvinlaan 115, 9000 Gent, Belgium stefaan.vanryssen@pandora.be It is not an obvious subject, but once you've come to think about it, the combination isn't surprising either: food and shelter are as essential to the development of civilization as fiber and fuel. Cooking and building both imply the transformation of (raw) materials applying energy while following rules to reach a final result: a meal or an inhabitable space. Time scales may be different, but if there is anything like coherence in culture, both activities must have at least some symbolic, structural, or metaphorical relationship. And that is exactly what the authors of this collection of essays are exploring or proving. Jamie Horwitz of Iowa State University and Paulette Singley of Woodbury University serve the meal in four courses. In Place Settings, the connections between food and locale are explored. Each essay looks at food from a different angle: the locality or globality of its production, regional culinary identities, the "consumption" of the colonies and the international tourist taste. In Philosophy in the Kitchen "the cleansing, cutting, and cooking of food form a routine that also doubles as a site for aesthetic experimentation. By drawing gastronomy out of the kitchen, the essays that follow shift the discussion toward the performative space of eating--a site that is inherently unstable, mutable, mobile and memorable" (p. 16). Table Rules, with its striking reference to Claude LŽvi-Strauss' magnum opus, The Origin of Table Manners, effectively honors the founding father of structuralist anthropology without copying his themes or imitating his approach. It is in these five contributions that the close connections between practical day-to-day architecture and interior design and the social and cultural meaning of food are analyzed. Watch out for 'Food to go: the industrialisation of the picnic' by Mikesch Muecke when you are next victimized by your fast food giant or when you're committing another take away. Embodied Taste, finally, targets the taste buds and its counterparts in the other senses. This is where art, architecture meet gastronomy and food production. Of course, George Battaille and Damien Hirst must pass in revue, as do Dal’ and Francis Bacon. All in all, I found the essays in this collection of uneven quality but almost all of them inspiring and certainly thought provoking. I found it difficult to stomach Donald Kunze's extravaganza on the Missing Guest but Susan Herrington's cultural and culinary portrait of Canada is both palatable, hilarious, and wise. Daniel S. Friedman's 'Cuisine and the Compass of Ornament: A Note on the Architecture of Babette's Feast' offers a grandiose reading of this intriguing film and is as clear and sparkling as a glass of spring water and the closing essay by editor Paulette Singley made me think again of marble and pork and why I am not disgusted at either. _______________ The Test Drive by Avital Ronell University of Illinois Press, Urbana and Chicago, 2004 356 pp., illus. 29 b/w. Trade, $35 ISBN 0-252-02950-X. Reviewed by Michael R. (Mike) Mosher Saginaw Valley State University, University Center MI 48710 USA mosher@svsu.edu The density, oh, the density! One mustn't pick up a book by Avital Ronell expecting breezy summer beach or front porch reading. The Test Drive is a curriculum, whose allusions and citations shoot the reader in numerous directions. One does not know it as a chemist might an unknown substance, by scraping off a few chips and running them under a battery of (intellectual, conceptual) reagents. The book demands to be unpacked over numerous re-approachings, so any reader of this review might question how deep an artist's reading of a big book by philosopher Ronell can go. Ronell states, "This work originally grew out of study devoted to Alan Turing". Turing sought consistency, completeness, and decidability: This computer has convinced us it is human; this computer can distinguish all dogs from all cats. The author seeks to interrogate the history of science and assumptions we live by, from hypothesis to scientific test to conclusive results. The basis of testing is provisional, the hypothesis: this might work. Yet despite experimentation, hypothesis, probability, and proof interrogated, the result may be a permanent dislocation and decentering. Ancient Greeks invented the test in the form of torture, when the seeds of scientific method saw Socrates seeking rigor in philosophical proofs. Freud's "reality test" designated what belonged inside and what outside the psyche, endeavoring to prove what went where. Ronell proceeds to draw from Plato, Nietzsche, Popper, Derrida, Lyotard and Levinas. Beneath philosophers, literary allusions are inserted like tesserae in a mosaic, from Frankenstein, Kafka's "In the Penal Colony", Pindar's Odes and others. Humanity becomes tangible through theory where real life is an endless library. Nietzsche's physics are ascertained by test or temptation, yet experience frees us from referential truth with its codes of refutability and restraint. Ronell quotes approvingly from Nietzsche's Beyond Good and Evil to upbraid the social tendency towards oversimplification, a vice for which she is certainly not guilty. Ronell's politics flicker, difficult to discern. Burglary leads to circumcision, and then greater intrusions; the hegemonic state as the poison apple in "Sleeping Beauty". "Who needs a cell when the state can get inside your body and resignify your cellular structure?", she asks. Holocaust-denier Robert Faurisson's demand for "proof" that the event really happened butts up against postmodernist indeterminacy. She finds a "Trial Balloon" in von Husserl, appreciates Levinas's critique of positivism, dances with Heidegger and Arendt. Ronell examines herself as she endures discomfiting medical procedures and the disconcerting wait for results and admits her fixation upon Frederick Nietzsche. "Supposing Philosophy is a woman - what then?" fretted mad Frederick. Ronell's penultimate chapter "Testing Your Love, or Breaking Up" personalizes her imaginary relationship to a long-dead philosopher, much as Susan Sontag did with her own, to Walter Benjamin, in "Under the Sign of Saturn". What is the best form for a philosophy book? The Test Drive sees its text broken into sections suggesting laboratory reports: "Testing 1", "Testing 2", "Prototype A", "Prototype B", "Prototype .01""Prototype .02", "Prototype 1.0". Nostalgically cyberpunk, the conceit brings to mind the late-1980s all-caps ASCII manifestoes of California artist Walter Alter. Ronell treats philosophy like a dance track, a DJ scratcher's sequenced voice commanding "Start over". She digresses, repeatedly adds Case One, starting over each time, not building upon results. Is each case, therefore, sui generis? She enjoys a postmodernist authorial freedom of subjective, fluid text. Perhaps she needs a blog? She writes one here, one with the density of five books. Chapters are introduced with an arrangement of Suzanne Doppelt's mysterious photographs, often like a decorative screen separating a room. Occasionally the reader stumbles into curious graphics embedded in the text, seemingly random pictos and motifs derived from warning signs. These add a certain 1980s industrial cool and further ambiguity as one tries to bridge meanings in picture and text. The word "AMGINE" appears, Enigma spelled backwards, which may allude to Turing's wartime codebreaking project. Or perhaps a contraction for a command to the imagination. Or perhaps a term for its imagination's lack, like asexual or ahistorical? Look how it evokes "Engine" too. But ultimately, what of it? Why are there candies stuck in this meat? How does the visually punning amusement advance her argument, or multiple arguments? There are uncaptioned layouts of microphotography, and a sculpture of eighteenth-century scientist Robert Boyle in both positive and portentously negative full-page images. One wonders if designer Richard Eckersley was given the sole directives to make it look cool, lab-like, deep, opaque. The book's title, The Test Drive, evokes the suburban Detroit proving grounds of American cars. Beyond bookending experiences in Michigan receiving and giving academic testing, this reviewer has experience observing user tests of products-computer hardware, productivity software, instructional books and interactive training-of a California technology manufacturer. The humbling epiphany is that any aspect of the computer experience seeming so clear and intuitive to its designers can baffle the majority of fresh users who encounter it, and must subsequently be rethought and redesigned. Ronell certainly has spent time reading of Nietzsche to provoke his effects upon her and her own effusive thoughts. Yet I'm not sure how much time the author has spent in testing or laboratory settings, or read scientific journals recounting procedures and results. I wonder if the University of Illinois Press has sent this volume to chemistry or biology journals to see how it plays among scientists. Whatever those results, this difficult encounter, with its mixed results, still makes this reviewer want to pick up Avital Ronell's earlier books Crack Wars, Stupidity, and The Telephone Book, if only for a test drive. _______________ Image Retrieval: Theory and Research by Corinne Jšrgensen Scarecrow Press, Inc.. Lanham, Maryland, and Oxford, 2003 360 pp. Trade, $47.00 ISBN: 0-8108-4734-5. Reviewed by Pia Tikka Researcher in University of Art and Design HŠmeentie 135 C, 00560 Helsinki, Finland Pia.Tikka@uiah.fi Image Retrieval: Theory and Research is a book, which serves, like its author Corinne Jšrgensen emphasizes, as a foundation for individuals in research communities, who need an introductory overview on the issues of image access, indexing, and retrieval. It becomes clear along reading this excellent book that it is intended for experienced readers: not only taken the quite fast "first-aid" coverage of the exhaustive fields that relate to image retrieval and systems design, such as the domains of cognitive psychology, the psychology of art, computer science, content-based image retrieval (CBR), and library and information science (LIS), but also due to the compactness of the information presented. It is made clear that the cross-disciplinary user-centered approach is a must in future research. The large body of literature concerning data models, pattern and feature recognition, database architectures, and machine learning and image understanding through applications of artificial intelligence, Jšrgensen notes, have concentrated on what is computationally possible, but not on the analysis of the needs of real-world users. The support outside of the scope of the book could be found in Andy Clark's [1] discussion on the inseparable reciprocal evolutionary relationship with the human mind and techno- environment, i.e. "specially constructed environments replete with artifacts, external symbols, and all the variegated scaffoldings of science, art and culture". Jšrgensen suggests unifying research technology, methods and goals that facilitate "a common, sensible, and easily understood framework" (p. 199) for the future interdisciplinary research community. Jšrgensen's preliminary research interest lies on identifying and categorizing image attributes in user-queries and image-indexing [2]. The research draws from the human tendency for "similarity" and "critical difference" judgment, which provides plausible tools for image feature categorizing. The perceptually stimulated attributes, such as the human figure, objects, color, and location, are typically recognized in images. Jšrgensen's research concludes that the contextual situatedness of each individual, be it a user or an indexer, fundamentally enables or constrains the image classification and management processes. Emotional moods, personal motivations, likes and dislikes, life-experience etc. affect the descriptions and feature attributes given to the images when indexing, and, in turn, to the information requested when searching images. Thus, an ecological view [3] surfaces towards the end of the book. Jšrgensen's wish list for the future research involves following: a test bed for comparative evaluation, which would incorporate large number of images from diverse domains (science, arts, consumers etc.); a visual indexing language for bridging the gap between, for example, the automated lower-level feature-based image-content analyses and the higher-level semantic content of images; an integrated indexing record accommodating multiple types of data from both human and machine process; widening communities of research and practice juxtaposing the researchers' experiences and interests from the domains of the CBR and the LIS. Final but not least in the researcher's wish list is a request for a new starting point: Understanding the needs of an end-user and her desired methods of interaction should guide creation of "real-world" applications, namely usable, flexible interfaces. The critical notion of the content of the book draws from the apparent two-facet problem of visual and textual information. Though the book emphasizes the research orientation towards the integration of visual and textual knowledge, it deliberately ignores this in the content of the book itself. In my view, both for less experienced readers and for researches not familiar with the computational image-analyzing systems, the access to the image indexing and retrieval topic might have become more feasible with some visualization. Use of images could have added to the discussion the actual dimension that the author's approach emphasizes, the visual knowledge and embodied understanding as an unexplored resource of image retrieval systems design. In concluding, I recommend the book as an excellent introductory text to the field of image processing. Jšrgensen's overview on image retrieval, i.e. human-generated and machine-processed practices and theory, makes an important contribution for the designers of a globally accessible image management. It seems, though, that the success or failure of the future research fundamentally relies on the application of what the Western science has become to know as embodied knowledge. References: [1] Clark, A., "Reasons, Robots and the extended Mind". In: Mind & Language, Vol. 16 No. 2 April 2001: pp. 121-145. (p. 142). [2] Jšrgensen, C., "Image Attributes: An Investigation", Ph.D. thesis, Syracuse University, 1995. [3] With ecological approach is referred to the idea that the organism evolves in continuous interaction with its environment. Here the reviewer assumes that the author relates with the view of Gibson, J.J. 1979. The ecological approach to visual perception. Houghton Mifflin, Boston. LEONARDO, VOL. 39, No.1 (FEBRUARY 2006) TABLE OF CONTENTS AND SELECTED ABSTRACTS Editorial < The Leonardo Scientists' Working Group > by Tami Spector _______________________ After Midnight < Spicy Little Things from Budapest > By Erika Katalina P‡sztor _______________________ Artists' Statements < The International Transentient Cartographicacy Project > By Kathleen Quillian and Gilbert Guerrero < A Walk on the Retinas of the Extreme Sciences: A Minimalist Concept for an Internet-Based Light Art FaŤade in Munich > By Tim Otto Roth < Generative Flowers as a Language of Forms for the Visualization of Binary Information > By Philip Van Loocke _______________________ Special Section: Space Art 2004 < From MŽliŹs to Galaxy Quest: The Dark Matter of the Popular Imagination > By Michael Punt, Martha Blassnigg and David Surman ABSTRACT: This paper was stimulated by the film Galaxy Quest (Parisot, 1999), which collapses the fictitious dimensions of space travel into the scientific perception of real space travel in order to reveal some insights into the cultural construction of space. The paper argues that an interrogation of cinema can reveal the fragility of our knowledge and the underlying imperatives that the social construction of space responds to. A revisionist overview of the issue of professional interfaces in the popular arts is followed by a discussion of the influence of space technology and natural space phenomena on human personal and collective belief systems in order to open the way for an outline of the concept of participatory cultures and the relationship between fiction and science. < What Is the MARS PATENT and What Does It Do? > By Claudia Reiche and Helene von Oldenburg ABSTRACT: The authors invite readers and others, including aliens (provided they claim to have female first names), to submit "things" to the MARS PATENT project for interplanetary exhibition on Mars and on the Internet. The MARS PATENT High Reality Machine will teleport sculptures, theories, web art and other things, imaginable or not fully imaginable, to the exhibition site on the red planet. The authors have also established the Oldenburg-Reiche Prize, an open competition challenging artists, scientists and others to come up with a satisfying explanation for how the High Reality Machine works. < Astro Black Morphologies: Music and Science Lovers > By Flow Motion ABSTRACT: A compressed series of possible histories of science in modern music, the text outlines the themes of poetic and historic correspondences between music, cosmology and the body that informed the making of Astro Black Morphologies/Astro Dub Morphologies, a multimedia installation and live sound-art performance by Flow Motion in which data from possible black hole Cygnus X1 is transformed into an immersive electronic sound-and-image environment _______________________ Theoretical Perspective < Infinity and Accident: Strategies of Enfoldment in Islamic Art and Computer Art > By Laura Marks ABSTRACT: Computer art and Islamic art, the two largest bodies of aniconic art, share a surprising number of formal properties, two of which are explored here. The common properties of computer art and classical Islamic art can be understood in light of moments in the history of Islamic philosophy. In these two cases, Islamic Neoplatonism and Mu'tazili atomism are shown to parallel, respectively, the logic of relations between one and infinity, and the basic pixel structure, that inform some historical monuments of Islamic art as well as some contemporary works of computer art. It is suggested that these parallels are in part a result of Islamic influences on Western modernism and thus that the genealogy of computer art includes classical Islamic art and the philosophies that informed it. _______________________ General Articles < Internet Artworks, Artists and Computer Programmers: Sharing the Creative Process > By Jean-Paul Fourmentraux ABSTRACT: Internet artwork no longer refers to the concept of a finalized object, but rather to a dynamic process, a collective, open and interactive device. Due to the increasing sophistication of tools, the design of an Internet artwork now requires hybrid skills. The necessary cooperation with computer specialists in order to create suitable programs thus changes the status of the artwork and its author. This paper presents an ethnographic case study of cooperation between a computer programmer and an artist. It examines the processes of shared design, negotiated authorship and artwork appropriation. From an analysis of the means of communication, various technical media and "intermediary tools," the author focuses on role allocation, task sharing and artwork appropriation as the artwork is modified throughout the creative process. < An Artist's Works through the Eyes of a Physicist: Graphic Illustration of Particle Symmetries > By Gyšrgy Darvas and Tam‡s F. Farkas ABSTRACT: The paper presents remarks by a physicist and a graphic artist on an artwork series produced by the artist. They associate the colors and twists represented in these graphics with the properties of subatomic particles---their structures and connections. The authors use graphical representation to visualize the inner structure of atoms, the classification of quarks and the metaphorical names of abstract physical properties. No textbooks that make visible these basic properties by means of art are currently available. Artistic visualization brings these "mysterious" physical objects closer to the understanding of students and the general public and leaves physicists better able to discover new secrets of the internal structure of quarks and their properties. _______________________ Special Section: Melilla Colloquium < The Philippine Triad and Western Dichotomous Philosophies: A Contest of Traditions in Three Audio-Performance Projects > By Fatima Lasay ABSTRACT: The author's three projects draw upon her analysis of a triumvirate of social roles in ancient Philippine society, especially in connection with the maintenance of ritual in that society. Each project addresses the historical function of ritual as well as how it is carried over into art. < Technoetic Pathways toward the Spiritual in Art: A Transdisciplinary Perspective on Connectedness, Coherence and Consciousness > By Roy Ascott ABSTRACT: The coherence of living systems may be due in part to an information network of biophotons emitted by DNA molecules. This network can be seen as parallel to the telematic networks that connect the planet. Nanotechnology can play a significant role in the emergence of a moistmedia substrate for technoetic art. Immaterial connectedness confers a spiritual dimension on both telematic art and quantum mechanics. Field theory supports the contention that the material body may be a consequence rather than a cause of consciousness. A technoetic art may locate its ground in the triangulation of connectivity, syncretism and field theory. < Contemplations on Our Physical Links to the Universe: Searching for and Finding the Hidden Harmony > By C.S. Unnikrishnan ABSTRACT: The author discusses the evidence and consequences of our indissoluble physical links to the entire universe. He finds that the apparent conflicts in fundamental physical theories regarding issues of causality and locality are not real conflicts based in the physical world. He presents an emergent worldview interpreted in the context of a cultural, philosophical and linguistic background in which a strong tension between inseparability of the whole and the local causal flow of events seems not to exist. The existence of the whole can be felt in the parts, as something real, measurable and undeniable. Its simplicity and harmony are spiritually enriching and emotionally moving. < An Open Letter to the Melilla Conference Participants > By Chris Alexander _______________________ Leonardo Reviews Reviews by Jan Baetens, David Beer, Roy Behrens, Martha Blassnigg, Andrea Dahlberg, Dene Grigar, Rob Harle, Amy Ione, Michael Punt, Kathleen Quillian, Eugene Thacker, Stefaan Van Ryssen LEONARDO NETWORK NEWS ______________________________________________________ < Common Systems: The Invisible Dynamics of the Pacific Rim and the Bay Area > San Francisco's Exploratorium and the San Francisco Art Institute are pleased to participate in the ISEA 2006 Pacific Rim New Media Summit through the presentation of some preliminary findings of its Invisible Dynamics research project using a panel format. They hope to make contact with individuals or organizations from around the Pacific Rim who may have related projects or research who would be interested in discussing possible points in common. The Exploratorium is planning to move to a new site along San Francisco's northern waterfront not far from the San Francisco Art Institute. To better understand the context of this area, The Exploratorium is collaborating with four SFAI programs: the Center for Art+Science, the Center for Media Culture, the Center for Public Practice and the Center for Word, Text, and Image. It has become clear that an all-encompassing view of the new neighborhood in relation to the city requires consideration of its relationship with the Bay and also with the Pacific Ocean. Moving from a view of the neighborhoods surrounding The Exploratorium's potential new home to the city of San Francisco in relation to the Bay region and to the Pacific Ocean presents a scale allowing a view of the multiple interactions and relationships that give this place its character. To investigate them, the Exploratorium forming partnerships with artists, scientists, scholars, researchers and practitioners and plans to include more institutional partners as its direction becomes more sharply defined. Five research projects are currently underway, including Hidden Ecologies, a micro/macro/biological and cultural cross-referencing of physical areas of San Francisco Bay; Trace, an exploration of the evolving urban wireless ("Hertzian-Landscape") networks of San Francisco; Cabspotter, an investigation of the social and electronic trails of a Bay Area taxi system; Move Here, a study of contemporary and historic strategies for compelling people to move to the Bay Area; and a piece installed by a team of Art Institute student researchers. When the Exploratorium and the Art Institute try to imagine what it will be like here in the future, many of the clues seem to come from farther west. The city's future, intertwined relationships with the communities and cultures of the Pacific Rim will define its way of life for many years to come. It may be instructive to look at and try to understand some of the dynamics of the Bay region as a step toward understanding the complexities of the systems that define the Pacific Rim. The Exploratorium is interested in sharing experiences about related projects around the Pacific Rim and looks forward to hearing from people or institutions that are developing similar viewpoints, in particular at the ISEA Conference. Contacts: Susan Schwarzenberg , co-chair; Peter Richards , co-chair; Meredith Tromble ; Paul Klein . _____________________________ < Design As Seeing As Thinking > The YASMIN art/science network (a Leonardo partner under the Leonardo Global Crossings initiative) has endorsed the Design as Seeing as Thinking Workshop at Kadir Has University in Istanbul, Turkey, 27 February--5 March 2006, and has encouraged interested young instructors or Ph.D. students to apply. http://www.media.uoa.gr/yasmin/ Kadir Has University 6th International Istanbul Interaction Design Workshop Young Instructor Training Program DESIGN AS SEEING AS THINKING Curator: Prof.Dr. Oguzhan Ozcan, Yildiz Technical University Supervisor: Prof. Roberley Bell, Rochester Institute of Technology http://www.idws.info Aim and Scope: The focus of this workshop will be to enhance the quality of teaching by young instructors in interactive media design through developing their knowledge base of the principles of design and the process of problem finding and problem solving. A series of design problems will be explored using local as a point of departure. Participants will explore the physical, geographic, cultural region of Istanbul and through systems of observation and methods of recording information address cross cultural interpretations of place. The design problem and process will be linked to methods and strategies for improving their skills in the classroom. Background of Applicants: We are looking for 21 highly talented young instructors or PhD Students who currently teach in basic design course at Department of Communication Design, Interactive Media Design, Multimedia Art or relevant art and design disciplines. Selection Process: We will select 7 applicants from Mediterranean Rim, 7 from Turkey and 7 from other countries. Important deadlines: Early application: November 30 Regular application: December 16 Late Application: January 6 Announcement of Participant: January 16 How to apply 1. Prepare a web site that includes your CV/resume and examples of your work (please do not send any PDF documents; only a web site is required). 2. Pay application fees Early application fee until November 30 10 EUR for Europe & Med.Rim, 10 YTL for Turkey 15 USD for others Regular application fee until December 16 20 EUR for Europe & Med.Rim, 20 YTL for Turkey 25 USD for others Late Application fee until January 6 50 EUR for Europe & Med.Rim, 50 YTL for Turkey 60 USD for others 3. Methods of payment: Bank transfer Transfer the money to: The recipient's name: Kadir Has University Garanti Bank Balat Istanbul Brunch Swiss Code: TGBATRISXXX Account Number: YTL 020 6299846 Account Number: USID 020 9098884 Account Number: EURO 020 9098254 4. Send your web site link and money transfer information to application@idws.info Without your bank transfer information, your application will not be processed. 5. In Subject line, type [workshop]. We recommend that you read the details of the workshop carefully before filling out the application. 6. We would like to remind all candidates to make their flight reservations at once considering the heavy winter traffic. _____________________________ < swissnex Joins Leonardo Organizational Membership Program > Leonardo is pleased to welcome swissnex to the Leonardo Organizational Membership Program. swissnex, under the current direction of Executive Director Christian Simm, observes, analyzes and interprets trends between people, countries and projects to promote science, education, art and innovation between Switzerland, the western United States and Western Canada. The Leonardo Organizational Membership Program, initiated in 2004, opens up new avenues for Leonardo Network members to work with the educational and corporate communities. Current organizational members include Intel, the San Francisco Art Institute, the Art Institute of Chicago, Rhode Island School of Design and the University of Texas. More information about swissnex can be found on their web site: . More information about the Leonardo Organizational Membership Program can be found at: . _____________________________ LEONARDO NETWORK NEWS COORDINATOR: Kathleen Quillian isast [@] leonardo [dot] info ______________________________________________________ BYTES ______________________________________________________ Director, Institute for e-Learning and Research in the Arts and Design College of Arts and Architecture The Pennsylvania State University The Director of the new e-Learning and Research in the Arts and Design initiative in the College of Arts and Architecture at Penn State shall be responsible for providing leadership and overall coordination of digital imaging and technology in the arts and design in three primary areas: 1) online instruction, 2) stimulating research in these allied fields, and 3) grant writing/fundraising to support the projects of the Institute. The mission of the Institute is to take a leading position in the future of the disciplines represented in the College of Arts and Architecture in the global advancement of digital practice, implementation, and research. The director shall supervise a staff comprised of two instructional designers, an online course manager, and clerical staff and shall be the primary point of contact for the Institute. The director shall serve as the liaison with the academic leaders of the College, shall establish and/or build upon internal and external collaborations in the areas of e-learning and research in digital technologies in the arts and design, and shall be responsible for proposal development for external support of research in these areas. For application details, a more complete position description, and information regarding this initiative, please see website: http://www.artsandarchitecture.psu.edu/e-learning/ Penn State is committed to affirmative action, equal opportunity and the diversity of its workforce. _____________________________ Director, School of Film and Digital Media, University of Central Florida The School of Film and Digital Media at the University of Central Florida seeks a visionary Director to lead its growth into an internationally prominent center for creative innovation and scholarship in film and digital media. The Ph.D., M.F.A. or comparable industry experience is required. The successful candidate must have a record of either scholarship or professional creative activity consistent with the standards for appointment as a tenured associate or full professor. A significant record of accomplishment as a successful cinema or media professional, or in a related academic field, is required. The Director must have demonstrated leadership skills, a clear understanding of the potential of the field and the ability to work effectively with the important constituencies of the university, community and industry. The School of Film and Digital media consists of the Film and Digital Media Divisions, the Center for Research and Education in the Arts, Technology and Entertainment (CREATE) and the Florida Interactive Entertainment Academy (FIEA). The Director would oversee all of these components. The University of Central Florida is a growing metropolitan research university in Orlando, enrolling nearly 45,000 students. The School of Film and Digital Media has more than 1,200 students and 37 faculty members with facilities on the main campus in East Orlando as well as a new graduate and professional center in downtown Orlando where CREATE and FIEA are housed. There is also a Downtown Media Arts Center. Bachelor of Arts degrees in Cinema Studies and World Cinema, a B.F.A. in Film (including Production and Screenwriting) and a B.A. and B.S. in digital Media (Visual Language, Internet and Interactive Systems) are offered. A graduate program beginning Fall 2005 offers the M.F.A. in Entrepreneurial Digital Cinema and the M.A. in Visual Language and Interactive Media. The M.S. in Interactive Entertainment will be offered pending approval. Applicants for the position should submit: 1) a letter of application, 2) a complete vita and 3) the names and contact information for three references. Applications should be sent to Dr. Mary Alice Shaver, Chair, Director Search, School of Film and Digital Media, P.O. Box 163120, University of Central Florida, Orlando, FL 32816-3120. Review of applications will begin on October 4 and continue until the position is filled. UCF is a culturally diverse university and an Affirmative Action/Equal Opportunity Employer. Search documents may be viewed by the public upon request, in accordance with Florida statute. _____________________________ INTERNATIONAL CONFERENCE New Constellations: Art, Science and Society 17-19 March 2006 Museum of Contemporary Art Circular Quay West, Sydney Australia New Constellations: Art, Science and Society - an international conference charting the ways in which art and science are gravitating towards one another within contemporary culture. The Conference will present the latest thinking about collaboration between artists and scientists and examine how the worldwide trend towards interdisciplinary engagement is changing the definitions, methodologies and practices they use and how they view the social implications of their work. Key Speakers: Ruzena Bajcsy, Immediate Past President, CITRIS (Centre for Information Technology Research in the Interest of Society), University of California, Berkeley, California; Elizabeth Grosz, Professor of Women's and Gender Studies, Rutgers, The State University of New Jersey, and Visiting Professor of Architecture, Princeton University, Princeton, New Jersey; Steve Kurtz, Founding Member, Critical Art Ensemble; Carnegie Mellon University, Pittsburgh, Pennsylvania; Roger Malina, Chairman, Board, Leonardo, the International Society for the Arts, Sciences and Technology; Co-Chair, International Advisory Board, Inter-Society for the Electronic Arts. The Conference has grown out of a collaboration between artist Mari Velonaki and The University of Sydney's Australian Centre for Field Robotics, an Australian Research Council - Australia Council for the Arts Linkage Project. The Conference is supported by Artspace, Australian Network for Art and Technology and Patrick Systems and Technologies and The University of Sydney. Conference enquiries: 61 2 9245 2484 or email [@] education [dot] mca [dot] com [dot] au For further program details and information go to This project has been assisted by the Australian Government through the Australia Council, its arts funding and advisory body. _____________________________ NEW MEDIA/INTERMEDIA Purdue University. Entry level Assistant Professor. Tenure track. Begins August 14, 2006. Applicant must be practicing New Media/Intermedia artist with strong theoretical basis. M.F.A. or equivalent professional experience required. Professional experience and university teaching preferred. Teach cross-disciplinary courses and develop curriculum in New Media/Inter-media across four divisions (Art & Design, Dance, Music, and Theatre) of Patti and Rusty Rueff Department of Visual and Performing Arts and Envision Center . Continue professional work in creative endeavors and/or research beyond Purdue University and participate in usual departmental activities. Send letter of introduction, resume of professional and academic experience, digital portfolio of representative work, copies of reviews of art works, statement of teaching interests and previous teaching experience, three names and contact information of current references, and SASE for the return of visual materials to Star Brown, Pao Hall, 552 West Wood Street, West Lafayette, IN 47907-2002. Purdue University is an Equal Opportunity/Equal Access/Affirmative Action employer. Women and minority candidates are especially encouraged to apply. (preferential): 1/31/06 or until filled. _____________________________ artBridge: a matrix of Inspiration artBridge is a matrix of information to bridge the science and techniques of Holography to the art and new media. artBridge has been initiated in 1989, by Dietmar …hlmann, artist and holograph with a group of artists in Liverpool. The goal was to communicate, produce and exhibits art using new technologies bridging all kind of parties. In 2000, the quarterly online magazine has been launched to exchange experiences about new media of light, optics and natural phenomena like holography and its impact in socio-cultural mutations. The concept of artBridge is based on the "Multidimensional Holographic Thinking" where everything is connected to each other in continuous changing situations, calling for major transformations in our structure of perception and acquisition. When we know that seventy percent of the information we store is transmitted by light stimuli, then holographic and light imaging have a big responsibility in new visualization. Contact: artBridge editor: Odile Meulien-Ohlmann, a sociologist, co-founder of the Art, Science and Technology Institute ASTI and former Curator of the Museum of Holography in Washington, DC, USA. Om [at] artbridge [dot] info artBridge creative technique manager: Dietmar Ohlmann, MA RCA, an artist and holograph. Do [at] artbridge [dot] info ________________________________________________________________ ___________________ | | | | | CREDITS | | | |___________________| Nisar Keshvani: LEA Editor-in-Chief Natra Haniff: LEA Editor Michael Punt: LR Editor-in-Chief Andre Ho: Web Concept and Design Consultant Roger Malina: Leonardo Executive Editor Stephen Wilson: Chair, Leonardo/ISAST Web Committee Craig Harris: Founding Editor Editorial Advisory Board: Irina Aristarkhova, Roy Ascott, Craig Harris, Fatima Lasay, Michael Naimark, Julianne Pierce Gallery Advisory Board: Mark Amerika, Paul Brown, Choy Kok Kee, Steve Dietz, Kim Machan fAf-LEA Corresponding Editors: Lee Weng Choy, Ricardo Dal Farra, Elga Ferreira, Young Hae- Chang, Fatima Lasay, Jose-Carlos Mariategui, Marcus Neustetter, Elaine Ng, Marc Voge ________________________________________________________________ _________________ | LEA | | PUBLISHING | | INFORMATION | |_________________| Editorial Address: Leonardo Electronic Almanac PO Box 850 Robinson Road Singapore 901650 lea [@] mitpress [dot] mit [dot] edu ________________________________________________________________ Copyright (2005), Leonardo, the International Society for the Arts, Sciences and Technology All Rights Reserved. Leonardo Electronic Almanac is published by: The MIT Press Journals, 238 Main Street, Suite 500, Cambridge Center, Cambridge, MA 02142 U.S.A. Re-posting of the content of this journal is prohibited without permission of Leonardo/ISAST, except for the posting of news and events listings that have been independently received. Leonardo/ISAST and the MIT Press give institutions permission to offer access to LEA within the organization through such resources as restricted local gopher and mosaic services. Open access to other individuals and organizations is not permitted. ________________________________________________________________ < Ordering Information > http://mitpress.mit.edu/catalog/item/default.asp?ttype=4&tid=27& mode=p Leonardo Electronic Almanac is free to Leonardo/ISAST members and to subscribers to the journal Leonardo. Note: Please use the addresses below to submit your order. Address all orders and inquiries to: Circulation Department MIT Press Journals 238 Main Street, Suite 500, Cambridge, MA 02142-1046 USA TEL: (617) 253-2889 (M-F, 9-5) FAX: (617) 577-1545 (24 hours) For queries contact: journals-orders [@] mit [dot] edu (subscriptions) ________________________________________________________________ ________________ | | | ADVERTISING | |________________| Leonardo Electronic Almanac is published monthly -- individuals and institutions interested in advertising in LEA, either in the distributed text version or on the World Wide Web site should contact: Leonardo Advertising Department 800 Chestnut Street San Francisco, CA 94133 phone: (415) 391-1110 fax: (415) 391-2385 E-mail: isast [@] leonardo [dot] info More Info: http://mitpress2.mit.edu/e- journals/Leonardo/isast/placeads.html#LEAads ________________________________________________________________ ____________________ | | | ACKNOWLEDGEMENTS | |____________________| LEA acknowledges with thanks the Rockefeller and Ford Foundations for their support to Leonardo/ISAST and its projects. ________________________________________________________________ < End of Leonardo Electronic Almanac 13 (11) > ________________________________________________________________