LEONARDO MONOGRAPH SERIES
Published by the International Society for the Arts, Sciences and Technology (ISAST)
425 Market Street, 2nd Floor, San Francisco, CA 94105, U.S.A.
1990 Version Copyright (c) 1990 ISAST. All rights reserved.
(ISBN 0-9625355-0-8)
1997 Version Copyright (c) 1997 David Rosenboom and ISAST.
(Original 1990, Revisions to Parts 5 and 6 and additional Appendixes, 1997)
Coordinating Editor: Pamela Grant-Ryan
Manuscript Preparation for 1997 Version: Karen Beardsley
David Rosenboom (b. 1947), composer, performer, conductor, interdisciplinary artist, author and educator, has explored ideas in his work about the spontaneous evolution of forms, languages for improvisation, new techniques and notation for ensembles, cross-cultural collaborations, performance art, computer music systems, interactive multi-media, compositional algorithms, and extended musical interface with the human nervous system since the 1960's. His work is widely distributed and presented and he is known as a pioneer in American experimental music. Rosenboom has been Dean of the School of Music, Co-Director of the Center for Experiments in Art, Information and Technology, and Conductor of the New Century Players at the California Institute of the Arts since 1990. He taught at Mills College from 1979 to 1990, was Professor of Music, Head of the Music Department, Director of the Center for Contemporary Music, and held the Darius Milhaud Chair from 1987 to 1990. He studied at the University of Illinois with Salvatore Martirano, Kenneth Gaburo, Lejaren Hiller, Soulima Stravinsky, Paul Roland, and Gordon Binkerd, among others, and has worked and taught in innovative institutions, such as the Center for Creative and Performing Arts at SUNY in Buffalo, New York's Electric Circus, York University in Toronto, where he was Professor of Music and Interdisciplinary Studies, the University of Illinois, where he was recently awarded the George A. Miller Professorship, New York University, the Banff Center for the Arts, Simon Fraser University, the Aesthetic Research Centre of Canada, the San Francisco Art Institute, and the California College of Arts and Crafts.
The original version of this monograph was written in 1989 and published in 1990. It's purpose was to document work that had taken place since the publication of my earlier book, Biofeedback and the Arts, Results of Early Experiments in the mid-1970's and the time of the original publication of this monograph. Subsequently, after all copies of the first edition had been distributed, the publication materials were lost in the tragic events of the Oakland fire. I have continued to receive many requests for copies of the monograph that I have not been able to fulfill. Now, with the advent of the World Wide Web and the assistance of ISAST, I am able to make a version of this document recovered from old computer files available again. Thanks are due to my assistant, Karen Beardsley, for conforming these old files to the published version as closely as possible. Hopefully, the ideas contained herein are sufficiently powerful to stimulate new ideas and inspirations.
PART 8: A NOTE ON MUSICAL HOLARCHIES
APPENDIX 1: STUDY FOR ON BEING INVISIBLE (1978)
APPENDIX 2: SOME NOTES AND UPDATES FOR THE 1997 RE-PUBLICATION
The purpose of this monograph is severalfold: (1) to give a detailed description of some work done in the mid- to late-1970s in which I was able to achieve the spontaneous generation of formal musical architectures with a computer music system by using a detailed analysis of evoked responses to features in those architectures recorded from a performer's brain; (2) to provide an overview of some historical events related to the development of artistic works that are in some way responsive to bioelectrically derived signals; (3) to describe briefly the emergence of the biofeedback paradigm and to discuss biofeedback modeling; (4) to survey accumulated knowledge regarding interpretation of electroencephalographic phenomena with particular emphasis on event-related potentials (ERPs) and their relation to aspects of selective attention and cognitive information processing; (5) to present a speculative model for the general interpretation of electroencephalographic waveforms; (6) to discuss some inferences and speculations relating these phenomena to musical experience; (7) to provide an assessment of some methods and techniques that have been applied to realizing works of art with these phenomena; (8) to describe some specific algorithms for generating self-organizing musical structures in a feedback system that relates a limited model of perception to the occurrence of event-related potentials in a performer's brain; and (9) to discuss the potential of new and emerging technologies and conceptual paradigms for the future evolution of this work. Finally, an actual score containing a conceptual scheme for a biofeedback work involving electroencephalographic phenomena and electronic orchestrations is provided in an appendix to stimulate further thinking and ideas for applications in the arts.
The writing is addressed to those with an interdisciplinary interest in the arts (particularly music) and the sciences (particularly those of the brain, psychology and perception, and the study of self-organizing systems). However, readers whose backgrounds are in the arts or sciences alone, or even other areas such as cognition, philosophy, computer science or musical instrument design, are encouraged to read on as well. Many references are provided with which the reader may enhance her or his knowledge in a particular sub-discipline. Those who may find some of the technical descriptions difficult should first skim through the entire document and then return to individual sections for further study.
It is hoped that the ideas presented herein may contribute in some way toward increasing our breadth of understanding concerning dynamic processes in the arts and sciences.
In the history of this youthful world, the best product that human beings can boast of is probably Beethoven; but, maybe, even his art is as nothing in comparison with the future product of some coal-miner's soul in the forty-first century.
---Charles Ives, 1920 [1]
Since the discovery of electrical pulsations arising from within the human brain, imaginative souls have speculated that through a direct connection of the brain to devices for sound production and visual display, internal realities would eventually be made externally, materially manifest. In turn, these would become enfolded through the senses into an evolving interplay among the fabricated models of cognition, the passages of consciousness, and the energetic, though capricious, environment. A global music, reflecting the morphodynamic holarchies of existence, might come into being.
My God! What has sound got to do with music!
---Charles Ives, 1920 [2]
EARLY SUGGESTIONS AND EXPERIMENTS
In a now-famous 1934 paper, the pioneering physiologists E. D. Adrian and B. H. C. Mathews reported on experiencing a translation of the human electroencephalogram (EEG) into audio signals. While listening to his own 'alpha rhythm' (large-amplitude smooth waves of 8-13 Hz) presented through a loud speaker, Adrian tried to correlate his subjective impression of hearing the alpha come and go with the activity of looking or not looking with his eyes [3]. Interestingly, this alpha rhythm, as it came to be called, was originally known as the 'Berger rhythm'. An inherently rhythmic quality had been observed in waves of electromotive force first detected on the scalp in the 1870s. However, it was Hans Berger who in 1929 provided the first comprehensive description of the human electroencephalogram [4]. Using crude, early instrumentation, Berger recognized spontaneous oscillations in the EEG, particularly when detected over the occipital area of the cortex (back of the head), where they are manifest with relatively large amplitudes. Since the occipital area is most heavily involved in the processing of visual information from the eyes, this Berger rhythm was subject to variation associated simply with opening and closing one's eyes. Most likely, the significance of the term 'alpha' lies in nothing other than that this 'rhythm', a relatively repetitive and coherent waveform, was just the first to be detected with early instrumentation. These instruments were relatively insensitive to other EEG components of much lower amplitude. During succeeding decades, numerous other scientists reported various methods of generating, in the auditory domain, stimulus frequencies that in some way followed brain rhythm frequencies. Usually this involved some aspect of alpha rhythm [5-9]. These methods were considered an aid in analyzing what was obviously a complex electronic manifestation of brain activity.
This use of auditory translations of EEG patterns allowed observers and investigators to employ the considerable integrative powers of auditory perception and feature extraction to guide them toward some insight into the form of these signals. Today we might listen to the sonic translations of complex number sequences to divine mathematical patterns that might otherwise go undetected. I have personally experienced the phenomenon of discovering and squashing elusive and subtle 'bugs' in large computer programs simply by listening to an audio translation of the raw machine code of these programs [10]. Stories are sometimes told of similar events achieved by programmers in the early days of computing, when we used to sit around and watch running program code displayed on old-fashioned panel lights. Other programmers have also been observed indulging in this kind of sensory analysis when all other systematic methods of axiomatic computer science have failed. The powers of the sensory-feature extracting mechanisms and the integrative powers of high-level image synthesizers in the brain are awesome. They simply await our conscious probing and comprehension through open focus, discipline and practice. We also live in a fantastically rich contemporary music milieu in which, as musicians, our ears are evolving even greater powers to help us manage sometimes immense and deep formal architectures. What we may yet discover by listening to our own biological computer, if I may invoke such a simple analogy, is practically unfathomable.
The observation can be made that, throughout the history of advances in science and technology, artists have always been ready to experiment with applications of each new breakthrough or development, almost as soon as it is conceived or realized. Brain science proves no exception. In the past 25 years, composers like Alvin Lucier, Richard Teitelbaum, myself and others have produced major works of music with EEGs and other bioelectronic signals. Lucier's 1965 work Music for Solo Performer achieved a direct mapping of a soloist's alpha rhythms onto the orchestrational palette of a percussion ensemble [11-13]. Greatly amplified alpha signals were used to activate, either acoustically or mechanically, an array of otherwise performerless percussion instruments. This produced the startling effect of a percussion ensemble seeming to activate itself, almost invisibly, but somehow following activities inside the solo performer's mind. Teitelbaum's Organ Music and In Tune, both realized in 1968, added heart beat and breath sounds, sensed with contact microphones, to EEG signals in the creation of an electronic music texture [14]. My own work with brainwaves began with experiments in musical production using alpha rhythms and explorations of the relation of alpha wave production to music perception and the various states of awareness and consciousness associated with music performance. Initially, this took place in 1968-1969 in the laboratory of Les Fehmi, an early biofeedback researcher at the State University of New York at Stony Brook, after a suggestion by E. E. 'Ted' Coons of New York University. I developed an environmental demonstration-participation-performance event entitled Ecology of the Skin in 1970-1971. It involved biofeedback monitoring of brainwaves and heart signals from performers and audience members and their translation into a musical texture, along with synchronous electronic stimulation of visual phosphenes (colored patterns often seen with eyes closed) at cerebral light-show viewing stations for the audience. The electronic setup for this work included the capability of adjusting the degree of brainwave control over sound for each of 10 participants according to a simple statistical measure, the amount of time spent per minute producing alpha waves (see Fig. 1).
Fig. 1. Ecology of the Skin (1970). Setup diagram for 1970 biofeedback installation/performance/participation work, Ecology of the Skin: DIF AMP = differential brainwave amplifier; LPF = low pass filter; BPF = band pass filter; ENV FOL = envelope follower; SHMT TRIG = Schmit trigger circuit; E/UT = events per unit time; DEC LOGIC = modular logic circuit system; D/A = digital-to-analog converters; CUR LIM = current limiting circuit; CMRA = video camera.
Eventually, my work led to the creation of a laboratory-the Laboratory of Experimental Aesthetics at York University in Toronto with the intent to study information-processing modalities of the nervous system in relation to aesthetic experience and states of consciousness while surrounded by an environment of artistic production. Under the sponsorship of York University, the Canada Council Explorations Programme and the Aesthetic Research Centre of Canada, many individuals carried out experiments and produced artworks there over a 7-year period. These works-representing such art forms as music, visual art, kinetic art and dance-are documented in my book, Biofeedback and the Arts, Results of Early Experiments [15].
Another early experimenter was Manfred Eaton, who carried out experiments in music and bioelectric phenomena at the ORCUS Research Center in Kansas City during the 1960s and early 1970s. Eaton described extensive explorations in applying various bioelectrically derived signals to artistic projects and the study of aesthetic responses to stimuli [16-18]. These signals resulted from measuring the EEG, pulse rate, respiration, galvanic skin response (GSR), blood flow volume, and the electrocardiogram (EKG). A variety of multi-sensory display systems were devised to follow changes in these measurements. Eaton also speculated on the possibility of employing sensory-evoked responses, requiring more sophisticated analysis capability than what was readily available at that time, in order to generate complex patterns for music, kinetic arts and television.
The biofeedback paradigm began to be clearly articulated when the work of Neal Miller, of Rockefeller University in New York, became widely known in the 1960s. Miller had devoted over 30 years to the study of animal and human learning and had gained the respect of the majority of investigators in his field. His work at this time undermined the long-held image of the nervous system as being divided into two inflexibly separated parts-the voluntary and involuntary-one subject to learning and conscious control, the other capable of executing only automatic, built-in programs. Miller, along with his associate, Leo DiCara, demonstrated that, indeed, animals and human beings could learn voluntarily to influence the behavior of bodily functions such as heart rate, blood pressure, vasomotor responses, electrodermal activity, salivation, urine formation, gastric motility and metabolic processes previously thought immune from conscious influence. This was applied to a host of visceral phenomena and was dubbed, 'visceral learning' [19].
In animals, these learning phenomena were often achieved by means of providing some form of reward for producing the desired response. In humans, the learning paradigm was dependent on presenting information to the subject about the states and trends of change occurring in these visceral phenomena, through some form of sensory feedback, while relying on the subject's own internal motivation to effect the change. Thus, the term 'biofeedback' was coined.
Biofeedback: Definition and Modeling The term 'biofeedback' will be used herein to refer to the presentation to an organism, through sensory input channels, of information about the state and/or course of change of a biological process in that organism, for the purpose of achieving some measure of regulation or performance control over that process, or simply for the purpose of internal exploration and enhanced self-awareness. Normally, this information will be of a type not otherwise available to that organism. It does not presuppose, however, that such an external indicator could not, through disciplined practice, be replaced by an internal mechanism of which the subject can achieve awareness without the aid of an artificial monitoring system.
Over a decade ago, the feedback paradigm, as understood from cybernetics, was being considered to offer alternatives to the previously dominant behaviorist school of psychology [20]. Behaviorism could be described as an open-loop paradigm relating behaviors (effects) to causes (stimuli). The feedback paradigm is a closed-loop one in which behaviors (effects) are also treated as one of several possible classes of causes (stimuli) of the same behaviors. Cause and effect can be traced all around the closed, feedback loop, creating a 'chicken-and-egg' problem as to which came first. This problem is not always resolvable, or even a meaningful one to pose. A process -oriented approach to modeling the self-organizing dynamics of a biofeedback system may prove more fecund.
It may be useful to recall the basic feedback paradigm as we have inherited it from cybernetics (see Fig. 2). In the simplest case, a defined goal state is assumed for a process, the output of which is measured and compared against the desired end (Fig. 2a). An error correction signal is then generated and combined with the input in such a way as to direct or tune the process so that it eventually arrives at the goal state. The cases of both negative feedback, which can lead to stabilization of particular behavioral states, and positive feedback, which normally leads to instability and/or system energy saturation, are shown (Fig. 2b and 2c). A partial view of learning may be described as the tuning of [K] such that the desired output is maintained with minimum expenditure of energy, while making [e], the error signal, tend toward 0. This is homeostatic behavior. In a biofeedback system we may conceive of a statistical distribution of behaviors along some relevant parametric axis, as shown in Fig. 3a, in which the desired goal is represented by the mean parametric value and the achievement of the goal by the variance around the mean. If in such a closed-loop system the probability of occurrence of a particular behavior is also increased as a result of the execution of that behavior, then we have a resonant system. In such a system we have negative feedback, which attenuates behaviors away from the mean goal state by means of an error signal, but we also have amplification of the behavior associated with the goal state. A system like this will be driven into saturation and possible instability unless the amplification of the goal state behavior in the feedback loop is mediated by other concomitant goal states or input from the environment outside the system. Lack of such mediation of positive feedback amplification may underlie such phenomena as addictive or obsessive behavior, the tendency of societies to self-destruct through degradation of their support environment or through unrestrained conflicts associated with out-of-control resonances in their political and economic systems.
Fig. 2. Basic Feedback Paradigms. (a) Feedback control mechanism in which an error signal is generated and added to the input in order to direct the system to a desired goal state. (b) Effect of negative feedback on system output. (c) Effect of unmediated positive feedback.
Today we have the language of dynamical systems theory to add to our lexicon of paradigms with which to probe behavioral patterns [21]. In a dynamics characterization of biofeedback, we would describe particular goal states as attractors in a behavioral phase diagram [22]. Figure 3b shows a modified statistical distribution in which the mean goal state is represented as an attractor bounded on either side by two repellers characterizing the state boundaries. This kind of lateral inhibition, which sharpens the definition of the desirable state, is seen in Fig. 3a to be the result of summing two Gaussian functions-the original, facilitative, behavioral distribution one, and a broad, negative, inhibitory one [23]. Behavioral patterns would be characterized by collective variables known as order parameters. The nature of the order parameters would be specific to the types of functions and tasks involved. The success with which this method provides a useful description of the system under scrutiny will depend, in part, on determination of the right order parameters to use. This is an area where creative solutions must often be found. Well-defined behavior patterns are attractors in the phase diagram, corresponding to stable collective states of the order parameter dynamics. Figure 4 shows a hypothetical behavioral phase space with three attractors. Behaviors occupying neighboring points in the phase space will eventually converge to the attractor region (basin of attraction). All types of attractors, static (fixed point, a set of dimension zero), periodic (limit-cycle, closed curve, a set of dimension one) and chaotic (where small uncertainties in initial conditions lead to large uncertainties in future behavior), as well as multiple attractors (multi-stable states), may reside in the phase diagram. Chaotic attractors may also have non-integer, fractal, dimensions and may be referred to as strange attractors. The system may be perturbed by environmental input or noise and shifted away from an attractor. Its characterization will then include various time constants, such as local or global relaxation time, the time required to settle toward an attractor state again. Probability distributions may be used to describe fluctuations around an attractor state (goal or mean), wherein variance corresponds to degree of fluctuation. The more stable an attractor is, the smaller the average deviation from the attractor state will be when a perturbing force of a given strength impinges upon it. It is outside of the scope of this paper to outline the analysis techniques of dynamical systems theory. However, its methods and language may lead us to a deeper understanding of the spontaneous generation of patterns, changes of state, and the self-organizing nature of human behavior, cognition and consciousness. I will return to the use of this language numerous times.
Fig. 3. Statistical Distribution of Behaviors. (a) Description for the behavior of a feedback control system using the language of statistics. The desired goal state corresponds to the mean of a Gaussian distribution of behaviors and achievement of the goal to the variance. A negative inhibitory function is also shown. (b) When the two Gaussians of (a) are summed, a function results with a positive peak at the mean or goal state surrounded by negative, inhibitory boundaries.
A biofeedback system may be mistakenly viewed as simply a method for stabilizing particular behaviors and, thus, as a static equilibrium system. Indeed, a surface understanding of the therapeutic application of biofeedback techniques as mediators of runaway processes deleterious to an organism, such as epileptiform brainwave patterns, heart arrhythmias, high blood pressure or uneven vasoconstriction associated with migraine headaches, may reinforce this wrong view. In fact, these processes all involve the self-organization of dynamical regimes within the organism, aided by the additional information feedback loops of the biofeedback mechanism, in such a way that the evolution of these regimes will tend toward a dynamic that promotes the self-renewal of the organism. The biofeedback system may qualify for description as an autopoietic system, capable of self-renewal merely by employing a process of self-reference. A biofeedback mechanism participates in the interaction dynamics of an autopoietic organism with its environment. Since this organism is in itself a complex, dynamical system capable of a certain degree of self-determination regarding how the information circulating in a biofeedback loop is used, a simple cybernetic view of biofeedback as a positive feedback loop is insufficient. Both positive and negative feedback functions may occur in the structure of the system's interaction matrix. Consequently, biofeedback, in its contemporary, sociocultural context, must be viewed as our participation in the 'process of self-reference' of dissipative, autopoietic organisms in interaction with an environment from which they import energy and to which they export entropy. In this light, the effect of positive feedback can be seen to be canalized (directed along chreods, lines that guide an organism's ontological development by the functions of the healthy organism in such a way as to lead to bifurcations in the evolution of the structure of behavior. The information circulating in a biofeedback process loop, therefore, can never be viewed as a static entity. The information, too, is subject to its own self-organization. Consequently, I will always refer to 'formation' as something that is 'in formation', that is, 'in the process of becoming formed'.
Fig. 4. Hypothetical Behavioral Phase Space. Depiction of a multi-dimensional, behavioral phase space in which the order parameters correspond to measurement scales needed to characterize the system behaviors. Three attractors are shown separated by boundaries, implying that system behaviors will tend to fall into one of the three behavioral regions. Behavior trajectories-describing movement among attractor regions-may be complex and subject to description in the language of dynamical systems theory.
Eric Jantsch has provided one of the most eloquent characterizations available regarding the nature of self-organizing systems [24]. However, even such a brilliant author as he succumbs to a major blunder by falling into a common trap of popular misunderstanding (which many scientists have) about the biofeedback paradigm. This is quite understandable, since biofeedback was interpreted in an almost science fiction-like manner during its faddish period of the early 1970s. A misrepresentation developed on which numerous short-lived careers of academic and commercial charlatans were based. Biofeedback was commonly seen as the extension of a conscious control hierarchy over processes of the body, achieving "Total control of rational thinking over the body!" [25]. Jantsch was horrified by such a prospect. This fearful misrepresentation must be debunked. In fact, it has been amply demonstrated that in many examples of biofeedback processes, such as with alpha wave production in the brain, control cannot be achieved by means of rational processes. Rather, the subject must find a way to allow alpha wave production to evolve rather than make it appear. I suggest that the Zen-like state associated with achievement of what we may wrongly associate with the word 'control', is, in fact, a striking example of the quality of subjective experience associated with true conscious participation in autopoietic self-organization, including feedback with the environment. Jantsch himself provides an appropriate characterization of this experiential quality. "To live in an evolutionary spirit means to engage with full ambition and without any reserve in the structure of the present, and yet to let go and flow into a new structure when the right time has come" [26]. This strikes at the essence of experience associated with my ongoing musical work, On Being Invisible, described at length later in this paper. We must therefore modify and evolve our characterization of biofeedback. It should be seen as the circulation of information about functions within an organism in ever widening feedback loops involving the consciousness of that organism, to serve its creative extension, beyond the structure of its own prior self-definition, in the natural meta-evolution of its self-organization dynamics-and not merely as the extension of a conscious control hierarchy. It is, rather, the holarchic extension of dynamical processes of the self in which the conscious mentality of the individual may responsibly choose to participate as a manager or catalyst. Thus, conscious human self-management is seen as an autocatalytic agent in its own evolution. (See Part 8, A Note on Musical Holarchies, for a brief discussion of the meaning of holarchy.)
Fig. 5. Self-organizing Biofeedback Mechanism. Schema for the flow of information among components comprising a self-organizing biofeedback mechanism, including those internal to the organism and those typically present in an experimental environment. Many setups described in the text for artistic production or research include some or all of these components.
A more complex and detailed view of a biofeedback mechanism capable of self-organization is shown in Fig. 5. A large, global feedback loop, in which information circulates across the boundary separating the internal functions of the organism from the external environment, is shown. This boundary is not conceived as one generating any significant degree of autonomy of the organism from its environment. Rather, it is merely an artificial construct made relevant by the need to manifest relationships between that which is sensed as information impinging upon the organism and actions synthesized by the organism. In reality, however, the organism and its environment are better conceived as a global system of mutually coupled transformation processes, inside each of which further feedback loops are active. The organism's sensor function transforms energy/information patterns from the environment. The results are combined with an endogenous input quantity that, in turn, results from the sum of information circulating in internal feedback paths plus disturbances arising from within the organism. This data is subject to measurement and comparison functions involving memory and cognition. The results direct and tune the organism's effector function and, possibly, update memory. The environment in Fig. 4 is assumed to contain an intelligent process, possibly an experimenter with programmable apparatus. A detector function transforms information resulting from the organism's behavior. This is subject to further measurement and analysis. The result is compared with directives arising from programs or goals within the intelligent system. As a result, a signal intended for the organism is synthesized and, possibly, the program updated. The signal, summed with environmental disturbances, forms the exogenous quantity input to the organism.
Subsequent Developments Outside the Arts Subsequent to Miller's work on visceral learning, the biofeedback paradigm was applied to functions of the organism that manifest observable effects in the electro-chemical operation of the human brain. The most obvious first candidate for exploration was the old Berger rhythm, now known as the alpha wave. Joseph Kamiya, Les Fehmi and Thomas Mulholland were but three prominent names among the scores of early pioneers. They were followed by many more. The literature expanded at a phenomenal rate throughout the 1970s. Books and articles flooded publishers' desks, the international press and media took a keen interest, careers were made and broken, and biofeedback became a fad. Underneath all this flurry, however, a steady stream of solid research and thinking flowed, though sometimes masked by opportunistic hyperbole. Some important repositories of documentation of this research include the Aldine Atherton annuals as well as Biofeedback and Self Control (begun in 1970 by Aldine Atherton, Chicago), the journal Biofeedback and Self-Regulation (begun in 1976 by Plenum Press, New York), and the excellent series Consciousness and Self-Regulation (begun in 1976 by Plenum Press, New York).
During the later part of the 1970s, things settled down somewhat. The chaff fell away, leaving the more solid and resilient research programs to produce of reliable results and a methodology.
In this paper I will not discuss the field of visceral learning, nor will I cover the medical or therapeutic aspects of biofeedback. I will concentrate primarily on the use of information that can be extracted from the brainwave EEG for artistic or musical purposes.
Some Applications and Cultural Implications To date, considerable success has been achieved in medical, therapeutic and self-improvement applications. Biofeedback has been used as an aid in controlling blood pressure abnormalities and heart arrhythmias, as a treatment for migraine headaches, and for suppression of epileptiform patterns in brainwaves. It is used extensively as a treatment for hypertension and as an aid in reducing stress. It has become a basis for certain kinds of self-improvement training programs, such as those for achieving control of mental states, focus, attention, relaxation and self-integration. Complementary exercises in tension release, visualization, meditation and open-focus training also are often employed [27]. Biofeedback with EEG parameters has added to our knowledge about consciousness, states of awareness and cognitive processes.
Originally touted by the press as a panacea for all that ails and the key to self-transformation, biofeedback is now perceived in a more sober light. However, biofeedback raises issues of self-consciousness that do not fit neatly into Western culture. The achievement of success with biofeedback requires discipline, intense and regular practice, and often meditative skills. These were consistent with views held in the 1960s of transcendence and the idealism of cultural transformation. These ideals faded with the rise of 'yuppi-dom' in the 1970s, as disillusionment grew when earlier hopes for change were seen to fail or to be forgotten and, in the 1980s, as self-realization was replaced by the necessity of socio-economic self-validation. In such a climate, lack of further substantive progress in applications can only be blamed on an unwillingness to pay the price of personal hard work to achieve transformation.
In the arts, it is not difficult to find individuals willing to apply themselves to the serious exploration of such phenomena. Transformation and personal explorations are a mainstay for experimentalists in the arts, the food of progress.
At first, the greatest expansion of artistic activities involving biofeedback occurred in the field of music. Teitelbaum's T'ai Chi Alpha Tala (1974), developed in our Laboratory for Experimental Aesthetics, involved transmission of alpha signals from an artist, Barbara Mayfield, engaged in the practice of T'ai chi Chu'an, by means of a tiny, encapsulated brainwave amplifier and FM transmitter attached to the artist's head. During the performance, alpha signals were extracted and used to trigger electronically synthesized melodies tuned to an Indian mode, while South Indian mrdangam master Trichy Sankaran instantly analyzed and embellished the resultant rhythmic patterns. In a later version, synchronized video processing of the performers' images was added in collaboration with video artists Dan Sandin and Jim Wiseman. Since that time, musical experiments and performances have been generated by many others in virtually all stylistic arenas, from contemporary popular music to the avant garde. Most involved the relatively direct following of some EEG phenomenon, usually amplitude of alpha waves, by a musical parameter, such as melodic shape or tone color. These processes were imbedded in a great variety of cleverly conceived, creative performance styles. Lucier's Clocker (1978) presents audiences with an intriguing image in which the progress of clock time seems to be warped by a performer whose GSR is used to change the settings on a time-delay device being fed the sounds of clock ticks [28]. A connection is established between changes in emotional states reflected by the GSR and the passage of subjective time. My own work in this area evolved throughout the 1970s and is documented [29-40] in several books, recordings, articles, television and video productions and films, as well as later in this document.
Applications in kinetic arts often involve installations or applications in performance. Pieces involving brainwave manipulated visual displays and interactive environments have been created by artists Jacqueline Humbert and C. Mark Nunn as well as myself [41]. Biotelemetry has been used in theater environments. One notable experiment, organized by Richard Lowenberg and associates in California, involved brainwave, muscle and accelerometer signal measurements from two groups of dancers, one located on the West Coast and one on the East Coast of the U.S.A. The signals were translated into sound and video displays and transmitted between the two groups by means of satellite communications provided courtesy of NASA. At the 1986 New Music America Festival in Houston, a dramatic show piece by artist Stelarc was created involving a large array of bio-sensors, a robot-like prosthetic arm, interaction with flashing lights and laser beams, and massively amplified sound [42].
Biotelemetry techniques for monitoring and transmitting muscle signals (EMG) and brainwaves (EEG) from dancers have been developed and applied in performance. Some of these have also been applied in the kinetic arts. One recent example is a system called MADDM (myoelectronically activated, dance-directed music system), in which techniques have been developed for detecting and processing signals from several muscle groups and transmitting them to a computer music system. In this way, dance movements are given control over the playing of a musical score [43].
My own work in biofeedback and the arts, begun 20 years ago, is experiencing a revival due to the fact that advances in technology now permit realization of musical concepts in performance that depend on complex real-time analysis of EEG signals. These were previously achievable only with cumbersome, non-real-time, laboratory-bound methods. Consequently, ideas which were impractical when they were proposed many years ago are now practical.
My earliest experiments involved the kind of simple parametric following referred to earlier. Soon, however, it became apparent that data from deeper statistical analyses of EEG trends would provide more meaningful signals with which to control musical forms. In Portable Gold and Philosophers' Stones (1972) [44, 45] a battery of such techniques was employed by a technician, who 'performed' with the analysis equipment, along with an ensemble of four biofeedback musicians. Most significant among these was a measure of the coherence time of EEG waveforms in various spectral bands, extracted by means of the autocorrelation function. This determined the range of direct control over elements of the sound texture given to each performer. As coherence times for a particular performer increased, the degree of influence over the sound texture allotted to that performer was also increased. Cross-correlations on signals from pairs of performers were sometimes used as well. Fourier analysis was used to extract EEG power spectra. These were mapped onto a series of weights applied to a group of resonant band-pass filters, in a device known as a 'Holophone'. This helped determine the spectral composition of the music (see Fig. 6). Measures of body temperature and GSR were also used to direct the tonality of the musical texture.
Fig. 6. System diagram for Portable Gold and Philosophers' Stones (Music from Brains in Fours) (1972). A diagram from the score of a musical composition that includes measurement and analysis of EEG signals, GSR and body temperature changes from a quartet of performers. Equipment for signal analysis is operated by a fifth performer, who also applies the results to a sound synthesis system. The frequency dividers and holophone produce a harmonically ordered musical texture, which is made to evolve according to changes in the bioelectrical information. See the text and references [11] and [31] for detailed explanations.
Vancouver Piece (1973) involved an exploration of subtle, visual thresholds in a biofeedback installation work [46, 47]. Pairs of participants facing a two-way mirror system observed the images of their own faces becoming superimposed on each other's bodies, producing a kind of identity exchange, when they were able to produce EEG waves, such as alpha, that were in phase with each other. Further coherent wave production caused faint wisps of light to trace out horizon lines at intensity levels near the threshold of perception, around an otherwise darkened, light- and sound-isolated room. Musical textures were also produced in response to in-phase alpha wave production from the two participants.
Many other art works and research programs were carried out during the 1970s. (See references listed in the section In Music in Part 1 for more examples.) The culmination of these musical applications was the production of On Being Invisible (1976-1977). In this work, detailed at length below, complete musical forms are constructed as a result of the self-organizing dynamics of a system in which both ongoing EEG parameters and event related potentials (ERPs), indicative of shifts in selective attention on the part of a solo performer, are analyzed by computer and used to direct the stochastic evolution of an adaptive, interactive electronic music system.
It is possible now to imagine large-scale, musical theater or operatic works involving biotelemetric presentation by human and even non-human performers interacting with audiences, other performers and environments. This could create a synergistic theater, linking participants in a large-scale organism, the ontology of which could provide a script of mythical proportions. The eternal quest to understand the role of human consciousness in determining when and how to initiate action provides the essential dramatic tension. This is the grand intent of my ongoing project, currently titled On Being Invisible II, which awaits adequate time and support for its full realization.
Here is a partial list of some interesting phenomena to explore with feedback, which can be detected relatively easily and for which the measurement techniques are practical: EEG (electroencephalogram, brainwaves), EMG (electromyogram, muscle signals), EKG (electrocardiogram, heart muscle signals), EOG (electro-oculogram, eye muscle movements), GSR (galvanic skin response, electrodermal, electrical skin resistance), ERG (electroretinogram), respiratory rate, body heat sensing at particular locations with thermistors, infra-red mapping (body heat profiles), and body movement tracking with video image analysis. All of these offer significant potential for probing aesthetic processes or for artistic production. In the remainder of this paper, however, I will concentrate primarily on the EEG, since it is packed with information about functions in the nervous system.
Research has shown that attention to an external stimulus is reflected by specific profiles in heart rate change. Generally, such attention allocation is associated with a mean drop in heart rate. However, two differentiated conditions for such attention, one involving recognition of a 'clue' in an ongoing stimulus stream and one involving later recall of the 'clue' showed clear signatures in heart rate change. Both involved a brief acceleration upon detection of the clue, followed by a significant deceleration and an acceleration rebound. However, the profiles were distinctly different for the two conditions [48].
The human brain produces a complex, multi-dimensional, pulsating, electromagnetic field, resulting from the electro-chemical behavior of masses of neurons acting in small to very large groups. Electrical currents are set up by means of the transport of ions across the cell membranes of these neurons. Changes are produced in the electrical potential across the membranes, measured inside the cell with respect to the outside. Current inside the cell is referred to as 'source current', current outside the cell as 'volume current'. Ion transport across a cell membrane usually begins at the dendrite end and proceeds throughout the cell body, possibly producing an action potential, which travels down the axon. Among a group of cells, this creates a population of current dipoles.
Outside the cranium, voltage differentials can be measured between any two points on the surface of the scalp or between a set of scalp points and some common, neutral reference on the body relatively distant from the scalp (such as the ear lobe). The voltage gradients that can be recorded from one of these points provide a rather one-dimensional look at the complex topology of surface pulsations that reach the outside of the skull. The transmission of internally generated currents to the extra-cranial surface is confounded by a number of factors. For example, the interior environment consists of a region of relatively high conductivity, the cerebral-spino fluid, which is surrounded by a container of low conductivity, the skull, which in turn is surrounded by a skin of relatively high conductivity, the scalp. The effect of this is to impose a low-pass limitation on the EEG bandwidth and to topographically smear out the waveforms observed, such that localization of the current source-sinks that originate the pulsations becomes extremely difficult. We could call this slow-pass filtration and spatial diffusion. Nevertheless, considerable information can be extracted about ongoing internal activities, just as the tracking of slow seismic waves can reveal much about the makeup of the earth's interior.
The EEG waveforms can be decomposed into a set of useful categories. I proposed a general schemata for this categorization in 1976 [49] and will attempt to give a more technical explanation here. In this schemata, the brain is viewed as a generator of locally singular events (see Fig. 7). It is considered a manifestation of process rather than an object. It functions as an open system, a dissipative structure, within a non-localized environment, containing universally distributed processes and a background, whether random-seeming or partially ordered. The categorization represents that which can be observed from what is termed point consciousness. I use this term to refer to a state of consciousness activated to produce cognizance of clear, localized, spatio-temporal constructs acting as a frame of reference in which to locate precisely the objects of experience as focal points of attention and awareness. It is assumed there are other states of consciousness, less involved with such spatio-temporal localization and the identification of singularities of experience. The brain's own ontology and all of its animate processes unfold within a continuum that extends from its environment inside to outside. The brain, then, is considered a point concentration of a relatively universally distributed process within its environment. This, in more concrete terms, also applies directly to the electrical manifestation of brain processes. These are electromagnetic field phenomena occurring within and interacting with other fields of varying extent and central strength or density. Indeed, we may define a universe, or some subsection of it we wish to address, as a single field, albeit one full of holes, and our objects of attention as point concentrations within that field. The components of the EEG we choose for study can be considered kinds of resonances with varying amplitudes, band widths, coherence and persistence. Their relation to mental events or other aspects of experience is the object of our exploration.
Fig. 7. The Brain as Point Concentration of Universally Distributed Process. The brain viewed as both an originator of singular events or experiences and as a particular, local concentration of processes requiring a larger environmental context in which to be expressed. A seemingly ubiquitous background noise-similar to that thought to arise from residual activity left over from the origin of the universe-is included.
After all conceivable methods of EEG waveform decomposition are exhausted, there remains a background of random-seeming phenomena, about which little is understood. An analogy can be drawn with the background noise permeating the universe. It may be that the mass action of neural circuits exhibits behavior like that of certain intrinsically random, dynamical systems [50]. From this may come irreversibility and instability, both necessary for ontological evolution to occur.
The next category is designated long-term coherent waves. It includes the traditional brain rhythms-alpha, beta, delta and theta. Emphasis is placed on the statistical property of coherence, in order to distinguish this idea from the mere presence of wave energy in a particular frequency band. Coherent waves are relatively stable, exhibiting high autocorrelation. There is a potentially serious pitfall associated with the use of simple band-pass filters to detect traditional brain rhythms. One may detect the presence of signal power in a particular filter pass-band, or even by means of Fourier analysis, in relatively incoherent, noisy signals with low autocorrelation. Since these coherent waves may result from widespread cortical synchrony among populations of neural circuits, it is critical to be able to distinguish this from pass-band energy resulting from broad-band noise. When EEG biofeedback information is derived from use of band-pass filters alone, experimental results can be confounded. This is particularly true for feedback of beta wave information, as discussed below.
Coherence is measured by the standard autocorrelation technique, in which a time-slice of the ongoing signal is compared with a series of successive time-delayed versions of itself, (see Fig. 8a). A curve is obtained, plotted as a function of time delay. Stable, repetitive patterns in the ongoing signal will periodically reinforce themselves during the processes of delay and compare. These will be revealed in the autocorrelation function, even if the original patterns are of very low amplitude and are buried in noise. Unstable, quasi-repetitive patterns will show up but will decay in amplitude over the delay axis, (see Fig. 8b). The rate of decay gives a measure of coherence time. Random signals will produce very rapidly decaying functions.
It is relatively easy to convert the autocorrelation function of a wave into a power spectrum, showing amplitudes and phases over the delay period for the major frequency components comprising the overall waveform. One way is to multiply the correlation function by sine and cosine waves being swept across the frequency range of interest. The resultant amplitudes are plotted as a function of frequency, giving the power spectrum. One can test for particular frequencies by using sine waves at or near those frequencies. The individual results for sine and cosine waves can combined in a way that gives a phase plot for various frequency components. The combination of these measures gives a much better picture of those long-term, coherent components that may actually be present in the ongoing EEG. Better data on which to base feedback indications for a subject can be provided than that obtained by simple band-pass techniques, which may also introduce their own resonances into the measurements. When autocorrelation and power spectrum measures are not feasible, it is still useful to employ band-pass filters. However, the experimenter must be aware of the nature of her or his instrumentation and proceed with caution. An unfiltered, raw tracing of the EEG on an oscilloscope screen or oscillograph should always be available for visual inspection and comparison with the behavior of the filters used. 'Long-term' in this categorization refers to coherence times of 1 or 2 seconds or more.
Fig. 8. Autocorrelation. (a) Diagram showing how the autocorrelation function of a signal may be calculated by means of sampling, successive time delays, multiplication and weighted time averaging (integration). (b) A typical autocorrelation function for a relatively coherent wave with a degree of instability. This instability is shown by how the function decays over the delay axis.
Coherent waves have been classified traditionally in four categories. The definition of these categories is, in my opinion, rather arbitrary and based primarily on a sequence of historical events. For example, alpha waves are called 'alpha' simply because they were the first to receive serious experimental attention and the easiest to detect with early, primitive instrumentation. The others followed in due course.
Alpha waves are defined as coherent waves in an 8- to 12-Hz band. They are relatively large in amplitude over certain areas, most notably over the occipital cortex. When reinforced with biofeedback, they tend to be associated with a Zen-like state of high attention without a locally specific focus, an object of attention, or the subject being engaged in making differentiations or abstractions. They are associated with open focus, clarity and alertness. One cannot learn to produce alpha voluntarily by effortful trying. Such conscious effort merely interrupts alpha. One must learn to allow alpha to occur rather than make it occur. Such a state has numerous parallels in the practice of the arts. One cannot force the process of creativity. It must be allowed to evolve within its own subtle conditions for occurrence.
Beta waves are poorly defined in the literature as nearly any coherent wave energy above alpha frequencies. Various researchers list different definitions. Most commonly, frequencies from about 12 to 20 Hz are accepted as beta waves.
In my experience, beta wave reinforcement is associated with a highly vigilant state of unfocused attention, a state in which one seems right on the edge of making rapid and complex abstractions, logical conclusions, calculations, observations or insights. It can be highly creative and productive.
Some researchers have reported strongly negative affective experiences associated with beta feedback. In my opinion, this is not due to any deleterious psychological implications, though it can be unsettling to some to be in such a vigilant state of attention. Rather, it is due to improper procedure and instrumentation. In my experience, when feedback is not given solely on the basis of the presence of signal power from beta-range, band-pass filters, but includes the use of autocorrelation analysis to measure waveform coherence, the experience is nearly always a positive one, sometimes even leading to elation and ecstasis. If, however, just a band-pass filter and amplitude follower are used, it is easy to give feedback inadvertently on the basis of highly erratic, incoherent, high frequency EEG activity. This can indeed be associated with negative feelings, feelings of dissociation, disconnectedness, jangling nerves, tension, anxiety, and a host of other bad experiences.
Possibly, this is experienced in conjunction with beta waves because the beta bandwidth is wide, compared to the other categories, and poorly defined in the literature. The beta bandwidth can be from 10 to 20 Hz wide in some cases. The alpha bandwidth, by comparison, is never more than 4 or 5 Hz wide. Consequently, the greater selectivity of an alpha filter makes it less susceptible to activation by irregular EEG waveforms, even though this is by no means the preferred method.
Theta is defined as relatively slow, coherent waves in a 4- to 8- Hz band. When reinforced with biofeedback, they tend to be associated with Yoga-like states of deep relaxation, or perhaps daydreaming-relatively unfocused. This state has more passive, but attentive, qualities.
Delta waves are those from about 1 to 4 Hz or below. Normally, they are present only in states of very deep sleep, states of anesthesia, or other relatively unconscious states.
These traditional categories of brainwave analysis have led, I believe, to a fragmented view of the coherent waves. I prefer to view them as occupying a continuum (see Fig. 9). In general they represent what might be termed idling states of the brain; that is, during the presence of strong coherent waves, the brain is not engaged in making spatio-temporally localized perceptions, differentiations among objects of perception, or cognitive categorizations. The brain is on the verge of doing so and is in a state of readiness to process information, whether of exogenous or endogenous origin, but is not engaged in such activity at that point. There may be a profound and strong awareness of everything in the environment, such as in the alpha state, but no abstractions are being made. One may be aware of the environment of a room, for example, but not engaged in abstracting the idea of squareness.
Fig. 9. Coherent Waves as an Integrated Continuum. Chart showing the relationship of coherent, EEG wave frequencies to qualities of states of consciousness, ranging from the relatively unconscious to the calculating states. (See the text for descriptions of these states.) It is proposed that viewing the coherent waves as part of a continuum may be better than using the rigid, traditional classifications of delta, theta, alpha and beta.
Qualitative differences in these waves are associated with different states of vigilance or alertness, as well as the propensity to process perceptions of changes in the external or internal environment in a particular mode, should they occur. At the low end of the continuum are what I term relatively unconscious states, like that represented by delta waves. Perceptions are not likely to be processed at all during the delta state. Above that we move into the theta state, during which perceptions or abstractions may be processed in a way that is semi-conscious, subliminal or dream-like. In the alpha state, of course, attention may become more highly activated, but this state is usually interrupted by the act of perceptual differentiation or cognitive categorization. During high beta production, one moves to the opposite end of the scale from the unconscious states into what I term the calculating states. Here the brain is vigilant and prepared to engage in quick, logical thinking with great speed. It races toward complex abstraction and differentiation when beta is interrupted. Unlike alpha, however, there is less emphasis on openness and on the free ranging consciousness associated with open focus. During both alpha and coherent beta production, there is a feeling analogous to that of being deeply engaged in something. Only the 'something' is not identified. During the alpha state, it is as if attention is high and equally distributed over the entire external and internal environment. By contrast, during beta, the feeling is more as if one were about to jump off a high diving board of consciousness into a particular logical or combinatoric peregrination of the mind. All these states have important analogs in the practice of music and other arts.
These long-term coherent waves may be summarized as regular-tending cycles of non-singular animation. The brain system is in a state of preparedness to process point-concentrated experience in a particular mode, but consciousness is neither ahead of the present (making predictions) nor behind the present (extracting time-bound associations); low frequencies tend toward the unconscious states, high frequencies toward the calculating states.
Sometimes the EEG will exhibit coherent energy distributed over a mixture of these band definitions. Relative levels of alpha, beta and theta bands may reflect intermediary states along the continuum described above. Usually, focusing attention will interrupt coherent waves, particularly alpha. Relations among ongoing, coherent wave categories and cognitive processing activities are not clearly established in the literature at this point. It has been suggested that complex topographic distributions of theta, beta and, to a lesser extent, alpha band intensities can reflect observable distinctions generated by a subject engaging in different types of cognitive tasks. For instance, discriminations between verbal and spatial tasks [52] and processes demanding internal mental focus and rejection of sensory input versus those requiring intake of sensory information [53] have been studied in this way. There is conflicting evidence regarding whether interhemispheric asymmetries in the coherent waves reflect engagement in different types of cognitive activity or differences in sensory intake versus sensory rejection modes. A tendency toward balanced topographic distribution of coherent waves, possibly reflecting a spread of cortical synchrony through resonant coupling of dynamical regimes, has been observed to accompany practices in meditation. Relative levels of theta, alpha, and beta amplitudes have been used in the composition Chilean Drought (which I composed with Jacqueline Humbert) to determine the audio mix played for an audience of three different vocal settings based on a text from a news account of the 1968 drought in Chile. A solo brainwave performer listens. The performer's states of consciousness, reflected by her or his relative brainwave amplitudes, influence how the news is heard in this musical setting [54, 55].
Some things can be observed in the ongoing EEG without the aid of spectral band decomposition. Detection of specific waveform signatures may also be of interest. For example, Roy John and others have demonstrated voluminous evidence for the occurrence of new patterns in the ongoing EEG that accompany learning [56, 57]. Changes in the EEG accompanying stimulus-response conditioning were observed early on in the history of this research. When an unconditioned stimulus is paired with a previously conditioned stimulus, a change from relatively low-frequency, high-voltage, synchronous (coherent ) waves to high-frequency, low-voltage, desynchronized activity occurs. As training progresses, this change becomes localized over cortical areas relevant to the learning task. Today we may be able to decompose this desynchronized activity by means of pattern analysis, extraction of ERPs, or techniques for probing extremely complex patterns from dynamical systems theory. Further work involved the use of what are termed 'tracer- conditioned stimuli' (TCS), an idea originally introduced in the Soviet Union in the 1940s and the United States in the 1950s. As an unconditioned stimulus becomes a conditioned stimulus, new patterns appear in response to that stimulus. The TCS leave a kind of modulation imprint on the EEG from certain anatomical regions of the brain and, as learning progresses, this imprint spreads to other regions originally not involved in the processing of the TCS. After the introduction of signal-averaging techniques, however, most of this kind of research became focused on ERPs.
Short-term transient waves, as contrasted with coherent waves, embody the epitome of singular experience. They are known in the literature as event related potentials (ERPs) or evoked responses (ERs). They are transient, non-repetitive waveforms associated with events that take place in the brain during the first 1,500 milliseconds (msec) or so after the onset of a clearly differentiated stimulus. Consequently, ERPs are associated with the making of a discrimination at some level in the sensory or cognitive information processing hierarchy. They represent highly singular, spatio-temporally localized phenomena, both from the point of view of the subject having the experience and from the point of view of the observed waveform. They could also be viewed as short-term resonance phenomena coupling the brain, as an electrochemical information processor, with events in the environment.
These ERPs, as recorded with surface electrodes on the scalp, are typically of very small amplitude, around 25 microvolts or so, and are buried deep inside the ongoing, complex EEG waveform. Thus, sophisticated measurement and statistical analysis techniques must be brought to bear on the task of detecting ERPs and extracting them from background noise. They are mutually exclusive with the long-term coherent waves discussed above. Since they are associated with opposing kinds of experience, ERPs always interrupt any coherent waves that may be present. Thus, ERPs reflect an important principle of perception and cognition; that is, the apprehension, perception or recognition of an entity is always achieved at the cost of exclusion of competing entities, at least temporarily.
A kind of quantum exclusionary principal of cognition reveals itself here. Recognition, as we normally conceive of it, is a discrete process. This is not to say that other modes of consciousness might not exist, in which awareness is distributed more evenly across the entities of perception or cognition. However, these are fundamentally different, requiring special terminology for their description. Thus, the long-term coherent and short-term transient waveforms are associated with polar aspects of conscious experience: non-singular awareness and highly singular differentiation. Both are fundamental to the maintenance of the viable human organism.
A large portion of the technical discussion in this paper will focus on use of ERPs in biofeedback paradigms that probe musical experiences. Feedback paradigms involving the coherent waves alone have been well described in the literature and explored extensively in artistic applications [58]. ERPs, however, have not been explored extensively in this context [59]. They offer considerable potential for applications in music, particularly as they relate to the perception and comprehension of formal musical architectures.
In this category I have placed what has not been analyzed in the preceding three categories. The EEG no doubt contains a complex, ongoing background component that is not random but is patterned with a degree of complexity to make any analysis seem an insuperable task. It is likely that as life experience progresses, aspects of this patterning evolve in a manner reflecting the self-organization of the information of experience. Complex patterns of baseline activation must build up in neuronal masses over time and in response to learning. Focused experience may interrupt these and superimpose temporary patterns necessary to deal with the localized, spatio-temporal constructs of singular events. The results of these discriminations, however, must, after an appropriate period of reverberation and widespread diffusion in the brain, submerge into the ongoing patterns. Each such experience may leave a minute tracing on, or in some way contribute to, the evolution of the electromagnetic field pulsations maintained by the organism as an integral part of itself. In some sense this electromagnetic field is an emergent, global property of the sufficiently complex self-organization of a critical number of individual, electro-chemical processing units. In discussing the representation of perceptions, John describes projection pathways of average, coordinated, spatio-temporal firings of neural ensembles from intracranial recordings, their spread of activation, and the establishment of complex gradients of ionic charge [60]. This is envisaged as a complex, three-dimensional volume of isopotential contours or convoluted charge surfaces, termed a hyperneuron. He hypothesizes that every representational system has its own unique hyperneuron, embodied as a particular distribution of energy established by the statistical properties of local ensembles contributing to the coherent spatio-temporal patterns within a volume of neural tissue. It is assumed that the emergence of a hyperneuron is dependent on the existence of a group of critical size, containing sufficiently organized processing units.
John lists several levels of organization of these processing units and their emergent properties, as follows: First-order, emergent processes are termed sensations. These result from the statistics of spatio-temporal patterns in stimulus-bound neuronal ensembles. Second-order processes are perceptions. This is the interpretation of the meaning of sensations by means of an interaction between sensations and memories. Third-order processes produce consciousness. Here a multi-dimensional representation of the state of the organism and its environment is developed. This is integrated with memories and the needs of the organism, generating emotions and programs of behavior addressing those needs. There can, of course, be many levels of consciousness. Subjective experience constitutes fourth-order information. It is derived by organizing information about the contents of consciousness and multi-sensory perceptions, memories, emotions and actions into episodes of differentiable experience. Fifth-order information results in the self. It is derived from the accumulation of personal history in long-term memory and its integration into episodes of subjective experience. Finally, sixth-order information comprises self-awareness, the interpretation of current subjective experience in the context of salient features derived from analyses of the accumulated history that constitutes the self.
Representations of multiple items on some lower levels often exhibit invariances that share a common informational feature. These invariances constitute the representation of information on higher levels. This is known as paradigmatic bootstrapping. Diverse and differentiable qualities are distinguished on higher levels from uniformly represented information at lower levels. This reflects the fundamental polarities of behavior that underlie everything the brain does: fragmentation and reduction versus synthesis and integration. The world is torn apart in order to be resynthesized and its image stored inside the accumulating history of the self in an individually unique way. There could not be individual beings without both of these processes operating in balanced concert. Access to that history is mediated by memory retrieval processes, which operate through independent mechanisms on every level of information analysis. This view of memory is distinct from the history that constitutes the self. Information retrieved from memory is added to and compared with incoming information from sensory channels to feed integrative processes on each level. The resultant organization of the organism is, on the other hand, the history that constitutes the self. As these orders of information develop, 'feedback' from higher levels down to the lower levels takes place, refining and tuning their evolving processes, while 'feedforward' from the lower levels up through the hierarchy, informing integrative processes on higher levels, takes place as well. All of this must be reflected in the ongoing life of the evolving electromagnetic entity that inhabits the body, the minute glimpses of which we record, analyze and call 'brainwaves'.
ERP research offers significant potential for probing detailed aspects of perception, cognition and conscious experience, as distinct from the more global aspects represented by changes in the long-term coherent waves [61]. In this section, I concentrate on details of the human ERP, as recorded from the surface of the scalp with traditional EEG monitoring techniques.
Highly localized ERPs can be recorded from electrodes implanted in specific neural tissue sites inside the brain. Such ERPs can often reveal information about the operation of neuron groups of limited size and topographic extent. Recent research has shown significant correlation between some of these intracranial events and conventional ERPs recorded from surface electrodes [62]. Sometimes they respond to more specialized processes [63].
ERPs normally are extracted form the ongoing EEG by means of an on-line, signal averaging computer. An epoch of the EEG lasting approximately 1,000 to 1,500 msec is sampled, beginning with the onset of a stimulus of interest or sometimes slightly before this onset, in order to capture features that may relate to expectancy, such as the contingent negative variation (CNV). (See Fig. 10 for a depiction of an idealized, auditory ERP.) In a typical experiment, a number of these EEG epochs (in the literature this number may range from a dozen or so to thousands) are arithmetically averaged to produce the ERP. The primary purpose of signal averaging is to reduce the presence of large-amplitude, background noise signals. These are presumed to vary randomly and thus will tend to be canceled out over the course of a time average. A single-trial ERP is difficult to obtain due to its exceedingly small amplitude (approximately 25 microvolts peak-to-peak) and because it is buried in large-amplitude noise. This constitutes a primary technical limitation. Averaging techniques can alleviate the signal-to-noise problem. The resultant signal is sometimes referred to as the averaged evoked potential (AEP).
Fig. 10. Form of an Idealized Auditory Event-Related Potential (ERP).
Approximately 450 milliseconds from an auditory ERP showing its most important waveform peaks. These may be analyzed using principal component analysis (PCA). Such ERPs are usually extracted from the ongoing EEG by averaging many epochs of the signal recorded synchronously with presentations of a stimulus event. All components are not always as clearly evident. P300, which may or may not occur, reflects processes of selective attention and may have endogenous origins.
Given an experimental situation in which the stimuli may be repetitive, it is possible that ERPs associated with the first few presentations of the stimulus contain the most important information. The remaining ones may merely indicate that no significant changes have taken place. Alternatively, some setups may involve significant changes over the course of trials due to learning, adaptation, stimulus generalization or categorization. An unweighted, averaged ERP measurement will be relatively insensitive to these phenomena. Consequently, various schemes intended to bias the averaged ERP so that it more strongly reflects the most recent trials are sometimes used. The calculation of an average can be weighted so that ERPs from the more distant past contribute less and less to the ongoing computation. Such weighted averaging schemes cause the ERP to reflect events from the recent past more strongly but incur the cost of degrading the signal-to-noise ratio. One simple scheme is to multiply the existing average by a factor, ß, and multiply the incoming signal by a factor, 1-ß, prior to adding them together. Another scheme involves applying a weighting factor, which decreases exponentially as a function of the time elapsed since the associated stimulus, to each member ERP contributing to the average. Eventually, this factor will decay to a negligible value and the associated ERP can be dropped. It has thus faded from the system's memory.
Sometimes it is necessary to examine ERPs from individual experimental trials. The extremely poor signal-to-noise ratio involved in such cases would make this all but impossible. However, C. D. Woody, some time ago, devised a more involved scheme for detecting single-trial ERPs [64]. It was designed to enable research into the variability of response latencies of individual waveform peaks in single-trial ERPs. Since such response latencies can vary quite considerably in different situations, across different subjects, in different task situations, or even within a specific experimental setup, such variations become a significant factor in the measurement statistics involved. This adaptive filter algorithm involves recording an ERP epoch, identifying the particular waveform peak of interest, and then recording its temporal position. This peak is then stored for later use as a template. After the next epoch is recorded, it is cross-correlated with the stored template several times in different time-shifted positions. When the largest crosscorrelation value is detected, the peaks are assumed to have lined up and the target peak is identified. The new waveform is averaged with the template, and its latency and amplitude recorded. This new average then becomes the new template for the next test. In this way, important waveform peaks, along with their latencies and amplitudes, are identified by comparison with a template being continuously refined. Woody also suggested making a rank ordering of trials on the basis of correlation with a template and separately averaging high- and low-correlation epochs to search for systematic differences among trials. The process is computation intensive, however, and only recently could be applied effectively in real-time situations with inexpensive computing resources.
There are other major limitations imposed by the signal-averaging technique. Since a number of epochs from the ongoing EEG must be sampled by the computer and combined into the deduced average, the timing of the epochs sampled must be consistent and meaningful. The computer cannot simply detect an arbitrary ERP buried in the ongoing EEG tracing. It must be told where to look in time to extract meaningful data. Consequently, the experimenter can only look for ERPs where they are predicted to occur. The most clearly defined and logical periods to look for these, of course, are during the first second or so after a clearly differentiable stimulus event. If there are meaningful ERPs to be found at points that are not as clearly defined, they will be missed. The detection of ERPs, then, can often be the result of a self-fulfilling prophecy. Nevertheless, by careful experimental design, much useful information has been culled from ERP experiments. Searching for ERPs in the ongoing EEG would require continuous matching to a repertoire of waveform templates. The computations are time consuming, the templates we have are rather imprecise in their definition, and ERPs in normal situations are subject to considerable, natural variability. This seems an obvious place to apply artificial intelligence techniques that can deal with imprecise set membership with calculated confidence ratings.
Contingent Negative Variation (CNV) is a general biasing of the EEG in the negative polarity direction. It has been theorized to accompany expectancy or anticipation, particularly when a response such as a motor action is intended to follow the expected stimulus. It may show anticipation of a cue for the orienting response. It can be specific to time, space, message content or a combination of these. Thus, CNV may be more generally described as a negative shift concomitant with the subject's anticipation of establishing a locally valid causal construct. It often precedes ERPs for expected stimuli.
CNV is easy to identify by visual inspection of ongoing EEG tracings, appearing as topographic differences. For instance, in an experiment in which a warning stimulus preceded a subsequent imperative stimulus requiring a non-discriminant motor response, a CNV, maximal over the frontal cortex, was elicited by the warning stimulus followed by a later CNV over the motor cortex preceding the imperative stimulus [65]. CNVs preceding voluntary motor actions are referred to as readiness potentials. In a later experiment, artificial CNVs, like those normally seen in the warning-imperative stimulus design, were successfully synthesized by adding components of voluntary readiness potentials and ERPs to the individual stimuli containing a negative after-wave that may reflect orienting or activation processes [66].
Operant control of components of the ERP by means of biofeedback in human beings has been explored only minimally, though there are some examples using animals [67]. During the early 1970s, at the Experimental Aesthetics Laboratory at York University in Toronto, I began to investigate that possibility and to explore applications of the results in a model for an adaptive, interactive, electronic music instrument that would be sensitive to different information-processing modalities of the nervous system [68]. In view of the relevance of various ERP components to the significance and meaning of stimuli, along with their discrimination and cognitive processing by a subject, this seemed an intriguing prospect. At that time, results were limited by available instrumentation, most importantly the lack of portable, inexpensive computing power with the speed necessary to provide meaningful results to a subject in a real-time feedback paradigm. Preliminary explorations, however, were encouraging. C.M. Nunn, then a graduate student and technical assistant in this lab, created and published an elaborate design for a feedback instrument based on detection of specific, principal component criteria in ERPs [69], along with an excellent survey of the underlying theory involved. His system included built-in capabilities for statistical processing of experimental data, as well. His intent was to compare a subject's performance in a signal-detection experiment without feedback with a situation in which feedback is provided for production of ERP components associated with correct detection of target stimuli. In this way, processes of sensory fine tuning, such as for learned pitch discriminations, could be studied.
Investigations into the nature of human ERPs have been focused on unraveling the details of their form with the hope of finding electrophysiological correlates of higher cognitive processing. The ERP can be described as a short-term, transient, slow wave, the principal components of which are spread out over roughly a 1-second time frame following the onset of a highly discriminant stimulus. It is important to note that recent research indicates this stimulus may be of external origin and thus may elicit exogenous components of the ERP. On the other hand, the ERP may accompany certain internally generated events that will elicit endogenous components. Furthermore, the ERP elicited by an external stimulus usually contains both exogenous and endogenous events. Since the ERP is not a repetitive waveform, it is not appropriate to use frequency-domain analysis methods, such as the Fourier transform, to extract important features. Rather, a time-domain-based analysis of the amplitudes and latencies of prominent peaks that are contained in the ERP and can be correlated with neurological or cognitive events is the method used to decompose the ERP. This is referred to as principal component analysis (PCA). Changes in the PCA analysis are primarily related to two broad categories of factors: the form of a physical stimulus and the significance of either an externally or internally arising stimulus.
A voluminous literature exists that details research into the significance of events observed in the PCA and their classification into meaningful categories [70]. I will attempt a summary of some of those important in the auditory ERP, since they may bear the greatest relevance for music. Many of the general principles and observations contained herein, however, have analogous principles and observations with respect to visually evoked responses. Their potential for investigation in fields such as kinetic arts, video and computer graphics are assumed to be quite significant.
The complete, idealized, auditory ERP consists of 20 or so peaks of interest (see Fig. 10). Normally, however, detection algorithms focus on one or more peaks for a given experimental situation. A polarity/latency nomenclature normally is used to identify peaks. For example, N200 would indicate a negative polarity peak occurring approximately 200 msec after the onset of measurement. I emphasize 'approximately' here to stress the fact that these latencies can vary widely in different situations and are not well defined or consistently indicated in the literature. During the first 10 msec, a series of small peaks, the so-called Jewett waves, are seen. These relate to the propagation of auditory nerve volleys on their way toward the central nervous system. Various other neurogenic or myogenic peaks follow, until about P80, which is probably generated in or around the auditory cortex in the temporal lobe. Most of these are relatively insensitive to factors of attention or any kind of conscious or unconscious cognitive processing. They are associated with propagation of raw sense-organ signals through the neural distribution network.
The effects of greatest interest for our purposes begin to occur at about N100. At this point, effects seemingly due to factors of attention come into play.
N100 is sometimes called the attention wave, although this description also applies somewhat to P200. It's peak amplitude increases when attention is directed toward stimuli in the sensory channel relevant to the situation at hand (for example, auditory or visual); it therefore represents a first stage of selective attention [71]. It can be differentially sensitive to the directing of attention toward signals from either ear [72]. It can also reflect hemispheric asymmetries, particularly in response to speech stimuli for which N100 amplitudes are greater in the dominant hemisphere [73]. It is interesting to note that the latency for N100 (100 msec) is too short for it possibly to reflect processes of attention that depend on the complete presentation of a typical complex stimulus, such as a word or musical phrase. It occurs long before the articulation of the word or phrase is complete. Consequently, it is thought that N100 may reflect the activation or directing of a response mode, rather than any finer discrimination as to what the word or musical phrase is or what its meaning might be. Examples of such a response-mode direction might be, 'respond to a verbal, language stimulus' or 'listen analytically to a musical rhythm grouping'. This kind of discrimination can be made before obtaining any knowledge of the content of the stimulus. The N100 peak is sensitive to signal detection, but cannot be used to discriminate among specific stimuli in the attended channel [74]. It increases in amplitude particularly when attention is directed toward detection of occasional, weak signals [75]. In summary, we may say that N100 reflects the action of an attentional gate and, possibly, a discrimination regarding which further sensory detection and feature extraction mechanisms to activate or which type of response mode to alert. It may reflect modality or location-specific attention. It appears when there is selective attention to a subset of stimuli and, therefore reflects the stimulus set. N100 is an exogenous component of the ERP, locked to the occurrence of a physical stimulus.
Large-amplitude P200 peaks also reflect selective attention processes, though of a type somewhat different from N100. Like N100, P200 peaks are large for stimuli that are attended to and enhanced by directing attention to relevant sensory channels, especially in the detection of weak signals [76]. P200 amplitudes do not show differentiation, however, in experiments involving attending to one or the other ear [77] or interhemispheric asymmetries in processing verbal stimuli [78]. P200 may have more to do with gating or activating a response-generating mechanism, generating a 'go' versus 'no-go' stimulus for a response process, after an attention-based sensory channel discrimination is made. P200 is also an exogenous component, always time-locked to a physical stimulus.
The N200 peak is difficult to observe because it overlaps with P200 and is often obscured by it. N200 events also differentiate relevant from irrelevant stimulus modalities for situations involving both auditory and visual sensory channels. They do not always provide differentiation among relevant and irrelevant signals within the relevant modality, however [79]. The latency of N200 has been shown to co-vary with reaction times in a vigilance task, such as detecting and responding to targets [80]. This supports the hypothesis that N200 is related to decision processes that control behavioral responses to sensory stimuli in a rather discriminant way, and must therefore represent some very early stages of cognitive processing. Other ERP component latencies do not co-vary as predictably with reaction times as N200 does, at least as shown in the literature so far. This study also relates N200 with the detection of infrequent targets. N200 is considered an endogenous component of the ERP, in that it is thought to be generated internally as opposed to being locked to external, physical stimuli. In the same vigilance study cited above, N200 peaks were observed when within a train of visual or auditory stimuli some stimuli were randomly omitted. In such a situation, N200 peaks are easier to observe because the obscuring, earlier exogenous peaks are absent. N200 may therefore be summarized as an endogenously synthesized component reflecting detection of stimuli that may or may not be physically present. However, it does not necessarily reflect stimulus identification or classification. It does act as a stimulus for response generation and further cognitive processing (see the section P300, below). It may reflect the making of preliminary decisions based on modality parameters rather than on stimulus specifics. N200 is the first peak in the ERP sequence to be observed in response to expected-but -absent physical stimuli.
The P300 peak has received by far the most attention from researchers interested in the physiological correlates of cognitive processes and conscious experience. It has a relatively large amplitude and is responsive to highly discriminant situations. It not only is responsive to relevant sensory modalities and to stimulus subsets but is elicited by detection and recognition of specific stimuli within the subset [81]. It is highly sensitive to the occurrence of infrequent events, that is, events with low probability of occurrence. In the vigilance study, P300 was associated with detection of targets, as was N200, but originated from different recording sites [82]. It is therefore considered to represent different aspects of brain function. Detection of signals is related to the occurrence of both N100 and P300. Recognition of signals is related only to P300. Clearly, recognition is only partially contingent on detection. While the earlier components, N100 and P200, are associated with detection by peripheral, sensory mechanisms of real physical signals that are being attended to, N200 and P300 may accompany the incorrect detection of absent stimuli. P300 amplitude has been shown to vary with the physical similarity between target stimuli and presented stimuli in certain situations-for example, the pitch of sounds [83]. N200 may represent the process of detection leading to response, while P300 may represent the process of classification leading to image formation and memory updating. In a signal detection experiment, P300 amplitude was shown to increase with the strictness of the criteria imposed on the classification task, independent of arousal [84]. Consequently, P300 may also reflect the quality of information received, relative to some internal measure. The progress of N200 and P300 may reflect the course of brain mechanisms building up probability descriptions of incoming events leading to response, on the one hand, and to cognitive classification and evaluation of stimulus significance, on the other. P300 is sensitive to the manipulation of psychological variables, independent from the physical aspects of the stimulus. P300 may be triggered by a definitive match between a sensory event and a stored neural template from memory. P300 is coupled with decision making in cognitive processes. The late William Chase summarized P300 by saying, "We'll call P300 the neural event associated with attention allocation for the memory control processing involved in memory reorganization" [85]. I referred to this as the allocation of energy and neural resources to the synthesis of the image, the idiolog, of an event that will be stored in memory along with all its attendant classifications and association pointers to other memory engrams [86, 87]. Donchin generally summarizes P300 as the manifestation of a routine activated whenever there is a need to update one's model of the environment in working memory [88]. Thus, P300 may represent the onset of control for the updating of memory. It may reflect the making of a decision to bring or not to bring mental processes to bear on the situation at hand.
P300 may be elicited in response to detected, omitted stimuli in a regular, stimulus train [89, 90]. John shows ERPs that allow discrimination between circumstances in which a constant, ambiguous stimulus is interpreted in different ways-for example, a vertical line being interpreted as a number or as a letter in different tasks [91].
A significant observation about this evidence is that it implies that auditory attention is not mediated by a peripheral gating mechanism but rather by a complex matrix of phenomena associated with internally activated selection mechanisms, image synthesis, retrieval and template matching processes, and stimulus-independent, perceptual decision making.
Some researchers refer to long, slow waves accompanying aspects of cognitive processing, learning, memory consolidation and pattern analysis. P300 appears with latencies too short for it to be involved in such higher cognitive processing. Consequently, one is stimulated to look beyond P300. The identification of components with very long latencies is technically difficult, however, because their effects become submerged in the ongoing EEG field complex. Researchers do not know what to look for and, therefore, how to guide their computers in the search for patterns. In an experiment designed to investigate the association of ERPs with semantically anomalous information, subjects were presented sentences in which the last word made sense, did not make sense or had some physically incongruous aspect, such as a sudden increase in amplitude [92]. An example from an experiment involving semantically incongruous endings is, "I take coffee with cream and dog," (my emphasis). ERPs were extracted for each word in the sentences. ERPs for ending words that made sense were normal. ERPs for semantically incongruous ending words exhibited a large amplitude, long latency, slow wave, termed N400. ERPs for endings with altered physical characteristics elicited a longer wave of opposite polarity, termed P560. N400 may relate to postulated, long, slow waves associated with higher processing. It could represent a phasic augmenting of the CNV, which may build up in anticipation of reaching the end of the sentence. In this case it is significant as a marker of complex linguistic processing. It may also represent a kind of cognitive 'double-take', a reprocessing of the final word in the sentence, which does not match the most highly probable predictions built up by the unfolding grammar and meaning of the preceding sentence content. P560, which accompanied the physically deviant ending words, is not well understood. However, the fact that it differs so markedly from N400 underscores N400's significance as an indicator of the cognitive activity required to resolve the semantic anomaly.
Those components of the ERP that are dependent on the meaning attached to a stimulus by a subject, that are elicited by absent (i.e. imagined) stimuli, and that are representative of memory readout are often termed emitted potentials (EPs). The endogenous components of ERPs, (i.e. N200, P300, and longer wave phenomena), are emitted potentials. In a well- known experiment, a graphic symbol that could be interpreted either as the letter B or as the number 13 was imbedded in number and letter sequences projected in subjects' visual fields [93]. The subjects were informed that the purpose of the experiment was to determine the speed with which they could name numbers or letters. ERP components-recorded from the frontal lobe, starting about 160 msec after stimulus presentation-showed significant differences, depending on whether the ambiguous stimulus was interpreted as a number or as a letter. Other experiments show that neutral or ambiguous stimuli can elicit ERPs that are typical of those elicited by stimuli that are expected to occur but in reality do not occur [94]. John has produced voluminous evidence for the occurrence of endogenous components in the ERP that accompany conditioning, stimulus generalization and learning [95, 96].
With these data, one can make a mapping of events on a timeline that begins shortly before the onset of a stimulus event and continues for a brief period afterward. Prior to the event onset, one may see EEG phenomena associated with anticipation or expectancy (CNV). Subsequent to the event onset, a relation can be seen between peak latency, or time after event onset, and the position occupied by internal events associated with specific ERP peaks in the information-processing hierarchy of the brain and its cognitive processing apparatus. The further downstream we are from the event onset, the more high-level the observed effects must be (see Fig. 11). This is obvious from the assumption that patterns of activation in neural populations will project widely, stimulating a spread of activation throughout large volumes of brain matter involved with broader and broader levels of classification and more and more refined aspects of response synthesis.
Fig. 11. ERP Peak Latencies and the Information-Processing Hierarchy. A mapping of some of the kinds of information processing taking place in the nervous system-thought to be associated with particular peaks in the event-related potential (ERP)-according to their latency or the time after a stimulus event at which they occur. Events associated with expectancy or anticipation of a stimulus-evidenced principally by a contingent negative variation (CNV) in the EEG-are also shown.
During the first 80 msec or so, we see signals concomitant with propagation of information from the peripheral sensory organ (ears), through the distribution network, and on to the auditory cortex. Then we see the effects of early gating, which may be modality specific, location specific, etc. Next, factors of attention and signal detection are observed at around 100 msec. From this point, information processing most likely branches out in several directions of endogenous pattern synthesis. Internally generated resonances, which have qualities of conscious decision making associated with stimulus detection and therefore involve memory activation, are likely to be seen at around 200 msec. (P300 comes too late to represent memory activation.) From here, information probably projects in two major directions simultaneously (see Fig. 12). One leads to the mechanisms that will assemble control signals required for the organism to generate a physical response; another leads to the mechanisms that will correlate the incoming patterns with those activated from memory tracings as well as formulate classifications and synthesize the idiolog or mental image of the event. The effects of this are observed at around 300 msec and, after, in late components and slow waves.
Fig. 12. Branching of Processes Associated with ERP Components. As the results of particular stages of sensory information processing become available, multiple, subsequent processes may be activated simultaneously. These may use the resultant data in different ways. The significance of ERP components may be affected when a stimulus acquires different meanings, appears in different contexts or is associated with particular tasks.
Many of the experiments heretofore cited involve topographic differentiations in the locus of important ERP or coherent wave recordings. I have outlined very little about EEG topography so far. However, distributions of differences in ERP components from different anatomical regions of the brain are often quite significant. The contribution of exogenous versus endogenous processes to the overall ERP varies widely for different brain regions. They can be used to track the spread of activation of processes through different regions of neural tissue and they can show different degrees of engagement of separate parts of the cortex in a given task. Topographic mapping requires, of course, that recordings be taken from many sites simultaneously. This demands multi-channel EEG recording and analysis equipment, which is often expensive and cumbersome. Furthermore, in artistic situations, it is often unacceptably restricting to encumber a subject or performer with many electrode attachments. It may be necessary for some types of measurements, however. Convenient methods of attachment and less expensive, high- quality hardware need to be developed.
An important type of data for biofeedback research involves the degree to which the cortical synchrony observed in coherent waves is anatomically widespread. A simple but useful method of showing EEG coupling between cortical potentials from different areas has been available for some time and could be applied in real-time feedback with the current generation of microcomputers [97]. This method simply classifies the EEG waveforms from several different recording sights as to polarity and direction, i.e. positive-rising, positive-falling, negative-rising and negative-falling. Using a standard information theoretic model of uncertainty reduction, coefficients of information transmission are computed to show the degree of coupling between recording sites. This method is computationally simple, fast and effective. In the study cited, coupling between sites was displayed during a variety of tasks, such as rapid, silent reading, examining the details of a picture, listening to Mozart with eyes closed, and composing a letter mentally with eyes open. Listening to Mozart was associated with the most widespread coupling.
Musicologists might recognize this method of classifying waveforms as strikingly similar to methods for classifying melodic and other musical parametric contours according to sequences of ups and downs-a technique pioneered, I believe, by Charles Seeger many decades ago [98]. I have used a related system for correlating melodic and rhythmic contours in my recent composition for percussion and computer music system, Zones of Influence, in which distances in a concept space are calculated, showing relative morphological similarities and differences among these contours [99-101]. This method of comparison reflects the degree to which the sequences of ups and downs of two shapes are similar. Recently, Polansky has developed a formal taxonomy of distance functions on shapes [102]. In this classification, morphological distances that are based on direction and that preserve order are called ordered linear direction (OLD) metrics. I suggest that such morphological metrics may prove useful in tracking the spread of activation patterns, as seen in EEG waveforms, across topological and temporal dimensions.
It is hypothesized that, during the execution of purposive behaviors, large masses of the brain are in functional communication with each other and that an increase in correlations among low-frequency macropotentials recorded from these separate areas may be seen as a result. This correlation is independent of the voltage of the macropotentials and may reflect coordination of mass neuronal processes. Pattern-recognition techniques can be applied to analyze correlations between these macropotentials and have been used to differentiate between types of mental judgments required in a visuo-motor task design [103]. This method revealed differentiations that were not evident from visual inspection of conventional, averaged ERPs.
The techniques of dynamical systems analysis are increasingly being applied to a host of bioelectromagnetic phenomena, including physiological rhythms, patterns in the neuro-musculature, the acquisition of learned physical movements, the generation of normal and arrhythmic heart-beat patterns, and systems of coupled, biological oscillators [104]. One primary technical advantage of this lies in the ability to describe complex behavior with a relatively small number of order parameters. Viewing the EEG as the manifestation of a dynamical system leads to interesting characterizations of movements among the subcomponents of the total electromagnetic pulsation pattern. (See the description of dynamics terms in the section Biofeedback-Definition and Modeling in Part 1.) If the coherent waves tend to settle into delta, theta, alpha and beta bands (and this is not absolutely certain), these bands could be viewed as basins of attraction. We could then examine the dynamics of transitions from band to band. What is the settling time under various conditions? What is the switching time from band to band? Can we more clearly characterize the perturbations that disrupt coherent waves?
Jantsch [105], in referring to the work of Freeman [106], makes the observation that the waveforms in the EEG can be interpreted as limit-cycles in the amplitude oscillations of extracellular potentials among groups of neurons. Freeman's methods emphasize analysis by means of phase-space plots, in which the dimensions map changes among EEG potentials recorded from different regions of the brain [107]. Jantsch further suggests that this limit-cycle behavior can be characterized as a primitive form of gestalt and that gestalt qualities may emerge at several levels leading up to thought and higher levels in the organization of life experience. The mind is described as the self-organizing dynamics of these phenomena on organismic, reflexive and self-reflexive levels [108]. I suggest that the EEG should be regarded as a low-dimensional projection of this complex, multi-dimensional activity. Low-level events-'limit-cycles' or 'primitive gestalts', to use Jantsch's terminology-become enfolded in the ongoing ordering dynamics of the brain on successively higher levels of organization. Thus, the results of sensations, experiences, thoughts and memories become widely distributed in the totality of the ongoing neural pulsation patterns. In part, then, the EEG is the holomovement of the electrochemical behavior of the brain. The concept of the holomovement has been explored by physicist David Bohm as a description of a primary ordering principle in the universe [109]. In this paradigm, the various energies and information associated with particular experiences are continuously being enfolded into the implicate order of the EEG holomovement. Objects of attention, either by the whole brain or by smaller neural groups, are made explicit as they are unfolded from the holomovement.
In order to highlight significant features of this holomovement, I proposed beginning with the decomposition of the EEG into the categories long-term coherent waves (low-dimensional), short-term transient waves and the complex, ongoing waves of hyperneurons (high-dimensional)-all combined with a random-seeming background. (See the section The Electroencephalogram in Detail---Categories of Useful Measures in Part 2 and Ref. [110].) This categorization is further depicted in Fig. 13. If the first three of these categories can be imagined to represent states of the EEG holomovement, a series of state transitions can be described, as shown in Fig. 14. One is naturally tempted to construct a mapping of these states in a behavioral phase space along the lines of that shown in Fig. 4. The existence of low-dimensional collective variables that can be used to describe observations on macroscopic neural phenomena has been pointed out [111]. However, a determination of acceptable order parameters for each dimension is difficult and tentative at present. The use of dimensions that scale correlation values between waveform phenomena as well as coupling of topographically or temporally distributed processes has been explored and may prove valuable [112]. Interestingly, sudden changes in spreading activation patterns, such as state changes in a phase space, have also been observed and studied in artificial and highly parallel networks of computer processors, such as the Connection Machine and the Hypercube [113]. A key determinant for such phenomena is a well-ordered network topology and a critical mass of processing elements. A region of network nodes (defined by an event horizon) that are actively engaged in a process can, under certain conditions, explosively and unpredictably expand their horizon or suddenly change their activation patterns. Groups of neurons or hyperneurons can similarly explosively spread their activation patterns or undergo sudden flips in their processing modes when certain thresholds are crossed.
Fig. 13. First Decomposition of the EEG Holomovement. A scheme for a first-stage decomposition of the EEG into four primary categories: I. the random-seeming background, II. long-term coherent waves, III. short-term transient waves, and IV. a complex, though ordered, ongoing wave. Our normal observations include only monodimensional EEG recordings. In fact, the EEG is comprised of multi-dimensional electrochemical pulsations.
Fig. 14. EEG State Transitions. Three depictions of the ongoing EEG changing states. These states are constituted when a dominant wave pattern is easily identified with one of the four primary categories for EEG decomposition. Each is associated with particular kinds of experience.
It has occurred to me that a reason it is so difficult to devise an appropriate set of dimensions for an EEG phase space with which to describe state transitions may be that the necessary dimensionality for various EEG states may not be constant. In fact, in considering the various states of consciousness associated with EEG waveform features, I have been led to the notion that the dimensionality of experience associated with these phenomena may itself be a variable mappable on some kind of axis. A tentative gesture toward visualizing such an idea is presented in Fig. 15, in which the expanding radius of approximately concentric circles represents the increasing dimensionality of experience. The diagram represents one possible projection of the concept onto two dimensions. At the center we have a dimensionless origin of the EEG holomovement. The origin of any space of this type is considered merely a conceptual anchor point. As one moves out from this origin, a region of space-time becomes defined. As the radius of the encompassed region of space-time increases, so does the order of dimensionality of the system. The inner circles contain the coherent waves, which are assumed to be associated with non-singular-tending experience of low-dimensionality. Such experience, though sometimes associated with intense awareness, does not normally produce highly differentiated perceptions subject to clear categorization through the normal cognitive mechanisms. Proceeding from the relatively unconscious states associated with delta and theta waves through alpha and the calculating states associated with coherent beta waves, we finally arrive at the outer region of complex, hyper-dimensionally tending experience (i.e. high-dimensionality), DH. This is where most of the time of daily life is spent. An ongoing stream of highly diff
