Digital Behaviors and Generative Music
Click here to download pdf version.

by Dave Burraston and Andrew Martin
Creativity and Cognition Studios
University of Technology, Sydney
Australia
andrewm [@] it [dot] uts [dot] .edu [dot] au
dave [@] noyzelab [dot] .com

Keywords
cellular automata, reaction-diffusion systems, computer music, generative art, digital art

Abstract
The use of computers in the generation of digital art and music has enabled the application of a single data source to the parameters of audiovisual modalities. This paper reports on an investigation within Electronic Arts practice where experimental works have been devised that make use of simulated natural systems. It explores essentially natural systems such as reaction-diffusion systems and the computational simulation method of Cellular Automata (CA). Contemporary artists’ practice is continually informed by scientific thought and understanding as new discoveries or conjectures are published. These are examined and presented in a non-mathematical format in the context of music composition.

Introduction
Artists and composers have commonly placed an ideal of nature within their practice. Artists’ concept of nature have been influenced by the scientific knowledge and understanding of the time. For example, Iannis Xenakis has described an “historical parallel between European music and the successive attempts to explain the world by reason” [1]. He links the deterministic nature of European music to the Platonic ideal of causality, and describes how 19th century statistical theories in physics influenced and changed our understanding of causality and reason. “It is only recently that knowledge has been able to penetrate chance and has discovered how to separate its degrees” [1]. John Cage frequently used the I Ching and produced “Indeterminate Music” based on chance operations [2]. Cage believed that the function of art was to imitate nature in her manner of operation. In the last 50 years the field of non-linear dynamic systems has become an important subject in physics, and therefore to the conceptual meaning of chance and randomness in nature.

	Figure 1: Musical visualizations of ordered, complex and chaotic CA behaviour Copyright © Dave Burraston and Andrew Martin

Since the 1960s, though, with developments in the theory of dynamical systems and computer experimental methods, rapid progress has been made in the study of these systems, and as a result many of nature’s processes have become newly understood [3].

Cellular Automata (CA) were originally conceived by Stanislaw Ulam and John von Neumann to study the process of reproduction and growths of form [4]. Konrad Zuse later proposed a concept of the universe as a type of CA computer that he termed “Calculating Space” [5]. Here Zuse poses the controversial question: “Is nature digital, analog or hybrid?” Ed Fredkin, a long-standing CA scientist, is convinced that the universe is digital (grainy) and has developed his own “Digital Philosophy” termed Finite Nature [6]. Fredkin believes that the digital mechanics of the universe is much like a CA, deterministic in nature but computed with unknowable determinism. Space and time in this view are discrete quantities, everything is assumed to be grainy. Our question is: Can this unknowable determinism be made tangible and contribute to artistic ends?

Cellular Automata
Complex systems such as logic-based CA produce global behavior based on the interactions of simple units. Their evolution is specified by local interaction rules that generate some form of ordered, complex or chaotic behaviour. This wide variety of behavior represents an important generative tool for the artist. Chaotic behavior dominates rule space, which has serious implications for the serendipitous use of these systems in artistic endeavour. An example visualization of ordered, complex and chaotic behavior converted to music is shown in Figure 1 (see page 1). Example spacetime plots are shown to the right of each music extract. The spacetime plots represent the cellular space horizontally and time evolving in discrete stages downwards. The different classes of behavior produced, whether ordered, complex or chaotic, make them interesting to artists and scientists alike. They are fascinating objects, producing more pattern than a single human is capable of observing within their own lifetime.

CA are dynamic systems in which space and time are discrete. They may have a number of dimensions, single linear arrays or two-dimensional arrays of cells being the most common forms. The algorithm is a parallel process operating on this array of cells. Each cell can have one of a number of possible states. The simultaneous change of state of each cell is specified by a local transition rule, applied to a specified neighborhood around each cell. Patterns produced by these systems were classed by Stephen Wolfram with one of four qualitative behaviors [7].

	Class 1:		Patterns disappear with time or become fixed.
	Class 2:		Patterns evolve to a fixed size with periodic structures cycling through a fixed number of states.
	Class 3:		Patterns become chaotic.
	Class 4:		Patterns grow into complex forms, exhibiting localized structures moving both spatially and temporally.

Other methods of behavior classification have been devised, an example of six categories is given in [8], although it is undecidable to assign a CA to a Wolfram class [9]. The relatively rare complex behavior of class 4 was suggested to occur at a “phase transition” between order (class 1 and 2) and chaos (class 3), termed the “edge of chaos” [10]. The concept of the “edge of chaos” and efficacy of Langton’s “Lambda” prediction parameter has also been critically re-examined in [11]. Langton produced an example schematic illustrating his view of rule space shown in Figure 2 (below left).

We have included spacetime plots of example 1D CA evolutions for each class. It is important to note on this diagram that the boundary between order and chaos contains complex behavior within it. This implies that the transition from order to chaos may occur either in a discontinuous jump, or may pass through a region of complex behavior. Langton also supported and promoted work on the global dynamics of CA [12], which offers a new perspective based on the topology of attractor basins, rule symmetry categories and rule clustering.

	Figure 2: Langton's schematic of CA rule space and example spacetime plots of their behaviour Copyright © Dave Burraston and Andrew Martin

Here one can compare basin topologies and measures between rules to gain insight into different rule behaviors. Attractor basin topology reflects the dynamics of a CA rule and can be used as a method of identifying ordered, complex and chaotic behavior. These are important concepts and more importantly not as esoteric or difficult to understand in our experience, when compared to much other CA literature.

Wuensche’s Discrete Dynamics Lab (DDLab) software allows for the exploration of global dynamics [13], as well as many other important aspects of CA and related discrete networks.

Wuensche has importantly shown that as the neighborhood size is increased, the proportion of chaotic rules rises very sharply in a random sampling of rules [14]. The magnitude of the numbers of rules is extremely large, Wentian Li has commented on the five neighbor rules:

Even if we can produce a spatial-temporal pattern from each rule in one second, it is going to take about 138 years to run through all the rules. Considering the redundancy due to equivalence between rules upon 0-to-1 transformations, which cut the time by half, it still requires a solid 69 years [15].

The total number of CA rules is a function of the number of states and the size of the neighborhood. The three neighbor rules amount to a total of 256, and are known as the elementary rules [16]. Table 1 shows a summary of the total number of rules for 1D binary CA with small neighborhoods. As the neighborhood is increased there is astronomic increase in the total number of rules. A popular method of reducing the number of rules for a chosen neighborhood size is to simply take the sum of these cells [16]. Rules computed in this manner are termed “totalistic” and are a very small subset of each rule type.

A CA state space consists of all possible global states. In a finite deterministic CA all state transitions must eventually repeat with period 1 or more. States are either part of an attractor cycle or lie on a transient leading to the attractor cycle. If a transient exists there will be states unreachable by any other states at the extremity. These extremities are called garden of Eden (goE) states. All transients leading to an attractor, and the attractor cycle, are termed the basin of attraction (boa) of that individual attractor.

An example basin of attraction is shown in Figure 3 (right). State space for a particular CA rule and size is populated by one or more basins of attraction, termed the basin of attraction field. For a deeper understanding of these concepts the reader is strongly advised to study the referenced literature, in particular Wuensche and Lesser’s book, now freely available on the Internet [12].

Reaction-Diffusion Systems
Reaction-diffusion systems were first proposed by Alan Turing [17] in a work aimed at proving that growth and form in embryology was the result of physical and chemical processes rather than Design.

	Figure 3: Basin of Attraction Copyright © Dave Burraston and Andrew Martin

Reaction-diffusion systems were later further developed in the form of non-linear equations to show realistic biological patterning and development [18]. Hans Meinhardt and Alfred Gierer formulated ‘activator-inhibitor systems’ that led to plausible simulations of the patterning of seashells [19].

	Figure 4a: A two-dimensional Reaction-Diffusion (RD) process, complete system of 256X256 cells) Copyright © Dave Burraston and Andrew Martin

	Figure 4b: Shows detail of the hexagonal cell structure Copyright © Dave Burraston and Andrew Martin

Tamayo and Hartman have proposed a model for the reaction-diffusion of clay particles in which the synthesis of inorganic molecules takes place, producing the necessary combination of elements for the origin of life to occur [20]. Tamayo and Hartman’s approach was to utilize CA for the simulation of reaction-diffusion systems. DDLab software uses three-state CA within a totalistic rule set to demonstrate Reaction-Diffusion dynamics. An example of a two-dimensional RD system from a binary CA is shown in Figure 4a (above left), modeled with DDLab software. The system contains 256x256 hexagonal cells, which can be seen in detail in Figure 4b (above right). The visualization shows a single time frame in the systems evolution. The temporal history of part of the system is shown in Figure 5 (below), where striking patterns are seen.

	Figure 5: Slice of a two dimensional RD system, the system is stretched out horizontally showing time evolving vertically downwards. Copyright © Dave Burraston and Andrew Martin

Within the domain of generative music access to a variety of behavior is essential. CA have played a key part in generative music for many years [21] [22]. The reflective practice method of research has also been utilized to investigate and describe generative music with CA of all types of behavior [23][24, 25]. CA behavior is often described based on a subjective assessment of its spacetime images. A thorough account of all behavior, automatic classification by various parameters and example rules is given in [14, 26]. Complex behavior arouses particular interest when studying these systems. According to Wuensche, complex behavior emerges (self-organizes) in spacetime from random initial conditions to form gliders, particles or self-sustaining patterns existing over a uniform or periodic background.

A glider can be seen as a dislocation or defect of this background. At least 150 cells in a 1D system, and more cells for larger neighborhood sizes, are required before complex behavior can emerge. Attractor basin topology will typically consist of long transients of complex behavior leading to small attractor cycles. An example of this attractor basin topology is shown in Figure 6. The left shows a subtree section entering the attractor cycle and the right image shows 8 transient levels of a subtree from a randomly generated seed.

	Figure 6: Complex rule behavior: Partial subtree of an attractor cycle (top) and a section of eight transient levels from random seed (above). Copyright © Dave Burraston and Andrew Martin

	Figure 7: Example spacetime plot of complex behavior (top) and a small section close up (middle). Locating an attractor, the shaded area, after several thousand generations (above). Copyright © Dave Burraston and Andrew Martin

An example spacetime plot of a complex rule is shown in Figure 7 (left). A close up section of the lower right edge is shown in Figure 7 (middle) where the complex behavior can be seen more clearly. In order to capture the complex behavior well before an attractor cycle is entered, it is helpful to use DDLab’s histogram analysis function. This gives information regarding how many timesteps are required before the attractor cycle is entered. Figure 7 (right) shows DDLab locating an attractor, the shaded area, after several thousand generations. The attractor in this case is a simple shifting sequence of states.

	Figure 8: Musical mappings of complex behavior. Chord-like structures including durations (top) and single note sequence including loudness (above). Copyright © Dave Burraston and Andrew Martin

A reasonably universal process for experimenting with generative music production by CA was suggested in [24]. Mathematica [27] can be used to generate CA data and convert the binary output into decimal format text files. Code for generating data files using Mathematica’s built in Cellular Automaton function is given in [24]. This code is quite general and can be used for any size, generations and number of data files. Musical mappings of complex behavior are shown in Figure 8. The mapping process was implemented using AC Toolbox [28] controlling note, velocity and rhythmic timing. A linear conversion was utilized in order to rescale the large data range to desired compositional ranges. Chord-like structures can be produced if the note onset time mappings are particularly low valued, as seen in Figure 8 (above left). Particular attention should be given to the single note sequence shown in Figure 8 (above right), a result of the linear mapping from the large data range. A deterministic CA will never enter an attractor cycle and then depart from this without reseeding, or changing the rule in a time dependent manner. A diversity of material has been produced from CA, and many of the authors’ compositions are freely available on-line at [29]. The results of the work described in this section have inspired new methods of sampling rule space [30] and alternative views of the CA rule space structure [31].

A recent generative music performance piece Babelitis has been realized to explore and question the universe as a computer concept, comparing this quest to the Tower of Babel [32]. It is realized through a hybrid of mediums, analog/digital synthesis and old/new computer technology. Events in the universe are mapped to microtones and real/synthetic speech events. Synthetic speech events using allophone sequences are created by mapping universe states over time. Real speech events, in the form of sound samples, are modified over time in a similar manner. This has the effect of creating a babble leading to the hypothetical medical condition Babelitis.

RD Music
The behavior of reaction-diffusion systems has been utilized in computer animation [33] and experimental digital art and computer music [34, 35]. Investigations into the application of Reaction-Diffusion (R-D) systems to generative music has produced MIDI based algorithmic compositions [36] where a number of values are derived from each cell in the system which are then assigned to control tempo, note durations, note onset, note velocity and note value. The initial approach was to utilize partial differential equations differing from the pure computational logic approach of CA. A two dimensional grid of cells was scanned from left to right and top to bottom, giving the effect of “self-modifying repetition”. Similar systems were also applied to the synthesis of sound [37, 35]. These investigations were primarily with 1D R-D systems of both 24 and 32 cells. The output of the R-D systems created scores for an additive/FM CSound system, the number of FM instruments being equivalent to the number of R-D cells. The temporal nature of timbre was emphasized in this study, with instruments having R-D control mappings for peak amplitude, FM rate, FM depth, attack and decay characteristics. In the piece cicada [35] multiple R-D systems have been connected in a tree branching structure in order to determine all audio parameters and event structure for an entire sonic composition.

Further investigations have taken place where the behavioral characteristics associated with the 'state' of the R-D system, such as equilibrium, stability and chaotic oscillation, control synthesis parameters when excited through interaction [38]. The oscillation of the sum of changes within a stable system is illustrated in Figure 9 (right). Further investigations have been carried out with this type of system where interaction with the system alters the system state and thereby the audio and visual control parameters [34]. A characteristic of this type of interactive work is that the results are stimulus/response (rather than cause and effect) where the outcome is undetermined.

	Figure 9: A reaction-diffusion system standing waveform of cell activity over time. Values are then utilized as sound synthesis parameters Copyright © Dave Burraston and Andrew Martin

Conclusions
Unknowable determinism can achieve material tangibility and contribute to artistic ends. Perhaps it is not necessary to understand the algorithmic details in order to make use of it artistically. However, the art constructed by automata and human symbiosis should be viewed as a collaborative process by the human. The artist must be prepared to investigate the theoretical background of these systems in order to successfully employ their digital behavior within compositional strategy.

The vast behavior space of automata challenges the limits of human perception. It will remain an untamed wilderness for artistic application unless practical background concepts are acknowledged, and actively researched in an artistic context. This philosophy should encourage the artist to understand automata, even though the automata is unlikely to appreciate art.

Acknowledgements
The author’s would like to thank Andrew Wuensche for his continued support in allowing us to use data and present images made with DDLab software.

References and Notes
1. I. Xenakis, Formalizsed Music: Thought and Mathematics in Composition (Bloomington/London: Indiana University Press, 1971).

2. J. Cage, Silence (London: Calder and Boyers, 1973).

3. Y. Kuramoto and H. Mori, Dissipative Structures and Chaos, English edition (Berlin: Springer-Verlag, 1998).

4. A. Burks, (ed.) Essays on Cellular Automata (University of Illinois Press, 1970).

5. K. Zuse, Rechnender Raum (Vieweg, Braunschweig, 1969); translated as “Calculating Space”, AZT-70-164- GEMIT MIT Project MAC

6. E. Fredkin, Finite Nature (1992), http://www.digitalphilosophy.org

7. S. Wolfram, “Universality and Complexity in Cellular Automata”, in Physica D, Vol. 10D, pp. 1-35 (1984).

8. W. Li, N. H. Packard, and C. G. Langton, “Transition Phenomena in Cellular Automata Rule Space”, in Physica D Vol. 45, pp. 77-94 (1990).

9. H. V. McIntosh, What Has and Hasn’t Been Done With Cellular Automata, http://delta.cs.cinvestav.mx/~mcintosh/newweb/marcowhat.html (1990).

10. C. G. Langton, “Life at the Edge of Chaos”, in Artificial Life II, Proceedings Vol. X. SFI Studies in the Sciences of Complexity, Addison-Wesley (1991).

11. M. Mitchell, J. P. Crutchfield, and P. T. Hraber, Dynamics, Computation, and the “Edge of Chaos”: A Re-Examination, Santa Fe Institute Working Paper 93-06-040 (1993).

12. A. Wuensche, and M. Lesser, The Global Dynamics of Cellular Automata: An Atlas of Basin of Attraction Fields of One-Dimensional Cellular Automata, Addison-Wesley (Available in PDF from http://www.ddlab.com) (1992).

13. A. Wuensche, The DDLab Manual Discrete Dynamics Inc. (Available in PDF from http://www.ddlab.com) (2005)

14. A. Wuensche, “Attractor Basins of Discrete Networks”, Cognitive Science Research Paper 461 (Univ. of Sussex, D.Phil thesis, 1997).

15. W. Li, “Complex Patterns Generated by Next Nearest Neigbors Cellular Automata”, Computers and Graphics Vol. 13, No. 4, pp. 531-537 (1989).

16. S. Wolfram, “Statistical Mechanics of Cellular Automata”, Reviews of Modern Physics, Vol. 55, No. 3, pp. 601-644 (1983).

17. A. Turing, “The Chemical Basis of Morphogenesis”, Philosophical Transactions of the Royal Society (London, 1952).

18. P. Ball, The Self-Made Tapestry: Pattern Formation in Nature (Oxford University Press, 1999).

19. H. Meinhardt, The Algorithmic Beauty of Seashells (Berlin: Springer-Verlag, 1996).

20. P. Tamayo and H. Hartman, “Cellular Automata, Reaction-Diffusion Systems and the Origin of Life”, Artificial Life (Addison-Wesley, 1988).

21. D. Burraston, E. Edmonds, D. Livingstone, and E. Miranda, “Cellular Automata in MIDI based Computer Music”, in Proceedings of the 2004 International Computer Music Conference (2004).

22. D. Burraston, E. Edmonds, “Cellular Automata in Generative Electronic Music and Sonic Art: A Historical and Technical Review”, Digital Creativity Vol. 16, No. 3, pp. 165-185 (2005).

23. D. Burraston, E. Edmonds, “Global Dynamics Approach to Generative Music Experiments with One Dimensional Cellular Automata”, Proceedings of the 2004 Australasian Computer Music Conference (2004).

24. D. Burraston, “Composition at the Edge of Chaos”, Proceedings of the 2004 Australasian Computer Music Conference (2005).

25. D. Burraston, “One Dimensional Cellular Automata Musical Experiments with Max”, Proceedings of the 11th International Conference on Human-Computer Interaction, HCI International, (2005).

26. A. Wuensche, “Classifying Cellular Automata Automatically: Finding gliders, filtering, and relating space-time patterns, attractor basins, and the Z parameter”, Complexity, Vol.4 No.3, pp. 47-66 (1999).

27. S. Wolfram, Mathematica, http://www.wolfram.com/ (2005).

28. P. Berg, AC Toolbox, http://www.koncon.nl/ACToolbox/ (2005).

29. D. Burraston, http://www.noyzelab.com (2005).

30. D. Burraston, “Variety, Pattern and Isomorphism”, Proceedings of the Third Iteration Conference, Monash University, Australia (2005).

31. D. Burraston, “Structuring Cellular Automata Rule Space with Attractor Basins”, Proceedings of Generative Arts Practice Symposium 2005, University of Technology, Sydney (2005).

32. D. Burraston, “Babelitis”, Proceedings of Generative Arts Practice Symposium 2005, University of Technology, Sydney (2005).

33. G. Turk, “Generating Textures on Arbitrary Surfaces Using Reaction-Diffusion”, Computer Graphics Vol. 25 No. 4 (1991).

34. A. Martin, “A Simulated Natural System for Electronic Art”, HCI International, edited by G. Salvendy, (Las Vegas, Nevada, U.S.A.: Lawrence Erlbaum Assoc Inc., 2005), CD-Rom.

35. A. Martin, “Reaction-Diffusion Systems for Algorithmic Composition”, Organised Sound, Vol. 1, No. 3, pp. 195-201 (1996).

36. A. Martin, “Two Dimensional Reaction-Diffusion System for MIDI Composition”, Synaesthetica ’94 Proceedings, Australian Centre for the Arts and Technology (ACAT), Australian National University (1994).

37. A. Martin, “Sound Synthesis for One Dimensional Reaction-Diffusion Systems”, Proceedings of the ACMA 1995 Conference, Australasian Computer Music Association (1995).

38. A. Martin, “Interactive Sound Synthesis with Reaction-Diffusion System Instability”, in Interaction: Systems, Practice and Theory
 Proceedings, University of Technology, Sydney (2004).

Author Biographies
DAVE BURRASTON, AKA Dave Noyze & Bryen Telko, is an artist, specializing in the areas of chaos, complexity, non-linear systems, applied to audio and visual mediums. He has been involved in technology and electronic music since the late 70s. He turned to electronic music after studying classical piano from the age of seven. His interest in chaos and complexity began in 1985, experimenting in BASIC on a Dragon 64 computer. He worked for 19 years at BT, spanning such diverse research areas as Artificial Life, Data Visualization, Spatial Audio/MIDI systems, Virtual/Augmented Realities and Environments. He recently completed a custom built Digital Signal Processor for Richard James (Aphex Twin). He is currently working on a Ph.D on Generative Music and Cellular Automata at the Creativity and Cognition Studios, University of Technology, Sydney.

ANDREW MARTIN is currently a Ph.D candidate at the Creativity and Cognition Studios, University of Technology, Sydney. His studies concern Natural Systems and Generative Music. In 1996 he completed a Masters degree at the Australian Centre for the Arts and Technology, Australian National University, where he completed research into Reaction-Diffusion Systems for Algorithmic Composition. Earlier Andrew graduated from Sydney College of the Arts in Visual Arts where major studies were in photography and time-based media.

Citation reference for this Leonardo Electronic Almanac Essay

MLA Style
Burraston, Dave and Martin, Andrew. “Digital Behaviors and Generative Music.” “Wild Nature and the Digital Life” Special Issue, Leonardo Electronic Almanac Vol 14, No. 7 - 8 (2006). 30 Nov. 2006 <http://leoalmanac.org/journal/vol_14/lea_v14_n07-08/dburrastonamartin.asp>.

APA Style
Burraston, D. and Martin, A. (Nov. 2006) “Digital Behaviors and Generative Music,” “Wild Nature and the Digital Life” Special Issue, Leonardo Electronic Almanac Vol 14, No. 7 – 8 (2006). Retrieved 30 Nov. 2006 from <http://leoalmanac.org/journal/vol_14/lea_v14_n07-08/dburrastonamartin.asp>.