Andrew Burrell
Academic, Faculty of Design Architecture and Building, University of Technology Sydney, NSW, Australia
Email: andrew.burrell@uts.edu.au
ORCID: 0000-0002-1690-7542
Rachel Hendery
Academic, Western Sydney University, Digital Humanities Research Group, Penrith, NSW, Australia
Email: R.Hendery@westernsydney.edu.au
ORCID: 0000-0003-0960-8859
Danièle Hromek
Designer and Researcher, Budawang/Yuin, Surry Hills, NSW, Australia
Email: daniele@danielehromek.com
ORCID: 0000-0001-5492-9657
Reference this essay: Burrell, Andrew, Rachel Hendery and Danièle Hromek. “Nura Yaman (‘Country Speaks’): Language, People and Place in Serious Games.” In Language Games, edited by Lanfranco Aceti, Sheena Calvert, and Hannah Lammin. Cambridge, MA: LEA / MIT Press, 2021.
Published Online: May 15, 2021
Published in Print: To Be Announced
ISBN: To Be Announced
ISSN: 1071-4391
Repository: To Be Announced
Abstract
Layered Horizons is a series of serious virtual reality (VR) games [1] that entice users to playfully explore linguistic information across the Asia-Pacific. This essay focuses on Barrawao, a version of the game based on the region now known as Sydney, Australia—a region consisting of complex language ecologies, in which speakers are often multilingual in a variety of traditional languages, and are connected to this traditional linguistic landscape through protocols, beliefs, and identity. This essay argues that interfaces for games generally are influenced by a ‘monolingual mindset’ [2] which leads to simplistic models of language, with each mapped to a single region. Games are localized by swapping one language for another, without regard to cultural considerations or the realities of the Country (e.g. people, place, linguistic and cultural environment) in which the languages are embedded. Translations of game content and interfaces are often generated by machine learning or other automatic processes which, disconnected from context, can reproduce frameworks of colonialization and globalization. This essay considers the interrelationships between machine, human, language and environment, and discusses the ethical and practical impacts of machines mediating between language and Country. We argue that, if co-created with people who have a deep knowledge of the physical and linguistic landscape, VR provides opportunities to mitigate a potential disconnect, through the embodied experience of the game––the literal use of the body as an interface––and the recreation of place in a virtual world to provide critical context for language. We relate the responsibilities people have to their Country in the physical world with the responsibilities we have to our machine worlds of code and data, and connect this to the concept of Data Sovereignty.
Keywords: Linguistics, digital humanities, research through design, virtual reality, serious games, first peoples, Country
Perspectives
Culture is the total of ways of living, including knowledges, customs, languages and behaviours built up by a group of people or society, which is passed on from one generation to the next. Culture defines us and makes us who we are. Cultural identity is the sense of belonging to a distinct group. First Peoples’ identities are deeply linked to culture, community and the land, and is a key factor to health and wellbeing. Language and culture are intrinsically intertwined. Talking or writing about language requires us to think, talk and write about culture, and to be aware of the cultural perspectives we bring to our perception of language. We do not want to talk about culture as though it is a third party to our project, but rather reveal the cultural perspectives inherent in our understanding of language and technology and that therefore influenced the direction of the virtual reality project we will be discussing in this essay. For this reason, we begin with cultural introductions.
Danièle Hromek:
I am a Budawang woman of the Yuin people, a Saltwater woman. I was born on Gadigal lands (now known as Sydney) where many of my cultural––and linguistic––connections lie. I grew up on Bundjalung lands on a farm near the beach and then Awabakal lands by a saltwater lake. I feel connected to all the places of my heritage, each with differing responsibilities, from a spiritual pull to obligations of care to a place of nurturing. I have been fortunate to grow up knowing how to walk in two worlds, learning how to relate (and speak) to both my Aboriginal relatives and my European relatives. I subsequently discovered this is called ‘code switching’, though for me it is more about relatedness and learning how to be understood. My Budawang Ancestors spoke Dhurga, which I understand to be very similar to a number of surrounding languages including Dharawal and Dharug, both spoken in the Sydney area. Like my Ancestors, who needed to be multilingual, I am being taught a number of different languages by Elders. It is painstaking work as word by word I learn and replace words in First Languages (henceforth Languages). As many Indigenous groups, the Budawang people are named after Country; the burrawang or buddawong is the name for the Macrozamia spiralis cycad, a plant Dhurga-speaking women had a specific responsibility towards in terms of cultural burning.
Andrew Burrell:
I grew up on Dharug land in the lower Blue Mountains, although it would not be until I was an adult that I would begin to understand what this meant. I was always taught that my town’s name meant “place of the eagle” in Aboriginal (sic), though it was never made apparent what language this came from, or even that there might have been many Indigenous languages. Any education I was provided as a child on the history of the land I was a settler upon was either framed via exotic othering or a deeply racist and biased colonial history. It would also take many years to start to come to understand what it means to be a settler on stolen land, a process of understanding that I know will never be complete. It was only very recently that I began to understand the relationship between Language and Country, and what it means for me, as a settler, to be speaking English here, and that the lack of connection between English and this Country goes part way to explain why my own connection to Country feels much more embodied and ‘unspeakable.’
Rachel Hendery:
Even as a Pākehā (white) child in Aotearoa (New Zealand), I was taught that my pepeha, my introduction to be used in a mihi, should include my local mountain and my river as part of my whakapapa, or ancestry. But “what is your mountain?” “What is your river?” Even “where are you from?” are questions I have difficulty answering, for a number of complicated and intertwined reasons: my whiteness; the nomadic nature of my childhood, the fact that I am an adoptee; adopted from a birthmother who herself was an adoptee. The closest I can give to a whakapapa is a list of the places I have lived: always as a guest in someone else’s space. I grew up on land belonging to the Ngāti Whātua-O-Ōrākei, Ngāti Ranginui, Ngāti Kahungunu ki Heretaunga, Ngāti Toa Rangatira, and Ngāi Tahu iwi. As an adult, I have lived in a number of countries, now in what is now called Australia, first on the land of the Ngunnawal people, and more recently on Darug land. The kind of Country that feels like home to me includes snowy mountains and cold rivers, green pasture, wet bushland, tūī song, snow and surf, but not a specific mountain or specific river, or a single patch of bush. In many ways this feels like I am missing access to some part of myself. While I often think about place, space, rootedness, and identity intellectually, for me a project like the one we discuss here is also an attempt to relearn or awaken a more personal connection to Country.
Language and Country
In the Call for Papers for this special issue, the editors suggested that “Language is a technology, as theorists including Martin Heidegger and Marshall McLuhan have argued, and yet its manifestation in both speech and writing is fundamentally human-centred: anthropological.”
In this paper we will present a perspective that disagrees with these ideas, offering a less anthropocentric viewpoint of language. The way First Peoples in the Australian continent think about Language is that it belongs to Country, and it is Country that mediates between humans and Language. [3] Language is not a human artefact, or a technology. Language comes from Country, and we have to be communicating with Country, and listening to how Country communicates with us. [4] As the concept of Country is central to an understanding of this paper’s premise, it is appropriate to introduce it by quoting from Uncle Greg Simms’ description of Country, that is central to the Barrawao experience:
Country is a place of belonging. We acknowledge Country because being Aboriginal, or even non-Aboriginal, we have to respect the land wherever we go. [5]
Country, for Aboriginal peoples, relates not only to the cultural group and land to which they belong, it is also their place of origin in cultural, spiritual and literal terms. Country includes not only the land but also waters and skies, and incorporates the tangible and intangible, knowledges and cultural practices, identity and reciprocal relationships, belonging and wellbeing. Country also includes people, more-than-humans, flora and non-breathing entities. Language, with culture, originates from Country. It was born of the land and thus perfectly describes everything within that Country. In fact, all entities of Country including animals, plants and non-humans communicate, and while it is not always our responsibility to know everything that is being communicated, as humans we must be receptive to the signals of Country. [6] From this perspective, it therefore is not the case that the manifestation of language is “fundamentally human-centered.” In the call for papers, the editors also do then question the centrality of the human to language, such that ultimately we do not believe our perspective is so different from that which they propose. However, they imply that this reduction of anthropological centrality is due to recent technological developments that erode the boundary between human and machine and create situations where language is used for communication machine-to-machine. We would suggest instead that some cultures have always taken the view that language specifically, and communication more broadly, is not something that humans own or create.
A virtual reality project such as this paper describes (Barrawao [7]) can act as a bridge between these concepts. It is a technology, but it has a relationship to place, reminding us of our obligations to Country at the same time that it educates about Language. It places language back in the appropriate spatial context, allowing users to connect with Language, land and technology at the same time. In this way, it is not language that creates an interface between humans and machines, but machines that allow the recreation of an interface between Country, humans and Language.
Background to the Project
This paper is intended as reflections that grow out of our experience creating a project that is one of a series of virtual reality (VR) serious games. The first in the series, Glossopticon, [8] was created by Burrell and Hendery, with input from Nick Thieberger of the PARADISEC archive, [9] in order to allow users to playfully engage with Pacific language material from said archive. Later, as part of the project Waves of Words, [10] we built on the Glossopticon experience to create a more flexible VR platform, Layered Horizons, for researchers to bring together and explore linguistic, cultural and archaeological data. More recently, a new branch of this project has been developed in a collaboration by the authors of this paper, as well as additional team members Shannon Foster [11] and Louisa King, as a serious game that allows users to experience some Languages of the region now known as Sydney. While these visualizations could be described in many ways, the phrase ‘serious game’ highlights the fact that they make use of many conventions of 3D first-person games, with a goal of both sharing and understanding relationships within information. For brevity we will simply use the term ‘game’ throughout the remainder of this paper. All three games in this series have in common that they spatialize and sonify (primarily) archival language data, and that the basic experience for the user involves exploring a map. They differ with respect to the features laid out in Table 1. In this paper we will primarily be discussing Barrawao, but we will occasionally compare and contrast it to the other games in the series.
Linguistic Background
The continent now known as Australia and the surrounding regions consist of complex language ecologies, in which speakers historically were and in many cases still are multilingual in a variety of First Languages, [12] and most of whom are still connected to this traditional linguistic landscape through protocols, beliefs, and identity.
The context for Barrawao specifically is the now-Sydney region. Now-Sydney is home and Mother to a number of Aboriginal groups. It has long been a destination and a space of movement as peoples from the north, west and south travel to the area, undertaking cultural care of Country obligations, staying with kin on the way. These groups maintain enduring connections with and responsibilities towards the area, and have differing names and Languages, which originated from oral traditions with multiple dialects. Colonizers have not always well understood Aboriginal ways of forming words, thus there are many ways of saying and spelling them. [13] Included groups are the Dharug (Darug, Daruk, Dharook, Oharruk), Dharawal (Darawal, D’harawal, Tharawal, Turuwal, Turrubul), Eora (Iyora, Lyora, Iora), Gundungurra (Gandangarra, Gundungera), Deerubbin (Dyarubbin, Dooraban, Deerabbun), Ku-ring-gai (Guringai, Kurig-gai, Kuringgai), Yuin (Djuwin, Djuuwin, Juwin, Yuwin). Systemic colonial practices of erasure and assimilation have caused complicated contestations between some of these groups triggering power struggles that disassociates those with long connections and belonging to place. [14] While it is acknowledged that now-Sydney is a contested space, it is not the role of this writing to address that contestation, rather to acknowledge it also as a shared space. Furthermore, in Aboriginal knowledge systems and worldviews there are many ways of knowing, many truths and diverse perspectives. Likewise, there are many truths about places containing diversity in the knowledges, stories, histories and understandings of that place. All are respected and acknowledged within these words. [15]
Most parts of the world are multilingual, layered with multiple speech communities, within which it is also usual for individuals to speak multiple languages. [16] English native speakers of European descent, in the United Kingdom, in North America, and in Australasia, particularly, are in fact noteworthy among the world’s population for their frequent monolingualism, even as they are often today embedded in cities where a multitude of languages co-exist. [17] The prominence of Silicon Valley in the history of software development means that this monolingual (and unknowingly monocultural) mindset has embedded itself into our technology as well. [18] This mindset presents itself right down to the languages used to program most contemporary technology. While many attempts have been made to create scripting and programming languages using languages other than English, the fact remains that the written English language is the base of all of the dominant programming languages. [19] While much work is being carried out in companies such as Google around adapting products to languages other than English, this continues frequently to be conceptualized in terms of localization: swapping one language out for another. As just one example, it is only recently that mobile keyboards have added functionality for switching easily back and forth between multiple languages within a single text message, and text prediction still does not account for common patterns of code-switching of the kind described by Srivastava et al. [20]
In our introduction above we argued that it is impossible to discuss language without considering culture. This is a fairly uncontested view. Linguists generally consider cultural context in their linguistic descriptions and analyses rather than leaving that entirely to the anthropologists or sociologists. In contrast, discussions of technology frequently take place as though it can somehow entirely be divorced from culture. Computer scientists, software developers, and allied specialists leave discussion of technology’s cultural context to cultural studies scholars and sociologists to consider. This results in a situation where the people making technology and driving technological change are not the people reflecting on its cultural context. In some ways, however, the consideration of cultural context, cultural significance and effect on culture is even more important for technology than for language, because of the power that technology has over our lives.
Automatic Generation of Language in Games
“Localization”
In part because of this ‘monolingual mindset’ that Silicon Valley has inherited, interfaces for games and other software generally simplify questions of language to a model in which each user speaks a single language. Further, developers often assume a nearly one-to-one mapping between language and region, rather than the complex language ecology in which users are actually embedded. [21] A more realistic approach would rather be a mix of voices and languages in a place, with fuzzy boundaries between locations, which do not map neatly onto political divisions such as state, country, or even tribal land ownership boundaries.
One of the consequences of the dominant simplified model is that a game can be ‘localized’ in a relatively automated process, by simply swapping out user interface text or audio with a corresponding string or audio file in the target language. In some software and some languages, the automation process goes even further, and translations may not even be human-generated, but rather based on machine learning approaches. Crowdsourced translations are another popular means of internationalizing game content using one of a number of internet services; but again, without consideration for cultural context this ends up being a processing of mapping one language directly onto another, potentially with the creators of the initial text and the translators of the text having never communicated intent directly, but rather through the mediating service. [22]
Semi-automated Selection of Audio
In the first game in this series, Glossopticon, influenced by this ‘monolingual mindset’, we experimented with auto-generating audioscapes from audio files in the PARADISEC online repository. We began with a list of languages and their centroid coordinates, as well as north, south, east and west boundaries for each language. We labelled each centroid point with the corresponding language and some metadata (number of speakers, kinds of archival documentation available), and intended to retrieve from PARADISEC a loop of speech in that language to play in the background when users entered the region in the virtual world.
For each location then, a language audio file associated with it was retrieved from PARADISEC, and a short snippet bookended by pauses was automatically identified. When the files were later checked by a human, however, a large number of them turned out to be either multilingual—containing a mixture of languages of the region, or a mixture of traditional languages and linguae francae—or they were the ‘wrong’ language; i.e. not the main traditional language one would associate with the coordinates at which they were recorded. These therefore did not match the language labels on the map, and had to be discarded. In the end, the task of identifying snippets with ‘the right’ language for each location was done much more manually.
As a purely technological problem, this would no doubt be solvable, for example using machine learning classification algorithms to sort audio snippets [23] and to identify and exclude those that contained more than one speaker (“speaker diarization” [24]). Other logistical problems such as the accidental inclusion of inappropriate content (e.g. offensive language, secret or sacred information, personal details) could no doubt also be surmounted. The PARADISEC archive includes, where available, time-stamped transcripts and English translations of the audio files, which could be automatically extracted alongside the audio and scanned for potentially problematic keywords.
Here, however, as so often in the world of technology, the barriers to automation of language, to speaking machines, are not just logistical. They are not only about how well we can solve technical problems, but rather they are also cultural, and about ways of thinking about language and technology. Further concerns with even the semi-automated approach to generating audioscapes that we took in Glossopticon include the danger of relegating important narratives or conversations to ‘background noise’, and reproducing biases in whose voices are privileged in the historical record. As former, technical, barriers to speaking machines begin to be surpassed, it is time to think more carefully about these ethical and cultural risks and how they might be managed.
Just as the hand of globalization reaches out from Silicon Valley and begins to define lives in other, very different parts of the globe––rewriting them to look more like itself––speaking machines embed and inscribe a language ideology that is disconnected from culture, and disconnected from Country. These von-Neumann machine style processes of globalization are a continuation and extension of colonization [25] that risk inflicting similar damage on both the colonizers and the colonized. Gretchen McCulloch highlights this when she points out that due to the dominance of English and a number of other languages with “big internet presences” many languages, including some described as “relatively substantial” are under threat. [26] The statistic she quotes here, pointing out the disparity between the around 7000 languages spoken in the world today and the 100 languages that Facebook’s user interface is available in, is also telling. The internet too is rewriting itself to look more like itself. These processes are not hypothetical future risks. Disconnection from our relationship with Country is at the source of real, practical, existential threats humanity already faces, for example in the ways mismanagement of the environment (one aspect of Country) has led to our current climate crisis.
Use of Language for Symbolic Purposes vs. Communication with the User
Other ethical and logistical issues arise when we consider language and the user interface. In Layered Horizons, many of the Languages of the region in question are no longer fluently spoken, therefore the communicative aims of our game cannot be met by simply swapping out English for Indigenous Languages. Using the languages of colonization for the interface is equally inappropriate, particularly when one of our aims is for users to better understand the relationship of Language and Country.
One particularly difficult aspect, regarding the visual interface, was how we might typographically represent Indigenous voices speaking Indigenous words within a virtual representation of Country. We understood that whatever choices we made here would be tied to western typographic traditions, and alternatives, such as representing words as images or 3D models would not relieve us of relying on western traditions of representation. The affordances offered by the virtual environment did however allow us to explore text within the virtual space of the experience (and hence off the page and back into a representation of Country). Ultimately, our choice to use spatialized and interactive typography – emerging from the virtual ground, constantly in a state of movement, but within reach of an embodied interaction – reflects the notion that Language is from Country, and in doing so, brings our wide intended audience to this understanding.
In creating Barrawao, we struggled with the question of how to communicate with the user in this situation. One of the protocols common to many First Languages of the region the game represents is that people should speak the Language belonging to the Country on which they currently are. This meant that many First Peoples of the region knew enough to communicate in the Languages of the places through which they were likely to travel. Yet, although users of the game are likely to be located in now-Sydney, we cannot today assume that they will have any knowledge of the First Languages of this region. We anticipated a broad range of users of this game, including members of the communities whose Languages are represented, but also users from other First Peoples’ communities, as well as non-Indigenous users, both Australian and international.
Representing the importance of the relationship between Language and Country is easier if it begins with a connection between them that all viewers are likely to be familiar with. Placenames form a very visible bridge between place and language in the world around us. The opening scene of Barrawao therefore uses placenames in an attempt to introduce viewers to, or remind them of, the connections between Language and Country. The viewer flies over the familiar, yet fragmented and abstractly layered coast and harbour of now-Sydney. The landscape is layered with natural textures, with fragments of city buildings and western cartographic features ‘breaking through’ (fig. 1). Placenames emerge from this landscape, using a location marker made familiar through Google maps, yet to many (settler) viewers the placenames may be unfamiliar, and even when they remind us of the familiar, the processes of colonial anglicization becomes apparent. We deliberately chose not to standardize the varying spellings of the placenames in order to highlight the complexity of the linguistic context, the messy and partial colonial processes of recording and renaming place, and the differences from the names that now-Sydney dwellers would be familiar with today. Some places had, and still have, multiple names, which reflects the complexity of the linguistic context in which different cultural knowledge holders from a place gave their names for it. This multiplicity of stories and histories associated with a place is therefore reflected in the language, and cannot be reduced, for example, to a simple machine-based labeling of places from a gazetteer or a single standardized orthography.
Most of the rest of the Barrawao VR experience takes place in small bushland scenes, in which words in either the Dharug, D’harawal or Dhurga Languages rise up out of the land around the player. The player can reach for and touch a word to hear it spoken aloud. Most of the words are nouns: terms for animals, birds, and parts of the landscape (fig. 2). We debated whether to translate the words to English, and if not, how to convey their meanings to the player. The approach we took in the end was to associate each word with audio relating to its referent, so that for example touching the word garraway triggers the call of a sulphur-crested cockatoo. Not all of the words could be uniquely associated with a sound, of course, and we decided that this incompleteness of knowledge was still a better solution than diluting the linguistic landscape with English text. [27]
Ideally, players who did not already know these Languages leave the experience with questions about what they have heard and seen, wanting to learn more. For anyone familiar with language learning software, the absence of translations should in fact be felt as a distinct lack, hopefully prompting a deeper search for meaning in the game beyond the learning of a handful of words. One of the aims of the project is for the unexpectedness of a Dharug, D’harawal and/or Dhurga-only linguistic landscape to have a twofold effect on players: it primes them to seek a reason why the convenience of translation has been eschewed, preparing them to understand the above-mentioned protocol of using the Language of the Country one is on; [28] and at the same time it confronts the player with an awareness of an alternative world that colonization has stolen.
Non-linguistic Elements of Communication and the Embodiment of Language
Beyond the use of First Language elements as content in Barrawao, there are similar questions about how to design a user interface in a culturally and linguistically appropriate way. Despite the potential for an embodied experience that VR affords, it is still a kind of machine mediation and therefore represents a counter disconnect that divorces the user from their physical environment. In the case of Barrawao, this disconnect risks falling back on a mindset of the colonizers by failing to acknowledge the inalienable link between Language and Country. In this case the conventional modes of human interfacing with the machine via a set of predefined linguistic codes need to be rethought to include a wider understanding of relationships between the machine, human, language and environment. As Pacific Studies scholar Katerina Teaiwa has noted, the relationship between language, land and human bodies is central in many Pacific (as also those in now-Australia) Indigenous communities “[…] the concept of te aba, or land, in the Kiribati language spoken by both I-Kiribati and Banabans unites the body of the land with the bodies of the people, as in most Pacific societies.” [29]
If a player is to feel truly present in the virtually represented place, and if language is inherent to that place, then the way the player accesses language must be by moving their body through the world. The response of the virtual world to real-world movement both reinforces the feeling of being physically present and also, more importantly, allows the virtual world, and through that, Country, to communicate with the player. As noted by Bawaka et al, “[…] animals, rocks, winds, tides, emotions, spirits, songs and humans speak. They all have language and knowledge and Law. They all send messages; communicate with each other.” [30] Hromek reminds us that “[w]e are only one part of Country, and being able to read the stories and hear the lessons and requests being sent by others in space is the responsibility of people as custodians.” [31]
Co-creation
In the second and third scene of Barrawao, the viewer is transported to the aforementioned virtual representation of a bushland glade and rocky beach. It is in these scenes that the viewer may reach out with their hands and touch the words in Language rising from the representation of Country they are immersed in. The representation of space is at a one-to-one scale, and while the environment is abstracted it is created directly from photographs of the Country it represents via photogrammetry (fig. 3).
By creating a virtual experience of a place, the creator also builds a relationship to the represented Country. Just as people have responsibility towards their Country in the real world, such creators have responsibilities to their virtual places, in terms of taking care of what they have created. Some such responsibilities are commonly acknowledged, expressed through, for example, the concept of copyright and intellectual property, liability, and when it comes to online virtual spaces, concerns about link rot, web archiving, and data management. But by representing a real place in a virtual world, or an imagined place that has a relationship to a real-world space, the creator also creates a link to that real place, and takes on some responsibility for understanding and transmitting the understanding of that real place, with its associated language and culture. Whether or not they realize it, this forms a connection to the people who have the real-world responsibility of caring for the place the virtual space represents. This understanding of the similarities between responsibilities for the physical world and the digital world is one of the impetuses behind the concept of ‘data sovereignty.’
In order for languages to flourish, Language as part of culture and Country must be cared for like the waters, plant life or indeed narratives are cared for. Those who hold responsibility for Language must pass it along to ensure it prospers for many generations; these days that may mean teaching orally, or recording the language in audio or writing so it can be re-awoken. Additionally, language learners must speak the Language of the land every day in order that the reciprocal communications with Country are maintained. Names of Country that are known must be used and shared as those names originated from that place and are indicators of what healthy Country might be like in that place.
That connection with the Country, its culture, its Language and its people can only be respected by taking the time to deeply know and listen to it. Otherwise the re-making of a place in the virtual world becomes a re-colonizing of it: taking something and making it one’s own and changing it in a violent way without understanding it or having the right relationship to it.
This consideration was foremost in our design process for Barrawao. Over a period of many months, we met repeatedly with each other, and with other Knowledge Holders and Elders from the region. A number of these meetings were on Country, where we sat beside the water, walked among the trees, picked and prepared weaving materials, or visited a culturally significant site. We also went to important sites around now-Sydney to record audio for the project, or to take photographs from which the colour palette of Barrawao was drawn. Following an approach outlined in Hromek’s (Re)Indigenisation of Space, [32] we used culturally appropriate co-design methods such as yarning, making, walking. We talked about the project, but we also told stories from all our perspectives, and talked about what is important about the places as we spent time in them. All this subsequently meant that when the VR experience was created, while it was an imagined space, it was one that was based on a collection of all these experiences, and thus deeply embedded in a real place, with its real community, its living culture, and language as it is being used by people of that place and in that place. A future direction for further research around this project would be a user-experience study, where we seek to understand how the design decisions the team has made come across to the Elders and Knowledge Holders who were not involved in the creation of the VR. This may help us better understand the relationship between the design process discussed here and the outcomes.
Conclusions
Central to this paper has been the concept of Country and its cultural significance to the First Peoples of the continent now known as Australia. Language is tightly bound to Country, to the point where it can be seen to emerge from, and be defined by, Country. As we have highlighted, the western worldview dominating the tech-industry, as exemplified by Silicon Valley, has meant that language is often thought about as separate from culture and that meaning, knowledge and understanding can all be easily translated or localized from one language to the next via a simple one-to-one mapping from English. Barrawao, on the other hand, uses the affordances of VR to place Language back into a spatial context in a co-designed virtual representation of Country.
In his book My People’s Dreaming, Uncle Max Dulumunmun Harrison takes us on a journey through his Country and his culture via embodied storytelling. [33] This is a journey Hromek and Burrell have participated in, in person on a mountain known as Gulaga in Yuin Country, New South Wales. Here an ancient narrative, handed down via embodied/spatial knowledge is encoded in the monolithic tors of the mountain, each stone an element of the narrative; Language and Knowledge embedded in Country and shared and understood via embodied cognition. This represents only a fraction of an Indigenous understanding of embodied cognition [34] that western thought is only recently returning to as both a field in itself, and as it relates to emerging technologies such as virtual reality.
The affordances of VR provide a perfect opportunity to explore the potential for creating an experience of Language embedded in Country experienced in a bodily manner. In this regard cognitive processes of remembering and understanding fragments of Language are ‘offloaded’ into the virtual environment, [35] in a similar manner to which it may be in the physical space of Country. This complex relationship between Language/Country/virtual environment then highlights the extent to which language in this case can exist outside of an anthropocentric domain without this being embedded in machine generation.
In previous serious games created as part of the greater research project, Burrell, Hendery and Thieberger found they were relying too much on their own Western worldview when attempting to automate the process of extracting linguistic audio samples from a large base of linguistic data, making the mistake of separating language from culture in the process. Through the co-design process of Barrawao, the reasons why this approach had not worked in previous projects became obvious. As an example of where machine translation or automated swapping out of words may at first appear simple, yet is ultimately inappropriate, missing cultural and social nuance, was the reacknowledgement of now-Sydney places with pre-invasion place names. Through the process of co-design, we also were able to come to an understanding that in the case of Barrawao, translating words and phrases presented in Language to English was inappropriate in the virtual landscape settings. Through the experience of interacting with Language in a considered and carefully designed representation of local (to now-Sydney) landscape, a player comes to understand the connection between Language and Country in an embodied manner. Whilst this experience may be machine mediated, it still very much instantiates a relationship between player and place, in which language is experienced in a culturally appropriate context. That relationship, along with the responsibilities to Country that it brings with it, is also created between the designers of the virtual world and the Country that virtual world represents. [36]
Notes and References
[1] Layered Horizons was developed as part of an Australian Research Council Grant “Waves of Words” (DP180100893).
[2] Michael Clyne, Australia’s Language Potential (Sydney: UNSW Press, 2005).
[3] See, for example: Aunty Julie Freeman, who is a Gorawarl Jerrawongarla cultural knowledge holder, and Uncle Stan Grant Senior, who is a Wiradjuri Elder, as cited in Danièle Hromek, The (Re)Indigenisation of Space: Weaving narratives of resistance to embed Nura [Country] in design, (PhD thesis, University of Technology Sydney, 2019); Margaret Somerville and Tony Perkins, Singing the coast: Place and identity in Australia (Canberra: Aboriginal Studies Press, 2011). Uncle Tony Perkins is a Gumbaynggirr Elder.
[4] Bawaka Country et al., “Morrku Mangawu––Knowledge on the Land: Mobilising Yolŋu Mathematics from Bawaka, North East Arnhem Land, to Reveal the Situatedness of All Knowledges,” Humanities 5, no. 3 (2016): 61.1–61.13. Bawaka is an Indigenous homeland on the water of Port Bradshaw in Arnhem Land off the Gulf of Carpentaria in the north of Australia. Research is conducted in collaboration with Indigenous and non-Indigenous people, with Bawaka Country as lead author in many of their writings.
[5] Uncle Greg Simms, “Yarn with Uncle Greg Simms,” Gadigal/Darug/Gundungurra/Yuin, on Darug Country, Sydney, interviewed by Danièle Hromek (28 October 2015). Uncle Greg Simms, an Aboriginal Elder living in Western Sydney with ancestral ties to the Gundungurra people of the Blue Mountains, the Gadigal people of the Dharug nation and the Budawang people of the Yuin nation, contributed both Language and traditional Knowledge to the Barrawao project that we describe in this paper.
[6] Hromek, (Re)Indigenisation of Space. Danièle Hromek is a Budawang woman of the Yuin people, a Saltwater woman with coastal relationships.
[7] See also: <barrawao.net>.
[8] <glossopticon.com>.
[9] Nick Thieberger and Lynda Barwick, “Keeping Records of Language Diversity in Melanesia: The Pacific and Regional Archive for Digital Sources in Endangered Cultures (PARADISEC)” in Melanesian Languages on the Edge of Asia: Challenges for the 21st Century (Honolulu: University of Hawaiꞌi Press, 2012): 239–253. See also <www.paradisec.org.au> for the archive itself.
[10] Australian Research Council Grant “Waves of Words” (DP180100893).
[11] Shannon Foster is a D’harawal Saltwater Knowledge Keeper.
[12] Maria M. Brandl and Michael Walsh, “Speakers of Many Tongues: Toward Understanding Multilingualism Among Aboriginal Australians.” International Journal of the Sociology of Language 36 (1982): 71–82.
[13] Hromek, (Re)Indigenisation of Space.
[14] Shannon Foster, Joanne Kinniburgh and Wann Country, “There’s No Place Like (Without) Country,” in Fundamentals of Placemaking, ed. Dominique Hes and Cristina Hernandez-Santin, 63–82 (Palgrave Macmillan: Sydney, 2019).
[15] Hromek, (Re)Indigenisation of Space.
[16] See: Noam Chomsky, New Horizons in the Study of Language and Mind (Cambridge University Press: Cambridge, 2000), 59; Tej. K. Bhatia and William. C. Ritchie, “Introduction,” in The Handbook of Bilingualism and Multilingualism (Wiley: Hoboken, 2012).
[17] Clyne, Australia’s Language Potential.
[18] Michael Kwet, “Digital Colonialism: US Empire and the New Imperialism in the Global South,” Race & Class, 60, no. 4 (2019): 3–26.
[19] See also: Gretchen McCulloch, “Coding Is for Everyone—as Long as You Speak English,” Wired, August 4, 2019, www.wired.com/story/coding-is-for-everyoneas-long-as-you-speak-english.
[20] Abhishek Srivastava, Kalika Bali, Monojit Choudhury, “Understanding Script-Mixing: A Case Study of Hindi-English Bilingual Twitter Users,” nn Proceedings of the 4th Workshop on Computational Approaches to Code Switching, 36–44 (2020).
[21] For an early perspective on this, see e.g. Nancy H. Hornberger, “Multilingual Language Policies and the Continua of Biliteracy: An Ecological Approach,” Language Policy 1, 27–51 (2002).
[22] See: Pieter Beens, “The Dangers of Crowdsourcing Translations,” Veraalt.nu – Translation, Transcreation, Localization and Education, June 15, 2016, www.vertaalt.nu/blog/dangers-crowdsourcing-translations (accessed August 19, 2019); Nikolay Bondarenko, “How Video Game Localization Works and How Much it Costs in 2018,” Medium, September 14, 2018, medium.com/@nikolaybondarenko_41585/how-video-game-localization-works-and-how-much-it-costs-in-2018-664e2748a121 (accessed 19 August 2019).
[23] See, for example, Ignacio Lopez-Moreno et al., “Automatic Language Identification Using Deep Neural Networks,” IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), (April 2014): 5337–5341, and subsequent works.
[24] Daniel Garcia-Romero et al., “Speaker Diarization using Deep Neural Network Embedding,” IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), March (2017): 4930–4934.
[25] Maria Bargh, “Romance and Resistance in the Pacific: Neoliberalism and Indigenous Resistance in the Pacific,” Revue Juridique Polynésienne, 1, no. 118 (2001): 251–274. Maria Bargh’s iwi is Te Arawa (Ngāti Kea/Ngāti Tuarā), Ngāti Awa.
[26] Gretchen McCulloch, Because Internet: Understanding the New Rules of Language. (Riverhead Books: New York, NY, 2019), 270–271.
[27] For a discussion of the term ‘linguistic landscape’, see: Durk Gorter, “Introduction: The Study of the Linguistic Landscape as a New Approach to Multilingualism.” International Journal of Multilingualism, 3, no. 1 (2019): 1–6.
[28] Max. D. Harrison, and Peter McConchie, My People’s Dreaming: An Aboriginal Elder Speaks on Life, Land, Spirit and Forgiveness, (Harper Collins: Sydney, 2013). Uncle Max Dulumunmun Harrison is an Elder and Lawman of the Yuin people from the South Coast of New South Wales.
[29] Katerina Teaiwa, “Choreographing Difference: The (Body) Politics of Banaban Dance,” The Contemporary Pacific 24, no. 1 (2012): 74. Katerina Teaiwa was born and raised in Fiji and is of Banaban, I-Kiribati and African American descent.
[30] Bawaka Country et al., “Working With and Learning From Country: Decentring Human Author-ity,” Cultural Geographies, 22, no. 2 (2015): 270. Burarrwanga, Ganambarr, Ganambarr-Stubbs, Ganambarr, Maymuru are Yolŋu.
[31] Hromek, (Re)Indigenisation of Space.
[32] Ibid.
[33] Harrison & McConchie, My People’s Dreaming.
[34] See: Lynne Kelly. The Memory Code: The Secrets of Stonehenge, Easter Island and Other Ancient Monuments (Pegasus Books: New York, 2017), for an investigation into the embodied and spatial mnemonic traditions of Indigenous people across the world.
[35] This terminology of off-loading was suggested to us by Margaret Wilson, “Six Views of Embodied Cognition.” Psychonomic Bulletin & Review 9, no. 4 (2002): 625–36.
[36] Barrawao can be translated as “to fly or to make haste” from the D’harawal Sydney Language. This paper’s title, Nura Yaman can be translated as “Country Speaks” and is from the Dhurga Language.
Table
| Primary audience | Scope | Interaction | Game type | 3D aesthetic | |
| Glossopticon | General public | The Pacific | Game controller | Open world | Realistic terrain |
| Layered Horizons | Researchers | Oceania | Natural hand gestures picked up by camera | Open world | More abstract datascapes |
| Barrawao | General public | now-Sydney | Natural hand gestures picked up by camera | Guided experience | Mixture of photogrammetry and abstract landscape |
Table 1: comparison of the serious games in our languages series.
Bibliography
Bargh, Maria. “Romance and Resistance in the Pacific: Neoliberalism and Indigenous Resistance in the Pacific.” Revue Juridique Polynésienne, 1, no. 118, 2001: 251–274.
Bawaka Country et al. “Morrku Mangawu – Knowledge on the Land: Mobilising Yolŋu Mathematics from Bawaka, North East Arnhem Land, to Reveal the Situatedness of All Knowledges.” Humanities 5, no. 3, 2016: 61.1–61.13
Bawaka Country, Sarah Wright, Sandie Suchet-Pearson, Kate Lloyd, Laklak Burarrwanga, Ritjilili Ganambarr, Merrkiyawuy Ganambarr-Stubbs, Banbapuy Ganambarr and Djawundil Maymuru. “Working With and Learning From Country: Decentring Human Author-ity.” Cultural Geographies, 22, no. 2, 2015: 270.
Beens, Pieter. “The Dangers of Crowdsourcing Translations.” Veraalt.nu – Translation, Transcreation, Localization and Education, June 15, 2016. www.vertaalt.nu/blog/dangers-crowdsourcing-translations.
Bhatia, Tej. K. and William C. Ritchie. “Introduction.” In The Handbook of Bilingualism and Multilingualism. Hoboken: Wiley, 2012.
Bondarenko, Nikolay. “How Video Game Localization Works and How Much it Costs in 2018.” Medium, September 14, 2018. Accessed August 19, 2019. medium.com/@nikolaybondarenko_41585/how-video-game-localization-works-and-how-much-it-costs-in-2018-664e2748a121.
Brandl, Maria M. and Michael Walsh. “Speakers of Many Tongues: Toward Understanding Multilingualism Among Aboriginal Australians.” International Journal of the Sociology of Language 36, 1982: 71–82.
Chomsky, Noam. New Horizons in the Study of Language and Mind. Cambridge: Cambridge University Press, 2000.
Clyne, Michael. Australia’s Language Potential. Sydney: UNSW Press, 2005.
Foster, Shannon, Joanne Paterson Kinniburgh and Wann Country. “There’s No Place Like (Without) Country.” Fundamentals of Placemaking. Sydney: Palgrave Macmillan, 2019.
Garcia-Romero, Daniel, David Snyder, Gregory Sell, Daniel Povey, and Alan McCree. “Speaker Diarization Using Deep Neural Network Embeddings.” IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), (March 2017): 4930–4934.
Gorter, Durk. “Introduction: The Study of the Linguistic Landscape as a New Approach to Multilingualism.” International Journal of Multilingualism 3, no. 1 (2019): 1–6.
Harrison, Max Dulumunmun and McConchie, Peter. My People’s Dreaming: An Aboriginal Elder Speaks on Life, Land, Spirit and Forgiveness. Sydney: Harper Collins, 2013.
Hornberger, Nancy H. “Multilingual Language Policies and the Continua of Biliteracy: An Ecological Approach.” Language Policy 1 (2002): 27–51.
Hromek, Danièle. The (Re)Indigenisation of Space: Weaving Narratives of Resistance to Embed Nura [Country] in Design. PhD thesis, University of Technology Sydney, 2019.
Kelly, Lynne. The Memory Code: The Secrets of Stonehenge, Easter Island and Other Ancient Monuments. New York: Pegasus Books, 2017.
Kwet, Michael. “Digital Colonialism: US Empire and the New Imperialism in the Global South.” Race & Class 60, no. 4 (2019): 3–26.
Lopez-Moreno, Ignacio, Javier Gonzalez-Dominguez, Oldrich Plchot, David Martinez,
Joaquin Gonzalez-Rodriguez and Pedro Moreno. “Automatic Language Identification Using Deep Neural Networks.” IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), (April 2014): 5337–5341.
McCulloch, Gretchen. “Coding Is for Everyone—as Long as You Speak English.” Wired, August 4, 2019. www.wired.com/story/coding-is-for-everyoneas-long-as-you-speak-english.
McCulloch, Gretchen. Because Internet: Understanding the New Rules of Language. New York: Riverhead Books, 2019.
Simms, Uncle Greg. “Yarn with Uncle Greg Simms.” Gadigal/Darug/Gundungurra/Yuin, on Darug Country. Sydney: interviewed by Danièle Hromek (28 October 2015).
Somerville, Margaret and Tony Perkins. Singing the Coast: Place and Identity in Australia. Canberra: Aboriginal Studies Press, 2011.
Srivastava, Abhishek, Kalika Bali and Monojit Choudhury. “Understanding Script-Mixing: A Case Study of Hindi-English Bilingual Twitter Users.” Proceedings of the 4th Workshop on Computational Approaches to Code Switching (2020): 36–44.
Teaiwa, Katerina. “Choreographing Difference: The (Body) Politics of Banaban Dance.” The Contemporary Pacific 24, no. 1 (2012): 65–95.
Thieberger, Nick and Lynda Barwick. “Keeping Records of Language Diversity in Melanesia: The Pacific and Regional Archive for Digital Sources in Endangered Cultures (PARADISEC).” Melanesian languages on the edge of Asia: Challenges for the 21st Century. Honolulu: University of Hawaiꞌi Press, 2012: 239–253.
Wilson, Margaret. “Six Views of Embodied Cognition.” Psychonomic Bulletin & Review 9, no. 4 (2002): 625–36.