In the introduction to Collins’ book ‘Playing With Sound: A Theory of Interacting with Sound and Music in Video Games’ she begins by outlining how passive listening has made up the majority of our perceptions of sound in media. As a result we have a limited scope of terminologies and methodologies with which we can approach the player’s relationship to sound in video games. Drawing on the importance of keeping the ‘player’ in mind when approaching sound in media, she goes on to explore multiple different theories on listening, but also interactivity as a whole in order to help us better understand what interaction entails so that she can seemingly use this a foundational basis to apply theories of sound within interactivity in later chapters.
Using Chion’s categorisation of the three basic listening modes:
Causal Listening: refers to the act of associating a sound to its producing action, whether it be conscious or not
Semantic listening: refers to the act of deciphering or interpreting messages in sounds that are bound by semantics , applied for example in a linguistic sense
Reduced listening: refers to the act of listening the traits of a sound, such as its tone or timbre (its acoustic properties)
She explains how none of these modes of listening are mutually exclusive of one another as the player may be listening in several different ways at once. Nevertheless she explains how these modes of listening, as individual approaches, can change the way a player experiences a game, using the sound of a signal beep in the game Fallout 3 as an example. Using these three modes of listening we can determine where the sound is coming from and what has produced it, what the signal is perhaps trying to tell us and also where on the frequency spectrum the sound lies.
What seemed even more so relevant in regards to interactivity was her expansion on these listening modes, using the musicologists, David Huron’s listening modes that he intended to apply for music. The first being Signal Listening, refers to hearing a sound in anticipation, implying a subsequent action. Using ‘New Super Mario Bros’ she touches on how players must listen to music time their attacks, demonstrating the presence of signal listening and an interaction between player and sound. This mode of listening can also help players determine navigational information, status information and semiotic information.
One of the other of Huron’s modes of listening that I found to be of particular importance was retentive listening, in which we try to remember what we have heard with the intention of repeating it. In a gaming context, an example of this would be when a player is required to actively remember a sequence of sounds in order to carry out a certain task. Collin’s refashioning of Huron’s listening modes in the context of video games helps us understand the intricacies of sound in interactivity on a deeper level.
In regards to interactivity as a concept in itself, she touches on how cognitive/ psychological reactions ‘always occur alongside other interactions in games’, putting it at the centre of all other forms of interaction, be it physical, perceptual, socio-cultural or interpersonal. She states that there is a danger in interpreting the word ‘interaction’ too literally, by equating it to a physical interaction between a user and a media object. Moving on from this she mentions that experimental games such as ‘Alpha World Of Warcraft, have demonstrated that players can use their alpha brain waves to change gameplay, thus further confusing any differences between the physical and the psychological’.
When thinking on how this progression of technology might apply to sound, it becomes a fascinating to consider, especially so when relating it to Collin’s ideas on evoking sounds vs creating sounds. If we are able to create sounds in a game using brain waves, what would the limits be on what could be created, and how would they be implemented? If the limits allowed the player to create unique sounds then would this then place the player as co-creator of aspects within the game? In correspondence with my thoughts, Collin’s too touches on the axis of creator and audience as interactivity increases as a result of technological feats.
She concludes the introduction to her book by talking about a term called ’embodied cognition’. This term theorises that ‘our cognitive processes use reactivations of sensory and motor states from our past experience.’ In order to understand further, a google search of this term gave the definition: ‘Embodied cognition is an approach to cognition that has roots in motor behavior. This approach emphasises that cognition typically involves acting with a physical body on an environment in which that body is immersed’. From this we can infer that sound can be explored in many ways through the medium of mentally reenacting our physical embodied knowledge .
