Journal

Thoughts/ Research
Categories
Collaboration (VR)

What Remains Of Edith Finch

‘What Remains of Edith Finch’ being the main reference given to us by the Games students for their Phone-booth VR game, I decided to delve further into the methodologies behind it.

It is essentially a narrative driven, first person game that uses storytelling, gameplay mechanics, music and a variety of visuals and movement to create a collection of short stories that are meant to shed light on the odd events that cause the members of the finch family their tragic demises. When beginning to watch a walkthrough video of the game I instantly noticed the uncluttered nature of the opening environment. Set in the woods, in which the main character is walking through, the sonic feel is uncluttered allowing us to enjoy the atmosphere of the woods whilst also registering every sound made through interaction, as well as the narrators overarching voice.

As the designated foley artist, I tried my best to pay special attention to the foley sounds throughout the game. The first thing I noticed however, which is also one of the first sounds we hear in the game, was the footsteps and how they seemed a little too robotic and without appropriate variation to recreate the feel of reality. Once the character reaches the house, once more I noticed that the sounds of the footsteps were inaccurate based on the amount of steps being taken, and did not sound appropriate the surfaces that were being tread on. Learning from this, I will make sure to pay special attention to variation and recreation when re-recording the foley for our group’s game.
On the other hand I noticed that in other parts of the game, footsteps had been omitted altogether, which I thought was done tastefully as it gave space for the music and narration and other sounds to take precedence. Being in a world that has been created for the purpose of narration means that the creator has the choice of what is heard, and in a way this omittal helped add to the suspension of fantasy one would find playing this game.

Considering the narrative nature of What Remains of Edith Finch, and it being the main reference, it made me see the Phone-booth VR in a different light; less about the completion of tasks, and more about the overall story or narrative. Using this newfound look on our work I wondered how else What Remains of Edith Finch could inform our game.
A couple things I noted was that the music at many points was intertwined with the soundscape, helping to underscore th narration as well as direct the emotional undercurrents. As well as this the music ever became an interactive experience of the game during parts of it; for example when winding the music box in the hallway to play the theme song. Lastly, descriptive sounds accompanied the narration at points, even being cued by it, bringing sounds in dependent on what the narrator was saying. This perhaps might be a handy tool to help concretise the spoken narrative that takes place in the phone-box in our game.

Categories
Collaboration (VR)

Mixing In VR

After our crits I decided it would be beneficial to do more research into mixing for virtual reality in order to bring an increased sense of sonic realism to the game.

While in two dimensional media such as movies, mixing involves adjusting volume levels, appropriate panning and reverbs, however for more interactive experiences like that of VR, these qualities of the mix rely on distance and direction. In softwares such as Fmod and Unity, as opposed to Ableton (which is what I am most used to using) we are able to automate and attenuate these qualities based on distance. For VR specifically, panning is replaced with HRTF, which stands for Head-Related-Transfer-Functions, which supposedly provides more accurate directional cues than panning.

To expand on this HRTF is seemingly a more advanced way of localising audio as it is ‘measured in a specialized facility under controlled conditions. During such a measurement, a sound is played through a loudspeaker from a certain direction and microphones placed at the entrance of your ear canal register how your ears ‘hear’ this sound’.

Moreover, while non-VR games can be played on any form of speaker, however this means somewhat compromising the quality of sound in order to ensure the final mix sounds good on all kinds of sound systems. VR headsets however are all equipped and played with headphones, allowing for a more consistent audio reproduction. However taking this into account, when mixing for VR, one has to be aware of user fatigue, requiring volume levels that can be listened to for sustained amounts of time as well as not cluttering the mix with too many sounds in the same frequency ranges.

Distance is also crucial to successful immersion within a game. For example when considering audio cues, if a sound is loud despite it being perceived as far away then it will most likely throw off the player and confuse them, breaking down the illusion of immersivity. The rule of thumb for physically accurate distance attenuation is apparently that ‘the doubling of distance is a halving of intensity’. Reverb is another important tool that, if used effectively, can create the illusion of distance also. Sounds that are far away we perceive to have more reverb.

All of the research into these techniques are sure to help me when coming together with the rest of my group for the final mix down and has only reinforced the importance of controlling and treating the relative qualities of each sound as a critical component in the mix.

Categories
Collaboration (VR)

Phonebooth – Designation Of Roles, Individual Contribution & First Crits

After creating an excel file and organising all the different components needed for our game with my group members, we discussed what roles should be designated to whom. Being given the task of remaining in charge of foley and atmospheres I decided to take my handheld recorder to a phone box on a busy road near mine to recreate the events in the game as accurately as possible.

Before this however, I made sure to meet up with the games students one more time, in order to play the game again, refreshing the mechanics and the general feel of the game in my memory.

After getting them to send me a video file of the game I overlaid it with the atmospheres and foley we had recorded prior to this. This experience provided to be quite rewarding in how the sonics came together. While there was some miscommunication on the musical aspect of things, I decided to make a simple composition using a Tibetan bowl, manipulated in Ableton to create an eeriness to the parts of the game that required it. When doing this however, it was with the intention of being temporary, eventually being replaced by a more collaborative effort for the eventual hand-in.

When evaluating our progress after showcasing our work to the rest of our class during the crits, it became apparent that we needed to focus more time on the immersive nature of the game, particularly so as it is virtual reality.

Categories
Collaboration (VR)

Recording Foley for Phonebooth VR Game

In the very early stages of designating roles to one another, we decided to visit the foley room as a group in order to take the first draft recordings for the game. We felt that by coming together to complete to the first task, we would gain more clarity on the other dimensions of the game, making it easier to designate roles within the group. In this session we recorded footsteps, mechanical sounds of the phone-box and the sounds of doors shutting using a Sennheiser 416 shotgun microphone. Whilst these sounds may not be used in the final product, if at all, we decided to get the ball rolling. This way we can trial and error sounds with the Games students and rerecord sounds accordingly.

As we are expecting to eventually put the sounds into Unity when mixing down the final product, we decided to record one shots of each sound required as well, in order to keep in line with the nature of working in Unity or FMod.

At Hwyels disposal, we used his vintage telephone mic to sample some recordings of the dialogue. Using this beat-down microphone, we thought, would give the audio an interesting nature. Unfortunately however, the audio was too distorted to be used due to a lack of a powerful enough pre-amp. Nevertheless the distortion gave us ideas on how to possible manipulate the dialogue using audio effects in the final mixdown.

An issue we ran into was the multichannel nature of the recordings as we incorrectly formatted the Zoom F4 that we were using whilst recording. However this was swiftly fixed by Inaki, extracting the separate audio channels on pro tools.

Categories
Collaboration (VR)

Collaborative Project – Phonebooth (VR)

‘A VR Narrative game, addressing the effects of depression and the process of reaching out to people around us. The player will call-up friends and family and make dialogue choices over the duration of their conversations. The consequences of your choices will visibly impact the environment around the player’.

After meeting with the Game students, myself and my classmates, Hywel and Inaki, had a better idea of what they wanted the sonically aesthetic nature of their game to be. Contrasts between a feeling of clarity and tension we discussed could perhaps be portrayed through music and atmospheres. Dialogue could also be warped to display the contrast of emotion too. Simply put, they required atmospheres, foley and dialogue. Music it seems has been composed already, by someone they know, however we may debate this with them in order to add our own compositions that we feel would benefit the game more than the simple single piano melody they had shown us during the meeting.

On an overall basis, the nature of the game seemed to us very sonically barren. The beginning and end of the game takes place in an empty black cavernous space, while the climax takes place in one spot, that being the phonebox. As a result we weren’t able to record much in the way of foley as it seemed like at this point there was not much of it needed. However the space left for a sonic environment meant that immersion through atmospheres and music may have to be the main focus of the game

Categories
Global Sonic Cultures

How Interacting Is Different To Listening: Reflection

In the introduction to Collins’ book ‘Playing With Sound: A Theory of Interacting with Sound and Music in Video Games’ she begins by outlining how passive listening has made up the majority of our perceptions of sound in media. As a result we have a limited scope of terminologies and methodologies with which we can approach the player’s relationship to sound in video games. Drawing on the importance of keeping the ‘player’ in mind when approaching sound in media, she goes on to explore multiple different theories on listening, but also interactivity as a whole in order to help us better understand what interaction entails so that she can seemingly use this a foundational basis to apply theories of sound within interactivity in later chapters.

Using Chion’s categorisation of the three basic listening modes:

Causal Listening: refers to the act of associating a sound to its producing action, whether it be conscious or not

Semantic listening: refers to the act of deciphering or interpreting messages in sounds that are bound by semantics , applied for example in a linguistic sense

Reduced listening: refers to the act of listening the traits of a sound, such as its tone or timbre (its acoustic properties)

She explains how none of these modes of listening are mutually exclusive of one another as the player may be listening in several different ways at once. Nevertheless she explains how these modes of listening, as individual approaches, can change the way a player experiences a game, using the sound of a signal beep in the game Fallout 3 as an example. Using these three modes of listening we can determine where the sound is coming from and what has produced it, what the signal is perhaps trying to tell us and also where on the frequency spectrum the sound lies.

What seemed even more so relevant in regards to interactivity was her expansion on these listening modes, using the musicologists, David Huron’s listening modes that he intended to apply for music. The first being Signal Listening, refers to hearing a sound in anticipation, implying a subsequent action. Using ‘New Super Mario Bros’ she touches on how players must listen to music time their attacks, demonstrating the presence of signal listening and an interaction between player and sound. This mode of listening can also help players determine navigational information, status information and semiotic information.
One of the other of Huron’s modes of listening that I found to be of particular importance was retentive listening, in which we try to remember what we have heard with the intention of repeating it. In a gaming context, an example of this would be when a player is required to actively remember a sequence of sounds in order to carry out a certain task. Collin’s refashioning of Huron’s listening modes in the context of video games helps us understand the intricacies of sound in interactivity on a deeper level.

In regards to interactivity as a concept in itself, she touches on how cognitive/ psychological reactions ‘always occur alongside other interactions in games’, putting it at the centre of all other forms of interaction, be it physical, perceptual, socio-cultural or interpersonal. She states that there is a danger in interpreting the word ‘interaction’ too literally, by equating it to a physical interaction between a user and a media object. Moving on from this she mentions that experimental games such as ‘Alpha World Of Warcraft, have demonstrated that players can use their alpha brain waves to change gameplay, thus further confusing any differences between the physical and the psychological’.
When thinking on how this progression of technology might apply to sound, it becomes a fascinating to consider, especially so when relating it to Collin’s ideas on evoking sounds vs creating sounds. If we are able to create sounds in a game using brain waves, what would the limits be on what could be created, and how would they be implemented? If the limits allowed the player to create unique sounds then would this then place the player as co-creator of aspects within the game? In correspondence with my thoughts, Collin’s too touches on the axis of creator and audience as interactivity increases as a result of technological feats.

She concludes the introduction to her book by talking about a term called ’embodied cognition’. This term theorises that ‘our cognitive processes use reactivations of sensory and motor states from our past experience.’ In order to understand further, a google search of this term gave the definition: ‘Embodied cognition is an approach to cognition that has roots in motor behavior. This approach emphasises that cognition typically involves acting with a physical body on an environment in which that body is immersed’. From this we can infer that sound can be explored in many ways through the medium of mentally reenacting our physical embodied knowledge .

Categories
Global Sonic Cultures

Thoughts On Practical Component

“In ambient music, Eno often imitated or borrowed sounds from existing locations, and organised them in compositions to produce new environments. He often used tape loops of differing lengths played simultaneously so that their interaction randomly produced ‘sound events in periodic clusters’, in much the same way the sounds of frogs, insects and birds in a natural environment occasionally seem to express chords and melodies.” – Oblique Music – pg 90

“Eric Tamm has suggested that texture and timber may be off the essence in the ambient style, but a few general remarks may clarify the style’s use of rhythm and harmony’. He examined thirty ambient pieces, and found that eleven of these dispensed with pulse altogether, the rhythm consisting of a gentle ebb and flow of instrumental colours.”

Harmonically, Eno’s ambient pieces often use static or ambiguous harmonies, sometimes suggestive of chords but just as often consisting of nothing but a drone with… pitches drawn from a diatonic pitch set appearing and disappearing. — Two textural principles – layering and TIMBRAL HOMOGENEITY. These principles were combined so that a typical ambient piece by Eno was composed of 3 to 7 distinct timbral layers.

Categories
Global Sonic Cultures

Ambience & Spirituality

The world of music composition, especially in the USA in the 1960’s has deep connections with spirituality, many composers were influenced and inspired by music and spirituality of the East (primarily India, China, Japan). The use of drones and extended durations is perhaps the most obvious result of this interaction – “The Theatre of Eternal Music delved fully into the acoustical universe of single sustained tones, compounding their deeply droning sound with extended duration, bringing each performer into a unified state” (LaBelle, 2006, p.71), also, about Young – “His music, in a sense, strives for the actualisation of the very perceptual tones, loud volumes, extended durations, harmonic frequencies, all encompass and overarching sonic commitment that seeks to make sound an experiential event beyond the human limits of time and space, exploiting the ear as a physiological device and the mind in its moment of perception of sound stimuli.”, and “Duration for Young is not a question of minutes and hours, but days and years. As Philip Glass proposes – “This music is not characterised by argument and development. It has disposed of traditional concepts that were closely linked to real time, clock-time…” (p. 73)

https://www.rastoropov.co.uk/arts/sound-art/

Tom Murphy: When you were studying Eastern mysticism did you find any connections between what you learned that route and the music around you at the time? How would you describe those connections?

Laraaji: I observed that drone music at that time reflected the sensation of eternal present time which is emphasized in eastern philosophy—the continuum of consciousness. Also deep yogic level relaxation and meditation as reflected in the music of Stephen Halpern. The heightened sensation of bliss and ecstasy as reflected in the music of Iasos at the time in the late 1970’s. Terry Reilly.

How did you turn a zither into an electronic instrument? Was anyone doing anything comparable at the time you started doing that? Did you process those sounds early on or was it more for amplification?

My first autoharp/zither was acoustic. And after exploring alternative tunings I investigated ways to amplify it. [I then purchased] an electric pickup made especially for autoharps. I dove into amplified autoharp/zither research and decided to add sound treatment with the MXR 90 Phase shifter. After recording the album Day of Radiance with producer Brian Eno my interest in other [effects] pedals expanded to include chorus, delays, flangers and reverb.

How did you meet Brian Eno and as a producer how involved was in shaping the sound of Day of Radiance?

Brian introduced himself to me while I was playing Washington Square Park [in New York City in] 1978 and extended the invite to join him in his Ambient album productions. His suggestions to depend more on live studio microphones and Eventide effects, mixing as well as overdubbing a second zither helped to shape the Day Of Radiance sound.

https://queencitysoundsandart.wordpress.com/2019/07/12/ambient-music-pioneer-laraaji-on-sound-and-spiritual-practice-vision-songs-and-laughter-meditation/

There is something primevally grounding and simultaneously mystical about the penetrating 

hum of a drone – whether it be Tibetan deep chant, Japanese gagaku, Scottish pibroch

piping, Aboriginal didgeridoo, or Hindustani classical music. [1]  A lot of this music has spiritual 

connotations and uses.  The Classical Indian tradition and Eastern spiritual philosophy 

and music had a steering influence over a group of European and American composers 

that emerged from the 1900s who were labelled as modernist, avante garde, atonal, 

serialist, dissonant, and minimalist.

Deeply concerned with the implications of the advancing technological world and affected by 

the impact of World War and the great Depression, they began to ask questions about 

music; its nature, structure and purpose.  These artists particularly set out to shake the 

foundations of formal musical structure.  Their music was mostly dissonant, chaotic, and

deconstructed.  Its purpose for them was much less about entertainment and more about consciously

finding something which was profound and purposeful.

This paper aims to explore the use of drones and dissonance in relation to a small selection 

of these composers; Dane Rudhyar, John Cage, Karlheinz Stockhausen, Ruth Crawford, 

Arvo Part, and David Hykes.  It also aims to look at their interest in Eastern philosophy and 

to enquire into the nature of drones and dissonance to see whether they might have some 

kind of ability to induce a profound or spiritual experience. It poses to raise the question of 

what makes music spiritual and to look at whether dissonant drones have a particular quality 

about them that can induce a spiritual experience.

https://www.soundtravels.co.uk/a-Dissonance__Drones__A_Spiritual_Experience-316.aspx

https://www.abc.net.au/radionational/programs/earshot/monotony-and-the-sacred/6448906

https://en.wikipedia.org/wiki/Drone_(music)

https://www.ableton.com/en/blog/drone-lab-creating-sustained-sounds-in-live-11/

https://www.screensoundjournal.org/issues/n1/06.%20SSJ%20n1%20Hayward.pdf

https://www.taylorfrancis.com/chapters/edit/10.4324/9781315249131-27/spirituality-mental-health-integrative-dimension

https://search.informit.org/doi/pdf/10.3316/informit.336502644557699

Categories
Specialising & Exhibiting Unit 01

Recording Foley For ‘We Need To Talk About Kevin’

Using the knowledge gained from the Pro Tools Linkedin short course, my classmate and I booked out the Composition Studio and Foley Room in order to record the Foley to the opening scene of Lynne Ramsay’s film ‘We Need To Talk About Kevin’.

After configuring the gain structure on both pro tools and the external preamp found in the composition lab I set about creating a new session in Pro-tools. I had in fact already prepared a session in the previous lecture, with the clip loaded, identifying where all the most important cuts were for atmospheres, FX and syncing of image to sound. When setting up the initial blank session, I set it to the audio file standard for working with video: 48khz, BWF (.wav), 16bit. For the i/o settings I chose stereo mix and then named and saved the new session. From there I created as many new mono or stereo audio tracks that I felt I needed, taking into account that we were overlaying and recording atmospheres and foley.

Using a Sennheiser 416 Shotgun mic, we recorded footsteps, cloth sounds, glass clinking as well as other sounds, syncing them up to the image. Given the dreamlike quality of the opening scene, instead of trying to recreate all of the sounds in the scene, which would have been a hefty task considering the sonic content of the scene, we opted to create various drones, time stretching and manipulating them using open source software such as Cecilia and Paulstretch, before overlaying them in order to create a ghostly effect. Unfortunately, due to gain staging issues, much of the initial foley was recorded too quiet, giving a very high noise floor when mixed appropriately. While this means we may have to re-record much of the foley, it is valuable lesson in avoiding the same mistake in future.

Categories
Specialising & Exhibiting Unit 01

Pro-Tools Linked-in Learning

Having gone through the Pro-Tools Linked in Learning I am now more familiarised with working and mixing for film within Pro-tools. Some useful tips and techniques I have come away with include, but are not limited to:

  • Changing the editing mode in the top right of the Pro-tools browser to Grid will allow me to keep my cursor accurate to the frame boundaries – Then we can change the grid value to reference frames instead of seconds – this way we can also make the background grid accurate to the frame and not the seconds.
  • Going to slip mode will allow me to go to a finer resolution when needed
  • It is useful and time-saving to separate the different elements of a film sound and route them to different outputs for ease of use in a later stage of work – these elements can refer to dialogue, music and sound effects
  • We can achieve this by sending each elemental group of tracks into an auxiliary track which acts as a bus. – essentially an auxiliary track acts as a pathway to route audio from one place to another.
  • This is done by redirecting the output of all the different tracks in a certain element/ group into the auxiliary track and changing the input of the auxiliary track to bus 1.
  • Its a good idea to colour code stems to keep track
  • Using these techniques we can make and save a working template which can act as a starting template for all my projects.
  • Using timecode and markers, that are labelled clearly, are also useful to set up before a recording session in order to identify the key scenes/ cuts for fx and atmosphere as well as sync points for image and sound.