I've been recently working on some sound assets for a game that a friend of mine is making about the management of a vending machine. I have some experience writing music and creating foley sounds for games, but I haven't done a lot of research into the sound design that goes into creating reactionary sounds and in particular, UI sounds. Sounds in video games are multi dimensional; pressing a button in game play doesn't sound that same as just a regular button or switch in real life, and selecting menu items in the UI always have their own particular flair to them too, so I want to get into the nitty gritty of why sound designers choose or create sounds for particular things in games.
Nintendo Sound Design (UI Sounds)
You can't rightfully talk about satisfying UI sounds without talking about the Wii. Nintendo's ability to create compelling audio (as well as visuals) is unparalleled. They invoke a sense of wonder and play while also accurately conveying the appropriate emotion for the action you're performing. Just starting up the Wii presents you with a welcoming echo that leads you into the menu and only becomes more and more welcoming with familiarity. After the initial "entrance" sound there is a repetitive arpeggio that floats comfortably in the lower end of the frequency spectrum, and it really is quite calming. When the user hovers over a button or makes a selection, there are audible confirmations for the action you have completed, not unlike flicking a light switch and hearing a "click". The sounds they have chosen for the Wii UI (especially the sound you hear at about 0:03 of the second video) almost feel like you are physically clicking something perform an action, but they have a kind of electronic sound to them as well, which I think works well for this kind of UI because it is fundamentally simple but has it's own kind of flair. The game that I am going to work on is essentially about managing the comings and goings of a vending machine, so sounds that feel similar to the actions you perform on a vending machine might work for the UI, but perhaps embellishing them with some kind of synthesis would make them unique.
The most interesting incite I could take away from Nintendo sound design is something that Shigeru Miyamoto said to Business Insider in 2016; "The player sees the grass and the rocks, but program-wise it’s obviously not really grass and rocks. But it’s really how we use the sound,” Miyamoto said. “So when you’re in a forest, we try to play sound effects that really remind you of a forest. So if a player goes into that forest, they’re reminded of a forest that they know.”
To me, this means that audio in video games must remind of you something that you are familiar with in real life, in order for you to better familiarise with the context of the game.
“Really, a game is about helping the player remember what they know” (Miyamoto, S. 2016)
My general sort of take away from this is that all the sounds I'm going to create within the UI and the game itself must reflect reality enough to remind the user of something they are already familiar with. For example, a majority of us will know what a keypad should sound like; if you can think of pressing a keypad on a vending machine or maybe even a eftpos terminal, or alarm number pad, we can expect sound feedback that all sounds fairly similar.
Another thing to take away, specifically from the Wii Ui sounds is that they all appropriately reflect the literal aspect as well as the intention of the action you're performing. This is important for things like accepting or cancelling an action, or on a simpler level, unlocking and locking something. To put this into context, imagine the sound your phone makes when you lock it and when you unlock it, it's reassuring in the sense that it lets you know that your phone is safe and represents that by playing a sound that you can easily familiarise with the sound of something locking. Similarly, when you unlock your phone it plays a sound that suggests the opposite action. Something like this is most common with phones like iPhones that have a very realistic sound for locking;
Obviously there are a lot of sounds in this video but just focus on the lock sound for now.
Now I don't own an iPhone, but I never hear this audio feedback on my phone anyway because I always have it on silent. But after looking into this I turned my phone off silent and checked out what the locking and unlocking sound was;
(Sorry about the low volume)
Rather than going for a literal sound, my phone unlocks with a tonal cue. Unlocking goes from a low beep to a higher beep, and locking is the opposite. Because it's something that I as the user am choosing to do, that has no real positive or negative outcome and so the corresponding sound should reflect that. It simply indicates that you are going one way with the upwards sound, which feels like a beginning or an opening, and then a downwards sound which represents the end of something. To put this into visual context we can consider a plane taking off and landing again; it's starts it's journey by going upwards, and then completes it by coming back down again.
At the 2017 TNW Conference, Henry Daw explains during a presentation that it is important to have grouped together sounds for particular actions and in some cases, entire user interfaces. He uses examples like the listen/confirmation sounds for siri, connecting and disconnecting or locking and unlocking. He also discusses the use of branding in sounds by keeping a similar principle, tonality, or textural content between all sounds used in a particular medium. He also makes a fantastic point about keeping things simple; two of his many principles when designing sounds for UI are to not over design sounds, and to design silence. Not over designing sounds refers to keeping things as minimal and uncomplicated as possible and always referring back to the user perspective as well as the function your are designing for. According to this principle, the sounds present in a UI should be simple and to the point in order to effectively communicate feedback to the user. His principle of designing silence is also interesting as he makes a fantastic point about adding sound for the sake of adding sound.
"If a sound isn't working either through your instincts, the UX designer instincts, or through user testing, then simply remove that sound. Silence is golden" - Henry DAW, 2017
I have a lot to consider now while creating and recording sounds for my project, especially when it comes to the user interactions sounds within the UI and the actual game. The game itself revolves around a vending machine, and so it is crucial to create sounds that will remind a user of operating a vending machine. As for the UI sounds, I feel like the principle of grouping together sounds should play a large role in the sounds that I create. Most of the music has a similar feel and instrumentation to it and I think it would be nice to incorporate some of that into the UI. This will hopefully have the UI sounds feeling like they belong in the game and fit nicely into the scenario you are experiencing them.
The few things that I want to keep in mind while I create all these sounds are;
- Sounds in games should remind the user of things they already know
- UI sounds and interactive sounds should above all represent the intent of an action
- Sounds should verify and reassure the users action has taken place
- Don't over complicate sounds and remove sounds if they aren't needed
BIB
Perry, A. (2016). The creator of 'Super Mario' explains what makes Nintendo games so good.
Retrieved from; https://www.businessinsider.com.au/this-is-what-makes-nintendo-games-great-2016-7?r=US&IR=T
Daw, H, & Anderson, A. (2017). UI Sound Design: Henry Daw, on The Small Sounds That Make A Big Difference | A Sound Effect.
Retrieved from; https://www.asoundeffect.com/ui-sound-design-henry-daw-small-sounds-make-big-difference/