Week 6: Spatial Sound Puzzle in Unreal

At the end of our last class, TK shared a very clever idea that I decided to run with a bit farther this week: he described attaching stems of Michael Jackson song to pickup cubes in the VR template. Since it’s also super fun to throw items in VR, I turned that idea into a game inside of giant cube pit of sorts.

Imagine entering a VR scene surrounded by walls of identical-looking cubes emanating sounds. Some of the sounds are similar, but not all. There are six special cubes each containing a stem of an oldie but a fun-to-dance-goodie if you can find and assemble them all. Toss the cubes away that don’t sound sense, but take care not to throw away a song stem. You’ll have really use your ears for this one. Here’s a hint: you’re listening for: vocals, drums, bass, keys, guitars, and another cube with some extra fill-in sounds. You’ll know when you find one: just bring it close and listen carefully.

This sketch picks up threads from my previous ones: in this minimal environment, sound is the key component for navigation as well as the primary motivator for engaging in the scene. But here the puzzle is to actively interact with the cacophony until you create a state of musical order.

A few notes on my process. I developed this project entirely with the Oculus Rift and touch controllers. Each pickup cube in the VR template is based off a blueprint. In order to embed a sound within a cube, I created a sound cue and attached that to the cube’s blueprint. Because I had many audio files, I created as many cube blueprints as I had sound cues. Once a cue was attached to a blueprint, I made multiple cubes with that sound. Adjusting any audio setting on the blueprint automatically applied to the related cubes. As a general practice, I checked to override attenuation in both the cue itself and on each cube’s blueprint. Though I set looping in the sound cues, I set the attenuation function in the blueprints to natural sound—I remember Michael (?) saying that with this the loop does not restart if you leave and walk back into the radius of hearing. The radii of the sounds spheres vary, but I made sure to increase those of the stems a bit to overlap on the player when the game starts—turns out that this is a hard puzzle to solve so it’s helpful to give a glimpse of the end goal.

Stems from here.

Additional credits from Freesound.org:
blip-plock-pop by onikage22
Crickets_04.wav by RSilveira_88
Cuckoo[1].mp3 by Navadaux
Ghostly Whispers by dimbark1
Mystery_chime.ogg by Unaxete
Pop!.wav by kwahmah_02
pulse echo_01.L-Joined-0001.aiff by martian