Music is a rich expression, digitally consolidated into a waveform, album art, and text data. What if we can build on top of that experience to bring out more of it's intended qualities?

When we started the prototyping phase of XR Bootcamp in fall of last year, I was deep into crate digging for a few upcoming sets (including an amazing b2b @ elsewhere Zone One last year with otugi, a phenomenal music producer and cinematographer). Commuting around the city was fitting time to have a healthy stream of audio, which also gave some life to idle moments sitting on the bus or walking down a few blocks. At the time, I was also seeing articles about the upcoming Vision Pro release, and something about Ray Ban AR glasses, and I thought -- "what would the listening experience look like in a world of wearables and mixed reality?".

Music listening is a focused activity, but not like movies or games that requires direct attention. With the listening experience having both of those passive and active qualities, what would an interface for an XR music experience look like? And, since we're building on top of an existing art medium, how can we amplify the original intent behind the music?

I thought a music player in augmented reality would enrich the intended experience of the work, but still let the listener focus on their direct surroundings; a heads-up-display (HUD) in a first-person experience that responded to events happening. The imagined concept blended the understated, dreamy and cooly-optimistic tones of Gen-X soft club with a suggestive, tech-addicted "cybernetic" feel -- your reality was slowly being taken over by the music... Audio hijack.

On top of work, I had around a few hours to a day to make the prototype concept. Using Unity and a Quest 2, I wanted my basic prototype to:

• Work with the Passthrough API
• Show song and artist data
• View a 3D visual or artwork related to the song, similar to Spotify Canvas

I didn't get to fully flesh out the player prototype, and only built the core "experience" of listening to music in XR. Some of my stretch goals included audio-reactivity (your perceived environment changing with the sound) integration with Spotify API for actual data, and hand tracking with the Oculus SDK for added interactivity (e.g. pinch-out to view more info on the artist). And if this were to be a real thing, an actual music player UX with previous and skip. Below is the result of the experiment: