playing Music with Cardboard squares
Why is cranking an aluminum volume knob so much more satisfying than swiping a volume bar on your phone?
Because touch matters.
I wanted to use AR to make the act of playing music more interactive. Playing music used to be a physical experience—placing a vinyl record on a turntable or loading a cassette into a boombox. But with Spotify at our fingertips, listening to music has ceased to be a kinesthetic experience for most of us.
I think physical & spatial interaction are really important, so I wanted to make an augmented reality experience with tangible controllers.
Using Unity and Vuforia, I wrote a program that would recognize image patterns and overlay them with albums and artists. Once I did that, I put the image patterns on cardboard pieces that could be moved and manipulated.
'objectifying' music
How do we browse and discover music? I took inspiration from a familiar (albeit dated) interaction: flipping through records. (Ben Frankforter highlights the hidden values of this simple interaction in a great piece on designing for MR). Flip-browsing has the added benefit of being completely unique to the music world, and so was a great move to leverage here.
making the tray
The first thing I had to do was make some album cards and a tray to hold them. I printed out unique visual patterns that would be recognized as 'target markers,' and hot glued a piece of cardboard into the shape of a tray to flip through my albums.
After playing with the user flow a bit, I ended up swapping things around so that the tray holds ARTISTS and the buttons display their respective ALBUMS.
Making the buttons
I wanted to add depth and dimension to the buttons, so it actually feels like you're pressing something down.
testing it out
The key is: all of these are dumb objects, not electronic/connected devices. They are just cheap pieces of cardboard, and 1) all the processing is being done via the smartphone: the pattern identification, the album art projection, the button press recognition, the music playing, etc. 2) By overloading the visual information that can be projected onto them, we are able to get a satisfying tactile experience with just a few objects, don’t need a whole huge sound board. AR allows us to overload them.