In ‘Sound Stories,’ Christian Marclay makes music out of Snapchats

A new exhibit now on view at the Los Angeles County Museum of Art (LACMA) highlights the sounds we unthinkingly produce when sending Snapchat videos back and forth. Called “Sound Stories,” it uses the sounds of snaps to comment on themes like social media’s effect on self expression, the nature of art, and interconnectedness. 

“It’s a new kind of audio-visual language,” artist Christian Marclay said. “We’re not really conscious of how we’re expressing ourselves, but we are.”

Perhaps the most curious of the five installations is The Organ: a keyboard that visitors can play, that plays a string of snap video snippets in tune with the pressed note. A chord becomes a collection of sound and moving images, produced by strangers the world over the world, all at once.

In 2017, Snapchat approached Marclay about collaborating on a project that used the publicly submitted snaps in Our Stories. Marclay — an established artist with deep roots in the avant-garde — told Mashable he never could never have imagined himself working with a brand. But the sheer amount of material Snapchat offered him access to was too important to pass up.

“As an artist I have a responsibility to react to what people are doing, and to make people aware of the things they’re doing,” Marclay told Mashable. “Social media cannot be ignored, it’s dominated so much. So this was a great opportunity for me to learn about this platform, and react to it.”

A prominent artist since the 1980s, Marclay often utilizes and comments upon sound, video, and communication devices. In the past, he has created new records by splicing together vinyls and playing them as one continuous LP. He is perhaps most famous for his work The Clock, which creates a 24-hour video collage of moments in time in film and television corresponding to the real time on the clock. 

Marclay said that with his interest in making art from what people produce, using Snapchats was a natural continuation of his work — even if the technical elements were new and challenging.

For the project, Snapchat engineers created a database of snap videos that Marclay could search by types of sounds, types of image, and even specific notes. That included building new algorithms to analyze videos and tag the moments that contain, for example, the note “C.” Using this database, Marclay produced five installations in collaboration with Snapchat engineers. (The exhibit was first displayed for five days at the Cannes Lions festival, and will live at LACMA from August 25 to October 19.)

Director of Engineering Andrew Lin explained that Snapchat already had a lot of the categorizing systems in place, thanks to Snap Maps, which analyzes Our Story snaps to populate the map. But categorizing by sound was a whole new challenge. 

Additionally, Lin is used to building tools for millions of users, not one lone artist. “It felt cool, it felt very different for me,” Lin told Mashable. “We were building for someone who has a special talent, and making them bionic.”

Lin felt that the project’s theme of exposing the music in the everyday was particularly suited to Snapchat. “I’ve always thought that Snapchat is a platform where people aren’t trying to show off or brag, they’re just posting like, now what’s happening?” Lin said. “We’re able to showcase that in a way that’s interesting and beautiful.”

Marclay expressed that sound was almost a backdoor way to highlight delight and personality in the totally mundane. 

“Most of it is quite banal and kind of stupid and silly,” Marclay said. “It’s just a little thought, a little thing. I like that lightness.”

The product is a five-room exhibit that visitors navigate through a series of dark hallways. The first room contains a semi-circle of iPhones. Each plays Snapchat video snippets that align with each other by sound and theme: people walking, people cooking, dogs howling, the crack of a crème brûlée. It’s impossible to watch them all at once, but you can certainly hear them all together; perhaps sound connects us more than we think.

The next room features two larger screens that face each other, both playing a series of snaps that seem to be in dialogue. Who are we talking to when we submit social media content publicly, anyway?

The third room contains The Organ. Pressing a note plays sounds and projects the videos making that sound on the far wall of the room. One note constitutes a photo-strip of five micro-videos; it keeps playing and generating new sounds and videos (in “B” or “E” or whatever note you’re pressing), for as long as you hold it down. Pressing multiple notes at once displays multiple photo strips. It is the world’s unintentional chorus.

The fourth room was the most technically challenging, according to Lin. It is a room strung with hanging iPhones that all say either “Talk to Me” or “Sing to Me.” When you do, the phones in your vicinity mimic you with people also saying hello or whistling a tune. 

Finally, the ceiling of the fifth room contains many domes, each of which can be looked into by only one person, standing below it, at a time. At the top of the domes are tablets playing videos of snaps in “turtle mode,” Snapchat’s slo-mo feature. The effect is almost a reverse “down the rabbit hole” feeling, mimicking the tunnel vision that occurs through endless social media scrolling.

An exhibit made of Snapchats sounds gimmicky. But the thoughtfulness and technical chops behind the pieces gave them weight and credibility. It was a thought-provoking way to wrap your head around the million moments of expression that live, usually unseen, on the Snapchat servers.

“There’s such a quantity that I wanted to react to that, the same way I did with vinyl and LPs,” Marclay said. “There is this element of the discarded and the recent past. They are these little mementos.”