Ultrachunk (2018)

Excerpts

Excerpts from performance at ASSEMBLY 2018, Somerset House Studios

ULTRACHUNK is a collaboration between performer Jennifer Walshe, and artist Memo Akten. At once surreal, spellbinding and deeply alarming, ULTRACHUNK is a live improvisational duet between a classically trained musician and her AI doppelganger.

It is one of the first (if not the first) of its kind performance where raw audio (and images) are generated live, in realtime by a generative artificial neural network. Not playing back pre-generated AI audio samples, and not a traditional sampler or synth controlled by a small neural network, but generative models trained on audio-visual data, generating in realtime.

And crucially, it’s interactive, listening and responding to human performers.

This work continues my research into large artificial neural networks as systems for real-time, expressive human-computer interaction, framing them as new kinds of instruments that can emerge through meaningful human control.

It also continues my research into data sovereignty. As models grew larger, they demanded ever more training data, and acquiring such vast datasets raised ethical, legal, and even spiritual questions. For this project, we asked how we might create all of our own training material—and our answer was ritual. Over the course of a year, Irish vocalist and composer Jennifer Walshe engaged in a daily ritual of solo improvisations in front of her webcam, wherever she happened to be in the world. Using the many hours of video and audio, I built and trained a machine-learning driven live performance system comprising multiple custom architectures and models.

During the performance, the video and audio output from the machine are neither recordings nor processed—every frame and sound is synthesized live in real-time, navigating the latent hypersphere, constructed from the fragments of memories in the depths of multiple neural networks. The original and virtual Walshe inhabit the Uncanny Valley together, singing in duet, improvising, listening and responding to each other.

Due to the improvisational (and unpredictable) nature of the performance, the duration varies and is usually 20-40 minutes.

During the performance, I’m also on-stage, off to the side with the computer, eyes on the monitor, twisting dials and faders on a midi-controller. At the time, people asked what I was doing during the performance. But the dials didn’t map onto simple properties like speed or pitch. They were so deep in the system that to explain what they did, I’d have to explain the whole system in great detail.

So I found myself turning to a metaphor: we had spent the last year training a wild beast, and now I’m up there on stage like a lion tamer, trying to make sure it behaves, and doesn’t try to eat the star.

Screen recordings

These are screen recordings of the system improvising in real-time. Both audio and visuals below are entirely generative improvisation sessions, synthesized in realtime.

Images

Credits & Acknowledgements

• Commissioned by the Chase Foundation and Somerset House Studios.
• Created during my PhD at Goldsmiths, University of London, funded by the EPSRC UK.