EMERGENT SYSTEM is a body of work including 60 drawings on paper, several videos, and an 11 minute experimental animated film. It explores the human desire to rebel against systems, patterns and rules.

The works include choreography by Faye Driscoll ("Fascinating... utterly original" —NY Times).

The film is synchronized to music by composer Missy Mazzoli ("Once-in-a-generation magician of the orchestra" —New Yorker).

The film’s finale was generated by an artificial intelligence algorithm trained only on images from the film itself.

This project was commissioned by PEAK Performances at Montclair State University. The film debuted on February 14, 2020, shortly before the COVID shutdown and shortly before AI entered our lives, with music performed live on six grand pianos.

In conjunction, an exhibition of drawings from the work was presented at the Kasser Theater in collaboration with the George Segal Gallery.

The film can also be presented with a recorded track.

Below you'll find—
Drawings
Film
AI
Themes
Context in my practice
Exhibition
Book
Credits

Drawings

For each shot in the film, all the animation "frames" are drawn on a single piece of paper, in a diptych, or, in one case, a triptych.

There are 60 works on paper in total, containing 1001 frames.

These works are, in part, a consideration of a moment in time — or a shot in a film — as an object with form.

They enable us to look closely at the subtle changes that occur in bodies in motion, revealing meaning and beauty. Like musical scores, they have a defined tempo, but the viewer can experience them at their own pace.

I worked with Faye and four dancers to devise the action: a series of relationships between pairs of people, and bodies struggling against abstract objects. The story I envisioned, Missy’s music, and Faye’s practice served as starting points.

Film

The experimental animated film Emergent System (2020, 11 minutes) is about systems and rules—and our human desire to push against them.

Both the music and the animation feature rules and rulebreaking. The pianos come in and out sync through a variety of different techniques, including musical notation, written instructions in the score, and imperfect conducting. In the animation, visual patterns are established in a variety of ways, and then broken.

I brought my outline for the film into the dance studio with Faye and the performers where we created choreography, which I drew.

I constructed the animation from these drawings, adding graphic shapes and colors.

The film dwells in the space between the narrative and non-narrative filmmaking. It begins abstract, and a sense of narrative grows as the piece progresses.

The film is divided into three movements.

I
The first movement begins with patterns of shape, color and rhythm moving in time to the music.

The visual patterns are inspired by the American quilt tradition and the punch-cards used in Jacquard looms and early computers—historical modes of using code for creative work, tied deeply to female creators.

In each scene a set of rules is established, and a form pushes against that system.

The “rules” are created by computer code that I wrote. The code generates animation from a MIDI data representation of Missy’s musical composition. These scenarios point towards early 20th century "visual music" works by artists like Mary Ellen Bute and Oskar Fischinger.

In each scene, I then break the rules by selecting “hero” shapes from the patterns, disconnecting only their code, and animating these hero shapes by hand.

By the end of the first movement bodies begin to appear, most notably Missy’s hand.

II
The second movement is a series of relationships between bodies, including falling, loving, and fighting. This is juxtaposed against a series of musical relationships between the pianos. The pianos and the people are trying to get in sync (or perhaps pull apart).

III
A narrative begins to emerge. A woman awakens in an abstract geometric world constructed from elements introduced in the previous movements. She attempts to affect the world—to make it better.

The third movement is a parable about "The Singularity", a notion popularized by Ray Kurzweil—the moment when artificial intelligence will surpass human intelligence, beyond which the nature of human civilization is difficult to fathom. In this film, the world beyond the Singularity is made from pieces of our world, remixed by machines. To make that future world better, we must repair our own world now, before our time runs out. Our world is the stuff the future will be made from.

AI

The film’s finale, lasting twenty seconds, was generated by an AI algorithm trained on data consisting only of frames from the rest of this film.

The AI rendered animations that it “believed” would look like my animation. The finale is twenty seconds long.

I worked closely with Professor Andrea Danyluk and Chan Woo Kim at Williams College to create this Generative Adversarial Network (GAN) model.

This AI is primitive compared to contemporary models, and is grown from a dataset that is much narrower and more intentional.

Collaboration between artists working in different media is at the heart of EMERGENT SYSTEM. This final “collaboration” with a computer algorithm reflects that theme, and opens up new ways of considering our relationships with technology.

Themes

The film is a series of layered collaborations and transformations. Forms and colors move in systems that arise from Missy's music. Bodies appear, manipulating the forms (and each other). The drawings of bodies repeat and become patterns themselves. A woman tries to change the patterns around her. Finally, frames from the film itself are reinterpreted by a computer algorithm. At each step, a pattern emerges and a rebellion is staged against that pattern.

The act of recognizing patterns, and patterns of patterns, is a model of how brains work used in both neuroscience and artificial intelligence. When we imagine our minds at work, somewhere in the stack of discrete patterns, the mechanics become too complex to contemplate and give way to emotion.

Context in my practice

The drawings continue my practice of creating physical objects that capture and transform time-based experiences.

The film is part of another body of work of mine: experiences that combine animation with live music, created in collaboration with composers. The combination of visuals and music, in dialogue with each other, immerse audiences deeply in the ideas and aesthetics at the core of each work. In a time when cinema can feel disposable—something we can order up or discard on our phone on a whim—these kinds of live experiences feel particularly resonant.

My other projects in this body of work include PLAN OF THE CITY, MANNAHATTA, and A MARVELOUS ORDER.

Exhibition

A selection of drawings and videos from this body of work were presented in a solo exhibition by Standard Space, a boundary pushing gallery in the Hudson Valley, in 2021. At the opening composer Missy Mazzoli performed and the film was screened—both with quadraphonic sound, designed by sound artist Ben Manley.

Book

Drawings from this project were included in a limited edition book, published by the Wassaic Project, produced by Small Editions, titled Now, More Than Ever. The drawings are given their own spread as well as included as flipbooks in the corners of the pages.

The way these flipbooks render animation in a physical form that the viewer can hold and touch fits wonderfully with the goals of this project.

Credits

Emergent System
Directed and animated by Joshua Frankel
Music Three Fragile Systems by Missy Mazzoli
Choreography by Faye Driscoll
Music performed on six grand pianos by Grand Band
Pianists Erika Dohi, David Friend, Paul Kerekes, Blair Mcmillen, Lisa Moore, Isabelle O'Connell
Dancers Lyric Danae, Sean Donovan, Paul Singh, Eliza Tappan
Director of Photography Clayton Combe
Deep Learning Engineer Chan Woo Kim
Assistant Animator Yupu Ding
Thank you Mana Contemporary, Andy Hamingson, Dr. Andrea Danyluk, David Van Brink, Jason Baruch, Carter McGowan, Penina Biddle-Gottesman, Justin Edward Call, Ryan Brack and Eve Biddle
Commissioned by PEAK Performances at Montclair State University, (USA).