11th Choreographic Coding Lab
By Naoto Hieda
The 11th Choreographic Coding Lab (CCL) organized by Motion Bank took place at Kunsthalle Mainz (Mainz, Germany) from April 28th to May 2nd, 2019. It was my third participation in CCLs; the first time was in New York in 2015, where I used Deborah Hay dataset to move a plotter. The second time was in Amsterdam in 2017, where various collaborations emerged; notably, Lads with Chris Matthews and the second iteration of Pathfinder with Mio Loclair, which became a single day workshop in Bucharest called CCL+Pathfinder during the NOVA Festival at Point theater in 2018.
In this edition of CCL, Florian Jenett and Anton Koch from Motion Bank made a new tool as a spin-off of PieceMaker, which is a platform for dance data recording and annotation. They made a player application called Effect Data Player that streams motion capture data of “Effect” choreographed by Taneli Törmä, which is the main piece for the exhibition “Between Us” at Kunsthalle. PieceMaker handles various data structures such as videos, annotations and motion capture data, but the downside is that it is not suitable for rapid prototyping because of its complexity. On the other hand, the new Effect player is a dedicated application for streaming motion data through OSC (Open Sound Control) or Websocket, which is easily accessible from frameworks such as Processing and HTML5.
This enabled many of us to work on the dataset and during the presentation we could see transformation of the dance data into visual and sound. It also helped that the traces of the dance piece are geometric so that we were not using the movement data as a random data stream. We shared a clear idea that we were extracting the choreography from body movements and expressing in other mediums. These movements were abstracted and no longer embodied, or re-embodied in a way that is not intended in the original dance piece.
During the week, I spent most of the time with Tristan Schulze and his students Stephan and Katja; I contacted him after discovering his algorithmic tapestry series at Kindl (Berlin), and while we had a coffee meeting, we planed an “excursion” to CCL together. At CCL, Tristan worked on a sound piece that is reactive to the dancers. I was observing his creation process and Erica, Alvin, Sonja and Abi choreographing for the system as well as recording movement data with the Captury system, which is a markerless tracking using color cameras and a machine learning algorithm similar to PoseNet. Unfortunately, they could not have live data stream by the presentation; instead, they work on recorded or offline data, and I think this made the choreographic process even more interesting. The dancers had to imagine how the sound would react to the movements without experiencing the reactive system. The process looked as if the dancers are coding through the choreography — unlike a conventional geometric choreography but rather like a simulation of a complex phenomenon, which is an interesting topic for further collaborations of media artists and dancers in a conceptual level.
As for my project, I started working on the idea of digitizing movement snippets and associating the movements to texts or 3D shapes. I am interested in translation of one modality to another; for instance, how words can be related to gestures or images, and how it is influenced by different perspectives (e.g., culture, gender or neurodiversity). At the moment, I focus on my perceptions, which I am most familiar with. As opposed to Motion Bank’s PieceMaker platform, which is intended to record a full dance piece (or its rehearsal) and to make annotations, my intention is to record short video clips of movements, which can be tagged with words, colors or even a Processing sketch, and potentially remix this movement vocabulary to create totally another piece. I am refining the tool as a web version, and hopefully, I can share next iterations online soon.
Huge thanks to the Motion Bank team for organizing the event and providing software and equipment; Marc Downie, Maria Judova, Constantine Nisidis, Liam Blackburn and Jean-Philippe Rivière for artistic and scientific inputs; Erica and Alvin for a warm-up session; Alexander Trattler for a TouchDesigner workshop; and all the participants for making such an amazing event together.