Body_Code
By Naoto Hieda
Naoto Hieda during a rehearsal. All photos in this article are by PERTE DE SIGNAL | Camille Montuelle
On March 3rd, 2018 during Nuit Blanche in Montreal, I participated in a performative open-lab showing at the Centre de Création O Vertigo (CCOV) as a collaboration between CCOV and Perte de Signal. The dancers invited by CCOV were 7Starr aka Lil Solow, Angie Cheng, Jeff Hall, Louise Michel Jackson, Simon Portigal and Elie-Ann Ross, and the digital artists were Cinzia C., Maxime Damecour, Naoto Hieda and Lucas Paris. Although I have never worked with Perte de Signal, I have been in contact with Maxime and Lucas and they invited me to join the project. My primary role was a digital artist, but I also participated as a dancer since my motivation is to blur the boundary between digital art and dance practices and simply I love to move. The showing was proposed as an open lab instead of a performance; Andrew Tay, the artistic director of CCOV, suggested to keep the format open (although we have decided a sequence of the scenes to be presented) so that the artists can try new ideas that they come up with during the performance.
It is worth noting that most of the artists have never worked with each other before this creation. They knew some of the artists within CCOV or PDS, but they have not necessarily worked together. Therefore, we started with a few brainstorming sessions to present each artist’s work prior to the creation. Among others I was inspired by Simon’s dance piece; he explained his performance that used green paint on his body as an analogy to a green screen. It reminded me of the video piece by Hito Steyerl and immediately thought about performing with a green screen and bring the idea further by projecting real-time graphics on the stage. Also Angie’s idea of using ordinary props as costume made me think of morphing the body, both physically and digitally. Then Maxime and I started testing a real-time video system with a green screen and a green cloth to hide ourselves in the video output. Since the nature of the project was rapid prototyping, I decided to use vvvv for chroma keying so that shaders and parameters can be tweaked on the fly, and Processing with Spout and LiveJS for live coding of the video content. I reused some of the sketches I have been making (CodingChallenges-Alt), which is a daily-coding project that I started this year, reinterpreting Daniel Shiffman’s Coding Challenge. Specifically, the codes I used were supershape, purple rain and pixel sorting.
Image from green screen and pixel sorting.
Real-time chroma keying turned out to be challenging as the color on the camera is affected by various parameters such as lighting color, intensity, camera shutter speed and white balance. I ported the green screen shader from Shadertoy, which assumes that the background color is close to RGB = (0, 255, 0). I relaxed the threshold parameter so that green textures (e.g., clothes and sleeping bag) are masked out too, hiding some body parts as shown in the picture above. Unfortunately, I had some technical difficulties during the performance, especially camera and video connection, but retrospectively this was expected as the setup was complex (Maxime and I were sharing the same camera and projector without a proper switching system). Also, I wish I had more time to experiment and investigate the relationship between the generated graphics and body movements. I had interesting conversation with Louise-Michel about fractal tree and movements originating from the center of the body but we could not finalize it. The other feedback was that the dancer cannot see the generated video while performing because the video was projected on the screen next to the green screen; it would have been useful to have a monitor facing the green screen for a practice. Nevertheless, this scene was driven by a clear motivation to morph a physical body in the digital domain, and also it served as an intervention to the other scenes that were more theatrical.
Work by Cinzia C.
The other scene I worked on was piezo sensors and interactive graphics. Cinzia and Lucas were already working on an interactive floor with piezo that reacts to dancers’ steps and changes the projected shapes (photo above). The geometry was reflected on the mirror and a light corridor was created with smoke. We had extra piezos so we attached them on the other stage and I was responsible for creating another interactive system with the sensors. First, I made a particle system that reacts to steps; however, the interaction was very similar to what Cinzia has done, so I changed the plan and instead of responsive geometry, measured vibrations were accumulated to affect the parameters of the supershape algorithm. For example, rubbing gestures will gradually change the shape.
I chose red for the supershape as Angie had a red blanket on her on the stage, which can be seen as a counterpart of the green screen. The interaction worked fine and brought another way of using piezos; however, during the performance, piezos were picking up the sound from the speakers and some subtle gestures were not noticeable.
Overall, the creation was very short and intense, but it was a great opportunity to work with talented artists. Like other dance-tech collaborations that I was involved, I found it is challenging to manage the schedule when digital artists and dancers are working together. Digital artists need time to implement and debug, which often make dancers wait for a long time; and when digital artists want dancers to test a program, dancers need time to warm up. I was confident that I know how to manage time in this situation, yet I could do a good job; I was overwhelmed that I had to prepare two scenes and also I was sick during the creation so that I was occupied by managing my own tasks. Nonetheless, the team had a great vibe, openly proposing ideas to experiment what technology works, what does not, and how to make it better. I am grateful for the opportunity and I thank all the directors, artists, staffs and audience to make the showing possible.