Project Type: Artwork
Category: Theatre
Client: Quanta Arts Foundation
Year: 2015
#Performing Art
#Real-time 360° Projection Mapping
#Kinect v2
Category: Theatre
Client: Quanta Arts Foundation
Year: 2015
#Performing Art
#Real-time 360° Projection Mapping
#Kinect v2
Second Body
After 'Seventh Sense', we continued to work with Anarchy Dance Theatre. The concept for this production, according to the piece's choreographer Jeff Hsieh, began with the perception that, while driving through lanes and alleys, he relied on his innate sense of the vehicle itself—its size, movement, and position—as if it were his own body moving through space. Body movements are primarily subconscious acts honed by knowledge, the senses, and practice, and when our body is able to utilize machines, a "Second Body" is made possible through practice.
The work "Second Body" focuses on concepts of the body, exploring how its existence and very definition may change in face of an increasingly technological world, where the external environment and physical space in which we live are themselves changing.
Service / Idea
With "body" as the thematic core of this piece, we wanted to create a system that could randomly alter the body's form and appearance. To this end we constructed an auditorium with an 8mX8m stage surrounded by the audience, wherein the dancer in the performance space are covered by 360° projection, and the floor's interactive projection becomes an extension of the dancer's own body.
Workflow / Detail
Before development began on this project—because the visuals and technological technical solution had not yet been decided upon—there was much in the way unknowing. To ensure the balance between technology and aesthetics, we extended the production to comprise three distinct stages.
Phase One: Technical Development
At this stage our aim was to experiment with visuals at a definite scale to both develop and understand the performance capabilities of our system. Our goal was to create projected images that perfectly adhered to the body of the performer.
The first problem we faced was how to create seamless projections on the dancer in real time. At that time, the most accessible advanced sensor was Microsoft's Kinect v2, and our detection system was thus developed using this sensor as its basis. After some trials and testing, we had decided to install four sets of Kinects around the 8mX8m stage, thus generating a 360°, seamless point cloud of the dancer for model creation.
The next issue was how to project images onto the body of the performer, the resolution of which comprised both linking the positional relationship of the projectors and sensors, as well as adhering the images to the model obtained by the detection system. The former required establishing a checkerboard grid that acted as the projector's utilization plane. The sensor could then determine its positional relationship with the projector by assessing positions on this plane. For the latter we used common environment mapping technology via 3D software, rendering the model's appearance into a mirror image, which the Skybox could then successfully reflect onto the body of the performer.
After establishing the basic system, we further classified the visual stimuli into two categories to test and develop their potential: the first type, pre-render videos, was established after many trials; the second type was generative visual during performance.
The first problem we faced was how to create seamless projections on the dancer in real time. At that time, the most accessible advanced sensor was Microsoft's Kinect v2, and our detection system was thus developed using this sensor as its basis. After some trials and testing, we had decided to install four sets of Kinects around the 8mX8m stage, thus generating a 360°, seamless point cloud of the dancer for model creation.
The next issue was how to project images onto the body of the performer, the resolution of which comprised both linking the positional relationship of the projectors and sensors, as well as adhering the images to the model obtained by the detection system. The former required establishing a checkerboard grid that acted as the projector's utilization plane. The sensor could then determine its positional relationship with the projector by assessing positions on this plane. For the latter we used common environment mapping technology via 3D software, rendering the model's appearance into a mirror image, which the Skybox could then successfully reflect onto the body of the performer.
After establishing the basic system, we further classified the visual stimuli into two categories to test and develop their potential: the first type, pre-render videos, was established after many trials; the second type was generative visual during performance.
Phase Two: Content Creation
After completing the first stage, members of the production team better understood the qualities and limitations of the detection system, and were able to flesh out the finer details regarding script and performance structure. The creation team switched focus to content creation and the development of a more user-friendly editing system. To better grasp the relationship between the dancer and the on-stage visuals, the team spent an entire month staying in the performance space rehearsing, honing, and refining movements.
Phase Three: Establishing an edition suitable for touring
Because of the piece's unique presentation and large-scale technical requirements, we specifically designed a hardware installation, logistics method, and calibration process for touring.
Event
2016.10.22 One Dance Week, Bulgaria
2016.06.17-18 Les Rencontres Chorégraphiques Internationales de Seine-Saint-Denis, France
2016.05.21-22 The Polytech Festival, Russia
2016.5.13-14 Centrum Sztuki WRO / WRO Art Center, Poland
2015.09.4-5 Ars Electronica, Austria
2014.10.31-11.2 Performing Arts School 36, Taipei
Second Body
Concept / Choreography: Chieh-hua Hsieh
Software Development / Visual Design: Ultra Combos
Sound Design: Yannick Dauby, Ultra Combos
Lighting Design: We Do Group
Custom Design: Yu-teh Yang
Visual Operator: Hsiang-ting Teng
Soloist: Kuan-ling Tsai
Dramaturgy: River Lin
Tour Management: AxE
Arts Management Project Mentor: Justine Beaujouan
Project Consultant: Kevin Cunningham
Commissioned by Quanta Arts Foundation / QAring
Supported by Ministry of Culture of the Republic of China
