THE DIMENSIONS OF DATA SPACE

By Scott deLahunta

29 January 2003

From 10-14 December 2002, the Monaco Dance Forum hosted the Motion Capture Tech Laboratory "Real Time and Networked: Sharing the Body". [1] The overall objective was to engage in an investigation led by artistic questions and processes into the use of real time and networked motion capture and computer animation systems. [2] The core research team included dance and programming artists who are already working with these systems and have complementary approaches. Two researchers documented the laboratory and facilitated reflection on its broader implications through interviews and group discussion. A commercial motion capture company provided systems and support and took part in all creative aspects of the laboratory. The motion capture systems used included two Gypsy exoskeletons, the Polhemus Startracker and the Motion Captor optical system. [3] The software was off the shelf programmes (e.g. Kaydara Filmbox) and customised code.

The research team comprised, in alphabetical order, Tania Barr (FR), Scott deLahunta (NL), Nik Haffner (DE), Maurice Kadaoui (FR), Bernd Lintermann (DE), John McCormick (AU), Thomas McManus (DE), Armando Menicacci (FR) and Emanuele Quinz (FR). Emmanuel Berriet (FR) and Mark Coniglio (USA) participated on a part-time basis. See Appendix for related URLs.

The team arrived on Sunday, 8 December to set up the spaces and continued this work on Monday [4]. Beginning Tuesday, the research laboratory was in operation from 10.00 to 18.00 hours daily. The space was generally open for those who wished to observe, and there was a formal open visiting period at the end of the day when the research team was available to provide demonstrations and explanations. The research laboratory concluded with a final fifty-minute presentation at 16.00 on Saturday 14 December to an audience of approximately sixty. This was followed by some further discussion and the formal close of the lab.

The following is a selection of three separate but related lines of thinking, inspired and progressed by the experience of the lab. The first attempts a description of motion capture systems in terms that are more cultural than technical. The second draws on the experience to make some observations on cross-disciplinary creative research processes. The third line of thinking seeks to make some relationships that might contribute to our understanding of working within computation systems. I have opted for the following self-imposed interrogative format as the best way to convey these lines of thinking that are currently incomplete and, like the research laboratory itself, exploratory.

Q: Why aren't you going to write about the technical systems? What if your readers don't understand what motion capture is?

R: I might have felt compelled to do this a few years ago, but I sense general knowledge of these systems amongst dance artists in particular has increased. There has also been growth in computing science and engineering research in the field of sampling, synthesizing and modelling motion in three dimensions. In addition, a number of small commercial initiatives (in different countries) seeking niche markets for customised motion capture solutions have emerged. The evidence for all of this seems easily available on-line, and I would like to encourage everyone to look on the Internet so that they see the contexts within which this knowledge is being developed.

What seems missing from the 90,000 plus pages that come up if you search on "motion capture" is any broad cultural analysis of this field. You do find some histories of the development of the technologies, but little else besides information related directly to building functional systems. I am speculating that this is partially because motion capture systems exist in a state of extreme instrumentality relative to the uses for which they are built. In other words, they are so tightly woven as systems to the purposes of either animation or motion analysis; that they seem to be pure instruments or tools. Almost no one involved in the creation and use of motion capture systems deviates from these trajectories of purpose. If the system is not used "properly" it generates "useless" materials, and in the context of either motion analysis or animation this deviation away from utility would just be too costly in terms of both time and money. This means that it can be very difficult for artists to intervene in these systems somehow; to hack into them, twist, challenge and allow for or cause accidental forms to arise from them.

Q: Why, are you so concerned about motion capture systems being so purely instrumental? How would you apply this thinking to the work in the motion capture lab?

R: One of the things at issue here is we need discourses that make distinctions between artistic, commercial and scientific research. They are not the same processes, and these days when we are being encouraged to collaborate across these sectors more often it is all the more important to develop an understanding of these differences. During the research laboratory in Monaco in more than one instance accidental data was being explored through, for example, the conscious occlusion of some of the reflectors for the optical system and the proposal to use an alternative calibration for the Gypsy. Thomas tried to 'outrun' the optical motion capture system one day to see if it could keep up with very fast movement. These are not trivial strategies; they underpin the types of investigatory processes that, in my opinion, we need to open space for in relation to motion capture systems. These are the conditions from which unexpected creative forms are going to emerge, and we were lucky to have the opportunity to explore this in the research lab.

Q & R #2:

Q: You have referred to the conditions of the laboratory as being very generative and stimulating. How would you describe this?

R: Well, it's crucial to remember that we were only together for a week and that as a group we were relatively new to each other. We needed to set up a good process for the exchange of ideas related to artistic practices. So, to begin with we did not pursue any single line of enquiry and had group discussions whenever they were necessary. These discussions tended not to determine work processes as much as respond to and guide them. This is important; it was conducive to an atmosphere of 'doing' and playfulness, trial and error, and a reliance on intuition. There were also many things happening simultaneously. So, the lab was more like a brainstorming session than following a predetermined set of designed or developmental procedures. This type of process stands in contrast to the instrumentality of these motion capture systems that I already mentioned.

It is important to note that the conditions included a primarily implicit commitment to open processes. We all know that the idea of an open (knowledge development) process has implications for intellectual property issues, in particular in the commercial and scientific marketplace, but we maintained this tacit contract between us to be as non-proprietary as possible. And this was not only amongst us, but also with all those who came to observe. It might be wise to underpin any future stages of research and development work by being more explicit on this topic; but at such a preliminary/ exploratory stage it is, in my opinion, okay and maybe even better to operate in good faith.

Q: Didn't you work under the understanding that you didn't have to have an "end product"?

R: Yes, it was made clear at the start that we were not aiming for any particular "end product". This didn't mean we would have nothing to show as the results of our research, we had plenty of things to show and discuss both during and at the end; but it helped to establish the ground from which a variety of ideas could be explored with an eye to the range of possible "end products"; for example; various art works, software solutions, compositional strategies, etc.

Q: Isn't deferral the danger of such an approach? It seems you could just end up with a series of endless demonstrations of things that have potential but are not finished.

R: You are right to mention this. Heidi Gilpin and Lorne Falk in their 1995 article "Demo Aesthetics" have written an interesting critical piece on the implications of the emphasis on description and demonstration that seems to permeate a lot of artworks using new technologies. [5] They write about an aesthetics that "invokes a work of representation that is unfinished" or that is in a state of endless reformulation. They situate this in the current techno-cultural climate as a tendency to reconfigure the prototype as a product, which makes it commodifiable as such. So, it is something to be aware of.

What is required in the context of experimentation in the performing arts field, in my opinion, is something in between this pressure on the one side to prove how technologies either succeed or fail in the context of the stage performance (as an end product) and on the other side the value of working processes that develop a clear understanding of the terms and context of artistic research in relation to other practices that are foregrounding innovation. Both situations can end up either generating or squashing new forms of expression and ways of thinking; so it's not an either/ or situation. But one thing is abundantly clear to me after observing many projects involving complicated digital technologies and live performance making. They really benefit from a generous amount of development time and being able to proceed in clear stages or phases. Each phase contains an evaluation of its own outcomes and this helps to determine the direction(s) for the next, sort of a recursive process you might say. With this in mind, I would characterize our motion capture lab in Monaco as a "preliminary research phase" that resulted in successfully establishing effective social relations and working vocabularies from which to depart.

Q: Why did you title this report "dimensions of data space"?

R: One of the major lines of inquiry during the lab was the question of "what are the properties of these motion capture systems?" In seeking to learn more about these properties we decided to spend as much time as possible just being in the systems - so that we had a constant physical experience of the dimensions of real space in relation to the dimensions of the data space. But how can we think about the dimensions of data space? One place to start is the concept of 'calibration'. Motion capture is essentially a measuring instrument and like all measuring instruments it requires calibration to align its internal units to the real world units. Calibration manifests a level of description within which other descriptions have meaning, and all motion capture systems, optical, magnetic and exoskeletons involve different procedures for it. For instance, calibration of the gypsy aligns the exoskeleton with the body that wears it; so the dimensions of data space lie very close to the mover. Without this level of description the system has no context to recognise the data being generated by the mover.

I think this is one of the keys to developing a better understanding of the relation between physical and computation spaces, the relation between real bodies and data bodies if you will. It is partly down to the organisation of levels of description that can be understood by the mover and the information system and can travel in both directions in and out. [6] Dance practitioners in general have difficulty with imagining the dimensions of data space in any tangible and therefore potentially creative way. What underlies this is the lack of an adequate set of formalisms for describing gesture and movement in terms that not only the system can interpret, but are equally accessible to choreographers. Motion capture is an interesting technology, but uses descriptions of motion based in mathematics and invented by computer scientists and engineers, physicists, bio-mechanists and human figure animators. There is probably no need to invent new mathematical descriptions based on the needs of choreographers; but to use what exists in new and innovative combinations that can be integrated with the working processes of dance makers. This is what I meant by levels of description that can travel in both directions in and out of the system.

This is not so much a matter of teaching choreographers to be mathematicians, but in developing an understanding of a range of co-meaningful representations, classifications, algorithms, notations and codes. My feeling, affirmed by the experience of the research lab and by some promising initiatives taking place, is that we are on the cusp is seeing a shift in this area. [7] If we can encourage and support growing awareness and understanding of the properties of motion capture and other information systems amongst choreographers and dancers this should stimulate imaginations and may quicken the emergence of these generative shared descriptions.

END/ END/ END/ END/ END

[1] Taking place for the second time (1st edition 2000) at the Grimaldi Forum in Monaco, the Monaco Dance Forum 2002 was a five-day international gathering comprising a diverse range of events including performances, exhibitions, symposia, multimedia installations, showcases and the International Dance Screen competition. http://www.mddf.com

[2] Since the early to mid 1990s, dancers, choreographers, multimedia artists and software programmers have been collaborating in exploring the uses of motion capture technologies in artistic projects; establishing precedents for the exchange of creative ideas and practice from which current and future arts researchers can depart. For some historical information and references to some of these artworks; please refer to: "Choreographing in Bits and Bytes", January 2000 http://www.daimi.au.dk/~sdela/bolzano/. (Also published in La scena digitale. A. Menicacc and E. Quinz, eds. Venezia Marsilio, 2001) and "Virtual Dance: a report on the Riverbed residency" http://www.dartington.ac.uk/staff/sdelahunta/uci/rivrep.html (University of California, Irvine, May 2001). To read about a similar project that took place in held in Athens May 2001 see the TRANSDANCE report. http://huizen.dds.nl/~sdela/transdance/report/. In addition, dance education institutions have begun to invest in experimentation with motion capture systems, e.g. the Environments Lab at Ohio State University http://www.dance.ohio-state.edu/workshops/mocap.html.

[3] URLs for these systems include: Metamotion (Gypsy) http://www.metamotion.com; STT (Motion Captor) http://www.simtechniques.com; Polhemus (Startracker) http://www.polhemus.com/; Also see Animazoo sites for sales/ services: http://www.animazoo-europe.com and http://www.animazoo.com/.

[4] The list below provides a general description of the technical requirements for the laboratory.

1. Sufficient space and type of floor for movement work.
2. Tool Kit: screwdrivers, pliers, gaffer tape, etc.
3. Adaptors (various), routers, hubs, splitters, etc.
4. Cables (video, ethernet and power)
5. Tables, Stands, Chairs, etc.
6. Broadband Internet connection
7. Lighting system (simple but controllable)
8. Sound system to include wireless microphones, amplifiers, speakers, etc.
9. Blank recording media (dvd, cd rom, dv tape, etc.)
10. PCs and Macs (portables and desktops/ workstations and servers) with sufficient processor speed, RAM, graphics cards, hard disk space, i/o ports, cd and dvd burners, etc.
11. Software (2-d and 3-d computer graphic software, audio/ video editing, etc.)
12. Digital cameras (still and video) and tripods
13. Data projectors and screens.
14. Wireless devices: transmitters/ receivers, etc.
15. 3-D Motion Capture Systems (optical, magnetic and exoskeleton)
16. Misc. input/ control devices, e.g. midi-keyboard/ slider; data glove, joystick, etc.

[5] Lorne Falk and Heidi Gilpin. Demo Aethetics. Convergence: the journal of research into new media technologies. 1:2, Autumn 1995. pp. 127-139.

[6] For a relevant discussion on "descriptions of culture" in relation to digitisation see: Garcia, David, et al. Content Integrated Research in Creative User Systems. Executive Summary of the Third Annual Report for the period December 2000 - September 2001. ESPRIT Working Group 29549. http://www.circusweb.org/

[7] For some of these initiatives see "Motion-e" at the Institute for Studies in the Arts, Arizona State University (http://isa.asu.edu/projects_motione.html) and "3d-traces: an interface for choreographers project" being developed by partners in the UK, Germany, France, Australia and Netherlands (http://huizen.dds.nl/~sdela/3dt).

Tania Barr and Maurice Kadaoui. Directors and owners of Animazoo Europe based in France. http://www.animazoo-europe.com, http://www.animazoo.com, http://www.metamotion.com

Mark Coniglio. New York City based composer, programmer and performance maker. Co-director of Troika Ranch. http://www.troikaranch.org, http://www.troikatronix.com

Scott deLahunta. Researcher (Dartington College of Arts, UK) and Writer based in Netherlands. http://huizen.dds.nl/~sdela

Nik Haffner and Thomas McManus. Former dancers with Ballet Frankfurt; now independent dancers and choreographers and both artists in residence at ZKM, Karlsruhe, DE. They are also members of the performance group "commerce".

Bernd Lintermann. Artist programmer currently artist and scientist in residence at ZKM, Karlsruhe, DE. http://i31www.ira.uka.de/~linter/

Mâa (Emmanuel Berriet). Software explorer and president of La Graine/ The Seed. http://www.lagraine.com

John McCormick. Co-artistic director of "Company in Space" and artist in residence at RMITs interactive information institute. http://www.companyinspace.com/

Armando Menicacci and Emanuele Quinz. lecturers, writers/ editors, researchers both working at the Paris 8 University Dance Department and with Anomos. http://www.anomos.org

We would like to thank the Monaco Dance Forum in particular Phillipe Baudelot, producer of the multimedia projects, and the technical support team lead by Nick van der Heyden.