INTERACT: Applying Theory and Methods from the Visual and Performing Arts to Robots

This paper considers the impact of visual art and performance on robotics and human-computer interaction and outlines a research project that combines puppetry and live performance with robotics. Kinesics—communication through movement—is the foundation of many theatre and performance traditions including puppetry and dance. However, the aesthetics of these traditions vary across cultures and carry different associative and interpretive meanings. Puppetry offers a useful frame for understanding the relationship between abstract and imitative gestures and behavior, and instantiates the complex interaction between a human operator and an artificial actor or agent. We can apply insights from puppetry to develop culturally-aware robots. Here we describe the development of a robotic marionette theatre wherein robotic controllers assume the role of human puppeteers. The system has been built, tested, and demonstrated in public settings. We then describe INTERACT, a proposed research project that stages the robotic marionettes in a live performance. The interdisciplinary project brings humanities research to bear on scientific and technological inquiry, and culminates in the development a live performance which functions as an empirical measure of our research findings.