Conference 5–9 August 2012
Exhibition 7–9 August 2012
Los Angeles Convention Center

ARCADE: A System for Augmenting Gesture-Based Computer Graphic Presentations

Los Angeles Convention Center, West Hall B

ARCADE is a system that allows real-time video-based presentations that convey the illusion that presenters are directly manipulating holographic 3D objects with their hands. This illusion is maintained by custom 3D finger tracking from Microsoft Kinect depth-image data, pre-authoring of smart interaction objects that can respond to a presenter’s gestures with high-level behavior, and a suite of rendering effects that convey the visual illusion of a floating-in-air holographic projection.

The system is currently used as a research platform for improving finger-based gesture recognition and as a production tool for creating compelling videos for distance education. This Real-Time Live!, ARCADE demonstration features ARCADE's application on a diverse range of topics, including computer graphics, mathematics, physics, chemistry, biology, and cartography. This project is inspired by the post-production effects typically used in Hollywood films, such as “Minority Report", “Harry Potter", and “Star Wars", but it enables non-professionals to author these experiences and then perform them live.

Murphy Stein
New York University

Xiao Xiao
MIT Media Laboratory

Jonathan Tompson
New York University

Charley Hendee
Tactonics Inc.

Ken Perlin
New York University

Hiroshi Ishii
MIT Media Laboratory