XRLive: Computational Tools for Live Performance

Explore the use of computational tools, technologies, and methodologies in the process of the production of live theatrical and dance performances. Sample technologies on which we focus include virtual, augmented, and mixed reality, facial and body motion capture, eye tracking, volumetric video capture, projection mapping, audio synthesis and analysis, lighting and special effects.

The project will produce its own live performances in addition to working with other performing arts groups and projects across campus.

Our on-going project is the creation of a 3D engine specifically designed for incorporating virtual and augmented reality with live theatrical performances supporting the technologies listed above. Questions around the effectiveness of the engine with respect to speed, accuracy, and the audience experience will also be addressed.

The project will explore the use of a variety of technologies including (but not limited to):

• 3D Programming Engines (Unreal, Unity, Godot)
• VR/AR Technologies (Meta Quest devices, Microsoft Hololens, Apple Vision Pro, Magic Leap)
• Body Motion capture (Sony mocapi, Microsoft Kinect, Perception Neuron)
• Facial Motion Capture / Eye Tracking (Meta Quest Pro)
• Volumetric Capture Devices (TBD)
• Projection Mapping (Touch Designer, Isadora)
• Theater cueing tools (QLab)

The technologies used for any particular project will depend on the needs of the live performance being produced.

• All computing majors (e.g. computer science, game design and development, software engineering,new media interactive development.)
• 3D digital design
• Film and animation
• Motion picture science

Irrespective of major, students should have a strong interest in live performance, theater, dance, music and/or theater technologies. Students should also have some basic computer programming skills.

The above list is just a sample of majors from which students may be interested. The list is not exclusive. All students with basic programming abilities and a passion for live performance are welcome to apply.

Meeting time will be set based on schedules of accepted students.

For more information and a list of recent productions, go to the XR Live Project website.