Student researches motion capture technology through MAGIC Spell Studios

When students in RIT’s College of Art and Design have a motion capture need for a project, they often look to Payton Schleh ’23 (motion picture science). 

Schleh has worked on co-op with MAGIC Spell Studios researching the latest motion capture technology and techniques. Motion capture is the process of recording movement of objects or people with the use of sensors and markers, and incorporating it into media such as video games, animations and films. 

Schleh operates out of a lab in MAGIC, where she has the cutting-edge equipment and know-how to meet the growing demand for motion capture technology in various student projects. Schleh’s technical and creative support has benefitted students in the School of Film and Animation and the 3D digital design program.

“I enjoy helping people out and seeing what they’re doing,” said Schleh, from Portland, Ore. “I’ve always enjoyed getting my hands on equipment and working with the tools and figuring out what they can do. Getting the chance to do that here, as part of my major, has been really great.”

She experiments with different motion capture systems. Namely, Schleh explores the technology through alumni-donated Vicon cameras and a Perception Neuron system that possesses industry-leading sensor technology and algorithms. 

Schleh has used the Perception Neuron suit to record motions for student animations, virtual production projects and even American Sign Language hand-tracking. She even attached motion-detection sensors to a stick-shaped object and simulated the movement of a whale for an aquatic character in an animation.

“It was fun taking some of the trackers we have and applying it to something that maybe wasn’t intentionally designed for tracking,” Schleh said.

Schleh’s research role evolved beyond motion capture. She also developed a familiarity with RIT’s state-of-the-art virtual production setup to bring the revolutionary media production technique to student projects.

Virtual production represents the deployment of extensive computer-generated graphics, 3D art, motion capture, in-camera visual effects and real-time rendering via game engines. The innovation uses on-set displays to create in-camera visual effects, saving a lot of post-production time compared to traditional methods like green screen compositing.

‌

A behind-the-scenes look at a film Schleh was instrumental in making in a virtual production class last fall.

Gargantuan LED screens and other novel technology stationed in MAGIC Spell Studios, in tandem with industry partnerships, provides a full research and learning ecosystem around these emerging workflows. 

Schleh is currently taking a class taught by Professor Flip Phillips as part of RIT’s virtual production curriculum and serves as a central figure for supporting virtual production-driven content on campus.

“I’ve kind of taken on the role of being the virtual production operator for a lot of the sets that want to use the wall and camera tracking,” she said. 

Schleh’s capstone research also relates to virtual production. It’s an investigation of issues centered around interactions between an LED virtual production wall and cameras. She is researching how to seamlessly blend motion blur and its additional processing time on LED displays and synchronizing the wall’s refresh rate and camera’s capture rate.

Schleh transferred into RIT and found community in the motion picture science program by joining RIT’s student chapter of SMPTE (Society of Motion Picture and Television Engineers). This year, she is the organization's president, facilitating connection with alumni and other professionals. 

In high school, Schleh had a penchant for technical theater — all of the components, from scenery to lighting to sound, that go into making a staged production. She advanced her understanding of the science and engineering driving the entertainment industry by enrolling in motion picture science, one of RIT’s New Economy Majors.

As she prepares to graduate in May, Schleh looks for her distinct skill set to set her apart.

“Motion picture science is definitely a unique major,” Schleh said. “In talks I’ve been having with companies as I go through my own job search, the experience we get in motion picture science, as engineers and artists, intrigues a lot of people. You’re able to tackle technical problems and communicate why certain technology works and how it works. It's definitely something in demand.”