RIT educates tomorrow’s experts in virtual production

A massive LED wall displays a stirring scene in which a helmeted actor mounts a prototype electric motorcycle before it appears to speed off down the highway. Meanwhile, a team of advertising company artists works with 3D digital design students alongside engineering, motion picture science, and film and animation peers to seamlessly produce the action.

‌

Sebastian Nazario-Colon/RIT student

In-camera view—A monitor shows the live, in-camera visual effects made possible by virtual production technology.

While it might sound like something that could only happen inside a Hollywood film studio or a production stage in New York City, the video for RIT’s Electric Vehicle Team was produced on campus last spring using the latest in virtual production (VP) technology inside MAGIC Spell Studios—RIT’s world-class digital media research and production facility.

VP blends filmmaking, 3D graphics, computational photography, and real-time game engine rendering to produce in-camera visual effects similar to those seen in the groundbreaking work on Disney’s The Mandalorian and Marvel’s Avenger films.

“Virtual production represents a sea change in the way content is being created today,” said Aaron Gordon ’13 (film and animation), founder and CEO of Optic Sky Productions, a commercial and digital experience advertising company borne from RIT’s Venture Creations business incubator in 2015.

A multidisciplinary team of RIT students and faculty, together with Optic Sky, collaborated with EPIC Games, creators behind Unreal Engine and popular video games such as Fortnite; THE THIRD FLOOR (TTF), an award-winning visualization studio co-founded by Chris Edwards ’97 (film and video production); and Production Resource Group (PRG), one of the leading providers of entertainment and event technology.

‌

Trisha Pickelhaupt ’21

Correcting the composition—Aaron Gordon ’13 (film and animation), left, and Sullivan Slentz ’14 (film and animation), both with Optic Sky Productions, assess composition for the foreground motorcycle and background virtual environment.

“Because MAGIC has such a strong educational component, it is fostering research and development with partners like us that other studios can’t do because it’s not the business model,” Gordon said. “For us, MAGIC has changed the game completely because we have both a research partner and a studio partner helping to bring virtual production to our clients.”

David Long, MAGIC Center’s director, explained that VP uses on-set displays to create in-camera visual effects. Compared with more traditional methods, like green screen compositing, VP can save weeks and often months in post-production.

“Real-time responsiveness and creativity is why it’s so cool now and gaining so much attention,” Long said, noting that RIT is in a unique position to educate tomorrow’s experts in virtual production. “We are in an exclusive club that is training the future technologists and engineers, alongside the artists.”

Last year, a grant that Long co-wrote with Shaun Foster for the development of a multi-departmental Virtual Production curriculum (using Unreal Engine) was funded by Epic Games. The curriculum has enabled RIT students to learn and research state-of-the-art techniques at MAGIC.

‌

Sebastian Nazario-Colon/RIT student

Learning experience—Students Carla Bello, left, and Malcolm Zale receive camera instruction from freelancer Dave Maslyn.

Foster, undergraduate program director of 3D digital design, had already been leveraging his own grant from Epic Games to explore and develop dynamic virtual content that has opened the eyes of many to the software’s possibilities. As part of the grant funding, he launched a new course called Cinematography and Pre-visualization during the spring 2020 semester.

One of his former students, Noah Mesh ’21 (3D digital design), credits Foster’s class with playing a key role in helping him land a job last summer as a previsualization artist at TTF.

“Shaun Foster’s ‘previz’ and cinematography class first exposed me to what previsualization is all about—its function as well as the basics of cinematography and storytelling,” Mesh said. “Previz is definitely heavy on animation, so I also was excited to have taken animation classes while at RIT.”

Today, as a previsualization artist, Mesh said he is building 3D-animated scenes based on storyboards and scripts developed for the project to which he’s assigned.

‌

Sebastian Nazario-Colon/RIT student

Tim Stringer ’19 (film and animation) navigates Epic Games’ Unreal Engine to refine the virtual environments. Assistant Professor Mark Reisch looks on.

“More specifically, we place virtual lights, cameras as well as real (digitized actors), and CGI (computer-generated imagery) characters in virtual environments and then animate them to create a high-quality conceptualization of each shot that needs planning,” Mesh said.

“It’s complementary to ‘real-time virtual production’ with big LED walls, using similar tools and techniques, just at a pre-production stage in the project,” he added.

Foster currently is teaching a previsualization and virtual production course that connects to 3D digital design’s emerging specialty in “real-time 3D” using the Epic Games Unreal Engine. He’s also taught a previsualization course with virtual production elements for the past two years. During the spring, Long and Flip Phillips from the School of Film and Animation co-taught their own virtual production class, which overlapped heavily with Foster’s.

Long said a top goal for MAGIC is to be able to teach on these platforms “so building a curriculum around them and showing the students from multiple programs how to use the technology is vital to our mission.”

‌

Sebastian Nazario-Colon/RIT student

LED display wall in action—Students Simon Yahn, Dallas Calkins, Malcolm Zale, and Johnny Gates observe camera operation.

Todd Jokl, dean of RIT’s College of Art and Design, points to virtual production as one more example of the growing number of learning opportunities inside MAGIC Spell Studios for students in a wide range of disciplines at the university.

“VP is not just for movies,” Jokl said. “It points to RIT’s progressive view of how a wider range of disciplines such as photography, imaging, and generated experiences are converging.”

Jennifer Hinton, MAGIC’s associate director, said successful advertising companies like Optic Sky and Mountain House Media are prime examples of “companies that are raising an ecosystem of economic development in the region by growing the economic impact of digital media production in Rochester,” including the hiring of RIT co-ops and graduates. The founder and CEO of Mountain House Media, Jeremiah Gryczka, is a former Gap Year Fellow and a 2019 graduate from RIT’s School of Individualized Study.

For its part, PRG officials expressed pride in “supporting the next generation of production, technology and cutting-edge creatives who are integral to the future of our industry,” said Andrea Vestrand, an account executive with the company.

‌

Still from Optic Sky Productions

This image comes from the final 4K video that is the culmination of weeks of work. The video was created for the Electric Vehicle Team at RIT.