Researchers develop manufacturing training that will include AI and virtual reality technology

Researchers at Rochester Institute of Technology are using augmented and virtual reality as part of a modern training platform to help address the skilled labor shortage in manufacturing.

Traditional positions in the manufacturing industry are evolving and require new skill sets to manage processes and equipment. Today’s Industry 4.0 and advanced manufacturing operations require knowledge about smart technologies and automation. Using artificial intelligence (AI) and augmented or virtual reality (AR/VR) applications as a training strategy can support novice trainees in learning critical manufacturing skills, said Yunbo Zhang, an assistant professor in RIT’s Kate Gleason College of Engineering and part of the research team developing the technology platform. Developing smart technology solutions can also be a means to retain the knowledge of master machinists and manufacturing engineers.

“There are challenges in the workforce in the U.S.; manufacturing industries require a qualified workforce, meaning they can work the machines and conduct the manufacturing process in factories. We believe the training would be an important way to improve the situation,” said Zhang, who teaches in RIT’s Industrial and Systems Engineering Department and has a background in advanced manufacturing and human-centered design.

Trainees would wear virtual reality goggles and through an immersive experience will learn to use the manufacturing systems in a factory setting. Instructions and training tasks would allow trainees to explore that setting safely and efficiently.

“Augmented or virtual reality might be new to people, and for some, it is hard to understand what their experience could be. Most people, when they are doing training, are reading material or watching YouTube. Our platform will not be like that,” Zhang explained, adding that actions required to use equipment will be modeled in the virtual reality system. Replication in the practice environment will help to imprint the knowledge for later use in the workplace setting.

The history of manufacturing in the U.S. is of apprentices being trained by seasoned journeymen with experience on multiple machines and equipment. Much of that former training was hands-on and took place over several years. Traditional positions are evolving and operations now include more integration of smart technologies and automation. The need for employees skilled in both areas is vital.

Zhang believes the virtual format will still allow for mastery of processes because the system would mimic the real-world scenario as close as possible. Online practice could safely model actual steps needed to use equipment and mistakes could be corrected. The process could also benefit companies who can administer a cost-effective training process for new employees and those needing to be re-skilled.

Both Zhang and Chao Peng, who is co-PI on the project, bring extensive backgrounds in industry and product development. Peng, an assistant professor in the School of Interactive Games and Media in RIT’s Golisano College of Computing and Information Sciences, will use VR and gamification technologies to build the immersive training system through head-mounted displays. He also sees this as an opportunity to share with his students that games can be used successfully in the field of training and professional development.

“Many of our students join our program because they want to make games. They are attracted by some of the entertainment elements,” Peng said. “Through this project, I want to bring in another perspective, that gaming is not just for entertainment. We are seriously teaching through this game-like application. Our students would have the opportunities to collaborate with people from other disciplines who may need their knowledge of game design and development.”

Zhang and Peng’s project, “Affordable and intelligent augmented reality platform for manufacturing workforce and education,” is funded by the National Science Foundation. The work is part of a larger project sponsored by Purdue University: “B1: Future Jobs and AI; Skill XR: An affordable and Scalable X-Reality Platform for Skills Training and Analytics in Manufacturing Workforce Education.” The overall goals of the research and development work is to expand the ideas of national apprenticeships to meet emerging industrial needs.

Going forward, the RIT team will engage other regional educators, social scientists, and AI researchers. They are also collaborating with the Strong Museum, located in Rochester, N.Y., on a new project to build a “fabrication corner” in the museum. Zhang and Peng are working with the staff on preliminary designs to deliver a simpler version of the workforce program as a way to increase children’s interest in the fields of manufacturing, engineering, and game design.

“We are first trying to focus on one application and solve the most critical points and then explore more applications and fields,” said Zhang.  “When you talk about technology, you want to make it in an appropriate way. We need the expertise form the social science domain. There is a much larger plan we have, but this project is just a starting project.”