Parents of deaf children can more easily learn sign language thanks to powerful tech collaboration

A powerful collaboration among two universities and a tech giant is working to develop a mobile app that will enable parents of deaf children to more easily learn American Sign Language.  

The Center on Access Technology at Rochester Institute of Technology’s National Technical Institute for the Deaf, in partnership with Google and Georgia Institute of Technology, is creating PopSign, which provides an extensive, interactive learning experience that parents can use anytime, anyplace. The app displays a video of a person signing to introduce new vocabulary, and then the user’s memory is tested. To advance to the next level, the user drags and shoots a ball to hit bubbles with written words that match the sign.

The game is available on Google Play and Android devices.

“We are thrilled to collaborate with Georgia Institute of Technology and Google on this project,” said Spencer Montan, associate director of NTID’s Center on Access Technology. “Our student employees are also actively involved in re-designing the app’s user interface and user experience and making it more ‘accessibility friendly.’ We aim to expand the list of signs and the number of developers and designers to make the game more fun and available for anybody who wants to learn ASL.”

In the United States, nearly 95 percent of deaf and hard-of-hearing children have hearing parents. Most hearing parents do not know American Sign Language, which can lead to struggles and frustrations in communicating with their children. According to a study by the National Institutes of Health, language deprivation may cause complications in a child’s mental health and development and, in this case, it is recommended that parents learn ASL. However, ASL classes are often inaccessible to parents, and watching ASL videos can feel distant and unengaging. 

PopSign began as an extension of a doctoral dissertation by Kimberly Xu, a former Georgia Tech student, to help hearing parents of deaf infants learn the ASL vocabulary needed to communicate with their children.

“My students have been developing PopSign since 2016,” said Thad Starner, professor, founder, and director of the Contextual Computing Group at Georgia Tech’s College of Computing. “While we designed the game to help parents learn ASL, the game may be useful for children who want to be bilingual in ASL and English. That is a surprising angle that we are just starting to explore.

“We are working with RIT/NTID students to improve the playability, accessibility, and enjoyment of the current game. However, the next games in development, PopSign Generation and CopyCat Mobile, are being redesigned from scratch with the RIT/NTID students, at their suggestion. These games will use a smartphone’s selfie camera to recognize the player’s signing as part of the game.”

RIT/NTID CAT Lab students Ikemefuna Chukwunyerenwa, a web and mobile computing student from Houston; Jinlan Li, a professional studies student from China; and Loam Shin, a human-centered computing student from Lawrenceville, Ga., serve as user interface designers, user experience analysts, and software developers, while Georgia Tech students Colby Duke, Sahir Shahryar, and Prerna Ravi serve as project managers and data collectors, and develop the sign language recognizers that PopSign uses.

RIT/NTID alumnus Sam Sepah ’08 MS (human resource development), lead accessibility research project manager at Google, also has been involved in the project.