‘Tangible’ Display Technology Makes Virtual Objects Appear Real

‌

A. Sue Weisler

Jim Ferwerda, Ben Darling

A new way of displaying virtual surfaces could reinvent how we color our world with paint, fabric and artwork, and how professionals preview digital pictures before printing.

Developed by Jim Ferwerda and Ben Darling at Rochester Institute of Technology, “tangiBook,” a tangible display system, offers a way to interact with virtual materials. Potential applications include accessing virtual museums, soft proofing—or previewing digital prints—and flipping through digital swatch books of paint, textile or printing samples.

“The way it works is you have a model of some surface like an oil painting or fabric,” says Ferwerda, associate professor in the Munsell Color Science Laboratory in the Chester F. Carlson Center for Imaging Science at RIT. “You can render an image of this model to the screen and as you move around the screen or tilt the laptop, the pattern of highlights and reflections changes the way a real object would.”

Darling, a doctoral student in the color science program at RIT, developed the software application that drives the tangible displays. The custom software blends computer graphic rendering tools and sophisticated lighting techniques to give virtual surfaces realistic appearances. The software interfaces with the standard components of a MacBook Pro laptop computer—the display screen, webcam and an accelerometer that tells how the computer is oriented.

What makes the technology fresh is the way the appearance of the rendered surfaces changes as the viewer adjusts the device or moves around the display. To create this illusion, the software tracks the real-time orientation of the laptop screen and the position of the observer, essential logistics for making virtual objects look realistic from any angle.

“Our system renders new images fast enough that it looks like it’s happening instantaneously,” Darling says. “It takes a lot of processing. For every location on the surface, the system has to figure out how light from every direction is affecting that particular pixel.”

The tangiBook is only the first in a series of display devices that has grown to include the tangiPod—piggybacking on the iPhone—and the tangiDesk, which uses a larger screen attached to an arm. Future possibilities include creating applications for digital paint that convey a richness of real materials and developing three-dimensional tangible displays.

For now, Ferwerda and Darling are exploring how to make the monitor seem to disappear. “We have a painting displayed on the tangiBook and it’s reflecting everything around it,” Ferwerda says. “So, it looks like a painted surface, not a monitor surface. You could hang it on your wall and, as you move around, the properties change the same way the real artwork would.”

Ferwerda and Darling debuted the tangiBook last year at the Imagine RIT: Innovation and Creativity Festival Website. Since then, they have shared their research at numerous conferences around the world. Darling received the 2009 Mitsubishi Electric Research Laboratories Award for best student paper at the Imaging Sciences & Technology 17th Color Imaging Conference.

Note: Rochester Institute of Technology is internationally recognized for academic leadership in computing, engineering, imaging technology, and fine and applied arts, in addition to unparalleled support services for students with hearing loss. Nearly 16,800 full- and part-time students are enrolled in more than 200 career-oriented and professional programs at RIT, and its cooperative education program is one of the oldest and largest in the nation.

For two decades, U.S. News & World Report has ranked RIT among the nation’s leading comprehensive universities. RIT is featured in The Princeton Review’s 2010 edition of The Best 371 Colleges and in Barron’s Best Buys in Education. The Chronicle of Higher Education recognizes RIT as a “Great College to Work For.”