Freshman Imaging Project

Intelligent Multi-Camera Video Chat uses facial detection algorithms on each of four Raspberry Pi’s (breadboard computers) to choose and control one of four capture cameras. If more than one camera identifies a face, then the face that is larger in the frame will be chosen. Then, based on the location of the face in the frame, our main computer can pan each camera independently using attached servo motors. Output from the selected camera is then passed through a video switcher for display. Our imaging system has many applications including surveillance, automated lighting, and more, but we chose to demonstrate its potential use in a personal and conference video calling system.

Watch Video

The focus of the 2012-2013 freshmen imaging project was X-ray Vision with a Multi-camera Array.

A multi-camera array is an advanced imaging system that utilizes multiple camera perspectives. Multi-camera arrays have been applied to synthetic aperture imaging, high-resolution still imaging, and special effects for cinematography, such as the “Bullet Time” scene from the film "The Matrix."

Our class designed and built a prototype system that successfully demonstrates synthetic aperture imaging using multi-camera technology. The large synthetic aperture of the prototype results in a small depth of field in the image, and the different locations of the component cameras allows users to "see through" occlusions in a manner that appears similar to "x-ray vision."

Final combined synthetic aperture image. Note how subject of interest (student on couch) is visible through obstruction (student in foreground).

Multi-camera Array System Workflow

1. Cameras Point Grey Chameleon cameras are set up in order to capture frames
2. Arrangement Each camera views the calibration target from a unique angle
3. Frame Grabbers Once frames are captured by the cameras, raw data flows to computers to be compressed
4. Processor The data is then sent to a custom computer where software is run to process a single image
5. Synthetic Aperture Effect The frames are combined to produce an unobstructed image of the target

An array like ours could be used in a variety of security applications. This technology can be used to track an individual or object in a crowded or highly obstructed environment. Watch our video below (originally produced for Imagine RIT) to learn more.

The focus of the 2011 freshmen imaging project was3D Imaging for Medical Applications Using Structured Light.

What did we do and why?

The Freshman Imaging Project class was asked by Dr. Bo Hu and Dr. Jack Wojtczak from the University of Rochester Medical Center to design a craniofacial phenotyper - a 3D scanner whose purpose is to take certain measurements along the curvatore of a person's face. Dr. Hu and Dr. Wojtczak have done research which showed that certain facial measurements can help doctors determine whether or not a physician would have difficulty inserting a breathing tube into a patient prior to surgery. The goal of the Freshman Imaging Project class was to create a craniofacial phenotyper which quickly, accurately, and inexpensively provides a physician with the data which would enable that assessment.

What is a structured light scanner, and how does it work?

This system uses Structured Light technology in order to gather 3D data. Several patterns of alternating dark and light bars with different spatial frequencies and orientations are projected onto the subject. Two cameras take pictures of the patterns, and the system computes the deviation of the bars with respect to a flat reference to determine depth information, The system renders a 3D point cloud that is used for visualization and allows the physician to obtain specific measurements.

The focus of the 2010 freshmen imaging project was a Polynomial Texture Mapping (PTM) device.

What are PTMs?
PTMs are a type of interactive digital image that allow users to view an object from an infinite number of light source angles. This allows the user to uncover hidden textures, blemishes, and other surface features not visible using traditional photographic techniques. PTMs have uses in historical document and artifact imaging, forensics, dermatology, and more. 

How do you create a PTM?
PTMs are created by taking many photographs of a static object from a fixed position using varying light angles. The individual image files are then run through software which models the luminance values at each pixel in the image and generates the final interactive PTM image. While this may sound like a straight forward process, there is no such thing as an "instruction manual" or assembly kit.

What did the IFE2010 students do?
The IFE students reached a number of milestones throughout the 2010-2011 academic year:

• Research PTMs: how they work, how to make them, what to use them for
• Design a robust system to capture images at multiple light angles, determining:
  • Structure - How to construct a dome or other setup to hold lights at different angles
  • Illumination - What type of lights to use, considering brightness, temperature, color, etc.
  • Capture (camera) - What type of camera, lens, and capture settings to use
  • Electronics - How to wire up and control the lighting and camera systems
  • Software - How to automate the system and process the imagery
• Construct the PTM system
• Demonstrate the PTM system at ImagineRIT
• Perform actual research - The PTM system and 4 IFE students traveled to the Boston Public Library just after the conclusion of the academic year. 

Check out the videos below made by some IFE2010 students!

Watch Video: What are PTMs?