RIT researchers developing image manipulation detector

‌

RIT is part of a select group of world-class researchers studying the fact and fiction of imagery in today’s digital culture by developing an algorithmic-based platform that can detect image manipulation.

Image alteration is a complex issue that government officials want to solve with a program known as MediFor—short for media forensics. Media forensics is the scientific analysis of recorded media in the form of audio, video and still-imagery evidence obtained during the course of investigations and litigious proceedings.

The researchers’ goal for the MediFor project is to automate the detection of image manipulations, provide detailed information about how these manipulations were performed, and determine the overall integrity of visual media.

 

Can you tell which image is real and which has been manipulated?

“It’s a fairly widespread issue that the government wants to get ahead of with this three-year project,” said Christye Sisson, the Ronald and Mabel Francis Professor and chair of the photographic sciences program in RIT’s School of Photographic Arts and Sciences. Sisson is RIT’s principal investigator for the project.

“The more mature digital imaging has become, the methods by which you can manipulate images become increasingly sophisticated,” she added. “The speed at which this is happening is outpacing the tools that people have for detection, so we’re trying to exhaust and analyze every possibility for how these images could potentially be manipulated and how that manipulation can be detected.”

Research teams include imaging experts from academia, industry and government agencies, including the National Institute of Standards and Technology (NIST). According to Sisson, RIT is creating high-provenance data and image manipulations as part of the project’s data team, led by PAR Government Systems Corp., a government contractor based in Rome, N.Y.

Sisson noted that applications of the research’s findings are widespread—from law enforcement and intelligence agencies to counter-terrorism and academic research integrity.

A growing problem

While some image manipulation is benign, performed for fun or artistic value, other “doctoring” is done for nefarious or adversarial purposes, including propaganda or misinformation campaigns, Sisson said.

In recent years, consumer imaging technology—primarily digital cameras and mobile phones—has become ubiquitous, enabling people the world over to take and share images and video instantaneously. Mirroring this rise in digital imagery, however, has been the ability for even relatively unskilled users to manipulate and distort visual media.

The manipulation of visual media is enabled by the wide-scale availability of sophisticated image- and video-editing applications that falsify images in ways that are extremely difficult to detect—either visually or with current image analysis and visual media forensics tools. Many of the forensic tools used today lack sophistication and address only some aspects of media authentication.

The MediFor project’s goal is to develop an “end-to-end platform” that will perform a complete and automated forensic analysis of imagery that does not exist today.

According to Sisson, RIT is creating the ground-truth data for the project by photographing a large variety of scenes and contexts with as many devices as possible to create a wide frame of reference for government officials. The RIT team uses these images to create manipulations based on real scenarios to test new algorithms of detection being developed by other partner universities on the MediFor project.

Along with these manipulations, Sisson said, the RIT team will create an “answer key” that will chronicle all actions made in a manipulation to see how well the algorithms perform in their evaluation.

RIT professors Ted Kinsman and Christopher Kanan are assisting Sisson on the project, along with several undergraduate students in the photo sciences, imaging science, motion picture science, and media arts and technology programs.

“RIT has this incredible and unparalled imaging continuum with our programs that you just don’t see anywhere else,” Sisson said. “We’re extremely well-suited for this project.”