Artificial intelligence and Google Street View could hold the key to stopping invasive plants

Invasive plants can quickly damage ecosystems, cause economic hardships for farmers and pose health risks to the public, but regulators tasked with halting their progress often have limited resources. So, scientists at Rochester Institute of Technology are turning to street view imagery and artificial intelligence to make tracking the spread of invasive plant species along roads more manageable.

The New York State Department of Environmental Conservation will award two RIT faculty a grant to map roadside infestations of five key invasive plant species in the Finger Lakes and Adirondack Park over the next two years. Assistant Professor Christopher Kanan and Associate Professor Christy Tyler will develop algorithms that analyze high-resolution images from Google Street View and other sources to help environmental officials identify where the species have spread and prioritize where to intervene. They believe this new approach will be a big improvement over current tracking methods, which rely on field reports and volunteer crowd-sourcing.

“There are limitations to crowd-sourcing,” said Kanan. “Rural roads are hard to get to and hard to monitor and often you can’t even pull over to the side of the road easily. If we can figure out where these species are, using artificial intelligence, organizations like the Department of Environmental Conservation can better allocate their resources toward managing these plants.”

The scientists have already developed working models for two species—common reed (Phragmites) and Japanese knotweed—which they will further fine-tune. They will also develop new models for identifying giant hogweed, tree-of-heaven and purple loosestrife. These plants are some of the most problematic species in New York because they quickly crowd out native plants, are hazardous to humans or host insects that wreak havoc on crops.

“We started with two target species that are pretty obvious, noxious, public enemy number one-type species that are common on roadways,” said Tyler. “We picked them because they are priority species and distinctive looking plants that can be picked up easily by computers. Moving forward, tree-of-heaven is an important one because it’s the host of the spotted lantern fly, a new invasive insect that has huge potential for crop damage, especially tree fruit including apples, pears, peaches and grapes. That will be a bigger challenge because it looks a lot like other species.”

Once the computer models are functional and begin making predictions about where the invasive species are located, Tyler’s environmental science students will verify the identification both in the images and in the field. Kanan and Tyler said the hardest part is training the computers to accurately identify the correct plants. Students in the environmental science senior capstone class, with Visiting Assistant Professor Kaitlin Stack Whitney, have already helped with this verification process. Ultimately, the results will be communicated to managers through the New York State Partnership for Invasive Species Management (PRISM).

“Most of the big successes in computer vision use deep neural networks with images that are fairly small. Google Street View images have over 1,000 times more pixels than the 224x224 images typically used. Standard algorithms totally break down in this situation. We have developed efficient methods for detecting regions of these high-resolution images that are likely to contain plant matter and then we determine if an invasive plant is present in the scene. For needle in the haystack type problems, this multi-stage process seems to be very successful.”

The project is funded by the New York State Department of Environmental Conversation’s Invasive Species Grant Program, designed to support projects that target both aquatic and terrestrial invasive species. DEC received 96 applications and awarded approximately $2.8 million from the New York State Environmental Protection Fund to 42 projects.