RIT researchers use AI to uncover surprising trends in media coverage of police

A new study from Rochester Institute of Technology and Carnegie Mellon University experts is challenging a widely held belief about the media—that local news outlets have become more critical of the police in recent years.

The study, published in Proceedings of the National Academy of Sciences (PNAS), used advanced artificial intelligence (AI) techniques to analyze a decade’s worth of local news reporting on policing across the United States. The team applied large language models to a dataset of more than 250,000 news articles from 10 politically diverse cities.

The researchers found that criticism of the police in local media has remained largely stable from 2013 to 2023. There was also little difference between reporting in conservative and liberal cities—meaning local outlets have not tailored their reporting on the police to the politics of their audience.

While spikes in police criticism did occur after high-profile incidents, such as the murder of George Floyd in 2020, the study found that these increases were temporary and did not result in a long-term shift toward more critical reporting. In fact, since 2020, local reporting supporting the idea that police are effective has slightly increased.

“The data told a different story than we expected,” said Ashique KhudaBukhsh, an assistant professor in RIT’s Golisano College of Computing and Information Sciences and co-author of the study. “Local journalism on policing has been stable, but how people read the news depends on their political preference. The polarization is less on the news writers and more about the readers.”

Perception vs reality in news consumption

This study builds on previous research by the team, which examined how major national news networks—MSNBC, Fox, and CNN—covered police issues. That paper found that cable news tends to steer coverage to align with audience expectations, creating more partisan narratives.

For this study, the researchers started by conducting a survey. They found that a majority of people—regardless of political affiliation—believe that news reporting on the police has become more critical over time. However, the researchers saw hardly any empirical evidence on how coverage of the police has changed over time.

“Our goal was to look at a broad time horizon—10 years—instead of focusing on a single moment in history,” said KhudaBukhsh.

‌

Provided

The plot shows how readers in different cities view news stories about police. Democratic-leaning cities included Houston, Denver, Tampa, Nashville, and Pittsburgh. Republican-leaning cities included Dallas-Fort Worth, San Diego, Jacksonville, Oklahoma City, and Omaha.

To ensure balance, the study gathered news from 209 outlets in comparable cities—five in Republican-leaning areas and five in Democratic-leaning areas. Researchers applied natural language processing methods to analyze 1.3 million excerpts from local newspapers and TV stations.

A key element was ensuring that diverse perspectives were incorporated into the AI system.

About 500 real Republicans, Independents, and Democrats read a sampling of police news excerpts to evaluate whether coverage supported or contradicted two key statements—“The police protect citizens” and “The police are racist.” Those human judgments helped train and fine-tune the large language models that measured the full police news data set.

“When building AI systems, it’s critical to include a diversity of viewpoints,” KhudaBukhsh explained. “If we only allow one political perspective to annotate the data, we risk creating an unbalanced evaluation with bias. Every data point in our annotation study was reviewed by a Republican, a Democrat, and an Independent.”

This collaboration between RIT and Carnegie Mellon highlights how AI can be used to study large-scale media trends.

‌

Provided

The researchers trained an AI model to predict how different readers would evaluate news sentences about police. The plot shows the percentage of sentences—each month—that were predicted as supporting and contradicting the hypotheses, “Police protect us” (left) and “Police are racist” (right). A few high-profile news stories are identified by spikes on the plot.

“This is the best time to do this kind of research because the AI landscape has changed and we have the tools and data to ask deeper questions,” said Sujan Dutta, a computing and information sciences Ph.D. student at RIT and co-author. “The biggest challenge was introducing the notion of political perspectives into AI and making a very comprehensive and balanced design that is representative of different viewpoints.”

The study’s findings suggest that local journalists covering local issues don’t fall prey to the type of partisanship seen at the national level. The researchers hope their work offers a clearer, data-driven understanding of how policing is covered in local news.

“Media informs a great deal about our current state of the world,” said KhudaBukhsh. “By better understanding how sources cover the news and people consume it, we can work toward bridging the polarizing gap that we have today.”