Leading AI assistants misrepresented or mishandled news content in nearly half of evaluated answers, according to a European Broadcasting Union (EBU) and BBC study.
The research assessed free/consumer versions of ChatGPT, Copilot, Gemini, and Perplexity, answering news questions in 14 languages across 22 public-service media organizations in 18 countries.
The EBU said in announcing the findings:
“AI’s systemic distortion of news is consistent across languages and territories.”
What The Study Found
In total, 2,709 core responses were evaluated, with qualitative examples also drawn from custom questions.
Overall, 45% of responses contained at least one significant issue, and 81% had some issue. Sourcing was the most common problem area, affecting 31% of responses at a significant level.
How Each Assistant Performed
Performance varied by platform. Google Gemini showed the most issues: 76% of its responses contained significant problems, driven by 72% with sourcing issues.
The other assistants were at or below 37% for major issues overall and below 25% for sourcing issues.
Examples Of Errors
Accuracy problems included outdated or incorrect information.
For instance, several assistants identified Pope Francis as the current Pope in late May, despite his death in April, and Gemini incorrectly characterized changes to laws on disposable vapes.
Methodology Notes
Participants generated responses between May 24 and June 10, using a shared set of 30 core questions plus optional local questions.
The study focused on the free/consumer versions of each assistant to reflect typical usage.
Many organizations had technical blocks that normally restrict assistant access to their content. Those blocks were removed for the response-generation period and reinstated afterward.
Why This Matters
When using AI assistants for research or content planning, these findings reinforce the need to verify claims against original sources.
As a publication, this could impact how your content is represented in AI answers. The high rate of errors increases the risk of misattributed or unsupported statements appearing in summaries that cite your content.
Looking Ahead
The EBU and BBC published a News Integrity in AI Assistants Toolkit alongside the report, offering guidance for technology companies, media organizations, and researchers.
Reuters reports the EBU’s view that growing reliance on assistants for news could undermine public trust.
As EBU Media Director Jean Philip De Tender put it:
“When people don’t know what to trust, they end up trusting nothing at all, and that can deter democratic participation.”
Featured Image: Naumova Marina/Shutterstock
