AI shopping assistants are popping up all over the internet, changing how we browse, compare and discover products. However, these helpful tools appear to have a serious security flaw. According to a paper published on the arXiv preprint server, a single manipulated web page can trick an AI assistant into promoting a fake product to unsuspecting customers.
Considering that fake goods and fake reviews are everywhere online, researchers Minghao Luo and Liang Chen decided to test how easily search-augmented AI systems can be tricked into promoting bogus brands.
AI testing ground
The researchers built a simulation tool called FORGE (Fake Online Recommendations in Generative Environments) to test 12 leading AI models, including models from Anthropic, Google and OpenAI. This allowed them to evaluate web content pollution without interfering with live pages.
They took real search results (the top web pages that pop up when you look for shopping recommendations online), identified the main brand being discussed on selected pages and swapped it for a fake one. They did this for 225 products spanning 15 categories, including apparel, supplements and digital electronics.
After rewriting these pages, they tested whether LLMs would fall for the deception and include a fake brand in their recommendations.
The answer was an unequivocal yes.
“Across 12 commercial and open-weight LLMs, all models are vulnerable: a single polluted page yields fooled rates of up to 27%, while the full top-3 replacement raises this to 73.8%,” Luo and Chen wrote in their paper.
So, just one fake page was enough to trick certain AIs more than a quarter of the time. And when the top three search results were manipulated, the models bought into the scam nearly three-quarters of the time.
In some cases, the models went even further, inventing positive comments about the fake brands, such as claiming they were popular in online communities.
Testing the defenses
The researchers also tested three defenses to see if they could stop the AI from falling for fake web content. These were skepticism, which tells chatbots to be highly doubtful of what they read; model-prior consensus, which forces AI to check recommendations against its own memory; and cross-document agreement, which requires AI to find the same brand on multiple websites before trusting it.
All three failed or caused new problems, as Luo and Chen noted in their paper. “Simple defenses are not enough. Skepticism prompting can backfire, while consensus-based filtering catches fake brands only by suppressing many legitimate recommendations.”
So what is the solution? The researchers argue that the fix cannot just happen at the chatbot level. Instead, search-augmented AI systems need stronger safeguards to verify the trustworthiness of web content before turning it into product recommendations.
Written for you by our author Paul Arnold, edited by Gaby Clark, and fact-checked and reviewed by Robert Egan—this article is the result of careful human work. We rely on readers like you to keep independent science journalism alive.
If this reporting matters to you, please consider a donation (especially monthly). You’ll get an ad-free account as a thank-you.
Publication details
Minghao Luo et al, One Polluted Page Is Enough: Evaluating Web Content Pollution in Generative Recommenders, arXiv (2026). DOI: 10.48550/arxiv.2606.13610
Journal information:
arXiv
The content is provided for information purposes only.