AI chatbots are recommending phishing shopping sites to users

A study by Podqi has shown that popular chatbots, including ChatGPT, Gemini, and Perplexity, may direct users to fraudulent or suspicious websites when searching for products. During testing, the neural networks provided links to resources mimicking well-known brands, which poses serious risks to the data security and finances of shoppers.

The authors of the study analyzed the models' responses to queries related to purchasing popular items. In a number of cases, AI agents included links to sites that have no relation to official stores but use similar designs or domain names to mislead consumers. The problem is exacerbated by the fact that users tend to trust chatbot recommendations, perceiving them as verified sources of information.

The study's results highlight the vulnerability of current AI-based search systems to manipulation by the owners of malicious resources. Since the models are trained on data from the open internet, they may unintentionally index and suggest phishing sites to users that have successfully passed search engine optimization or were mistakenly identified by algorithms as legitimate stores.

For regular users, this means they need to double-check links provided by AI assistants before entering payment details. Chatbot developers will need to implement stricter source filtering mechanisms to prevent fraudulent sites from appearing in recommendations, especially in the context of commercial queries.