Researchers assessed the impact of dialogue history on brand recall in LLM

Researchers conducted a series of 45 runs in ChatGPT, Perplexity and Alice AI to determine the dependence of AI Visibility on the conversation context. The experiment showed that the same final query, asked after a different dialogue history, leads to significant differences in the sets of mentioned brands.

During testing, it was analyzed how the previous user and model replies affect the final selection of companies in the responses. The results confirm that independent prompts used to evaluate AI Visibility do not provide a complete picture of the behavior of models in real communication scenarios. The variation in brand output for an identical final question reaches significant levels, which raises questions about the accuracy of current methods for measuring the impact of AI on consumer choice.

The study's data highlights the need to revise approaches to testing information search and recommendation models. Instead of evaluating isolated queries, the authors suggest considering the dynamics of the contextual window, as it is the interaction history that determines the priority of the entities being output. This is critical for companies using AI tools for marketing research and analyzing brand market visibility.

The results obtained indicate that the ranking algorithms in modern chatbots are extremely sensitive to semantic surroundings. The stability of responses decreases with the complication of the dialogue structure, which requires the introduction of new verification standards for systems working with recommendation content.