Your AI Shopping Agent Is Less Consistent Than You Think
A single outside review shifted one AI shopping agent’s product choice by as much as 99 percentage points. New research from the Wharton School finds that the sources an AI shopping agent sees, the order in which it sees them, and even the way its tools deliver them can substantially influence what it buys. The study was conducted by Anushka Kumar, Lennart Meincke, Dan Shapiro, Lilach Mollick, Stefano Puntoni, and Ethan Mollick, with affiliations spanning the Wharton School, WHU Otto Beisheim School of Management, and Glowforge. As more purchasing decisions are delegated to AI agents, the findings raise a basic question: what is steering those choices?
Key Takeaways
- One outside review can substantially shift an agent’s go-to pick.
Without prior source material, each model had a strong baseline preference among eight fitness watches. Showing the agent one review first, including a Reddit thread, a Wirecutter roundup, or a lifestyle listicle, changed selection rates by as much as 99 percentage points in the Wirecutter condition, although the effects varied widely by model and source. - More sources do not necessarily produce a balanced choice.
When agents saw two or three sources recommending different products, their choices did not average across the recommendations. Wirecutter-containing combinations often continued to favor the Wirecutter-recommended product, and for some models that effect became stronger. - Even the order of the same information can change the outcome.
Reordering the same three sources significantly changed product-choice patterns for four of the six models. One other model showed a marginal effect, while Claude Haiku 4.5 showed no significant sensitivity to order. - A one-line hobby memory can pull agents away from the strongest visible deal.
In a modified product grid, one watch was the cheapest, highest-rated, and most reviewed. Drawing on a previous user comment from the system’s memory (“I love hiking!”), shifted four models toward a much more expensive, hiking-oriented watch; the other two did not show the same response. - Even the plumbing matters.
In the final study, four models received the same three recommendations either at once (“bundled tool call”) or in three sequential calls. All four produced significantly different choice patterns, and bundled delivery more strongly favored the Wirecutter-recommended product.
Real-World Application
Short on time? Here’s the takeaway:
AI shopping agents do not evaluate products in a vacuum. The same request can yield different results because of prior content, source order, stored user information, and implementation choices that users may not see.
Using ACES, a mock e-commerce environment, researchers ran five controlled studies in which frontier AI models selected a fitness watch across 200 runs per condition. Six models took part in the first four studies, while four took part in the tool-call study. The experiments separately varied single reviews, combinations and orderings of reviews, user-memory statements, and whether tool results arrived together or sequentially.
In one study, the grid included a watch that dominated the displayed alternatives on price, star rating, and review count. Even so, the user memory of “I love hiking!” shifted four models toward a much more expensive, hiking-oriented Garmin. The results suggest that sellers may find agentic shopping harder to predict or optimize than traditional SEO, and that companies deploying shopping agents should test for consistency.
This content was created with the assistance of generative AI. All AI-generated materials are reviewed and edited by the Wharton AI & Analytics Initiative to ensure accuracy, clarity, and alignment with our standards.
About Wharton AI & Analytics Insights
Wharton AI & Analytics Insights is a thought leadership series from the Wharton AI & Analytics Initiative. Featuring short-form videos and curated digital content, the series highlights cutting-edge faculty research and real-world business applications in artificial intelligence and analytics. Designed for corporate partners, alumni, and industry professionals, the series brings Wharton expertise to the forefront of today’s most dynamic technologies.