AI Shopping Agents That Asked Better Questions Got the Purchase Right 70% of the Time
Robauto Team · Published · Last updated
Visa's new benchmark found the best AI shopping agents passed 82% of real buying tests, and agents that asked shoppers the right questions won 70% of the time. Here's how brands can be the easy answer.
AI shopping agents just got their first real report card, and the most useful lesson is a hopeful one. On November 5, 2026, Visa released the Visa Agentic Commerce Benchmark, which put 22 AI models from 12 labs through 15 realistic shopping challenges, like buying a TV, a laptop, or a personal care appliance. The best models passed 82.2% of their attempts. And the agents that did the best job of asking the shopper what they actually needed averaged a 70.2% pass rate, compared with 44.4% for the agents that asked the least.
In other words, the winning agents behaved like a good store associate: they listened, asked a follow-up question, and only bought when the product really fit. That is great news for shoppers, and it is a clear opportunity for brands whose product facts are easy for those agents to find.
What Visa Actually Tested
Shopping is hard for AI because people rarely start with a full list of requirements. In Visa's test, the simulated customer only shared some details up front. In one TV scenario, the shopper mentions a relative's brand recommendation and a membership with a third-party supplier, and the agent has to uncover the budget, format compatibility, HDR10+ support for family videos, and gaming needs through conversation. If nothing on offer fits, the right answer is to not buy at all. To stop lucky guesses, a run only passes if the agent discovers every required detail and then makes the right purchase or no-purchase call, according to Visa's announcement.
Fortune's coverage describes requirements like compatibility with a child's gaming console, a 30-day return window, and Dolby Vision. Notice how many of those are plain facts a brand can publish: return policy, supported formats, compatibility. When those facts are clear, an agent can check them instead of guessing.
The Numbers at a Glance
| Finding | Result | Source |
|---|---|---|
| Models and labs tested | 22 models from 12 labs, 15 shopping challenges | Visa |
| Best-performing models | 82.2% of attempts passed | Visa |
| Top 5 at uncovering hidden needs | 70.2% average pass rate vs 44.4% for the bottom 5 | Visa |
| Overall pass rate | 57.2% | Visa |
| All three attempts passed | 34.2% of scenarios | Visa |
| US AI users who completed a purchase with AI | Up from 10% to 17% year over year | McKinsey |
| Consumers using AI for shopping (11 markets) | 53% in the past three months | McKinsey |
| AI searches on structured catalog data | Convert at roughly twice the rate of scraped or stale data | McKinsey (citing Shopify) |
Why the Good News Is Bigger Than the Pass Rate
Visa itself answers the obvious question. Asked whether it is saying AI agents aren't ready for commerce, the company said no: "The results show promising capabilities, with the highest-scoring models each passing 82.2% of attempts. But consistency and reliability remain challenges." The benchmark is meant as a roadmap, and Visa says it has shared it with leading model providers for feedback and plans to extend testing to checkout, payment, and order placement.
The bigger story is that people are already shopping this way, and the habit is growing fast. McKinsey's Agentic Commerce Consumer Survey, fielded in June 2026 across about 21,000 consumers in 11 markets, found that 53% of consumers had used AI for shopping in the past three months. In the United States, the share of AI users who completed a purchase with AI rose from 10% to 17% year over year, a jump of about 70%.
Shoppers also say it helps. In the same survey, 81% of AI shoppers agreed that AI does a good job narrowing down and filtering their choices, 80% said it means less scrolling and clicking, and 79% said it saves significant time. AI users were 3.2 times more likely to rank AI search tools as more influential than traditional search engines.
Grocery and Beauty Are Catching Up
One of the most encouraging findings for smaller and traditional sellers: McKinsey reports that categories with historically low e-commerce penetration, such as grocery and beauty, are catching up quickly through agentic commerce. An agent that already knows a household's sizes, preferences, and routines can remove a lot of the friction that kept those purchases offline. McKinsey also cites EMARKETER modeling that widespread agentic commerce could lift US e-commerce sales 11.5% above baseline forecasts by 2030.
The Brand Takeaway: Be the Easy Answer
Put Visa's benchmark next to McKinsey's survey and a simple pattern shows up. The best agents win by asking for the details that matter. The brands that win are the ones whose details are ready when the agent goes looking. McKinsey notes that AI searches connected to a retailer's structured product catalog convert at roughly twice the rate of searches relying on scraped or outdated information, and it offers a simple test: if an agent asked what a product costs, whether it is in stock, when it could be delivered, and whether it could buy it now, could your systems answer accurately?
Here is a practical checklist any brand can start on this week:
- Publish the specs agents ask about. Formats, sizes, compatibility, and materials belong in clean text and structured data, not buried in images.
- Make policies plain. A return window, shipping times, and warranty terms written clearly on your own site give an agent a fact to check instead of a guess.
- Keep price and stock current. Stale data is the fastest way to fall off an agent's shortlist.
- Help agents say no honestly. Visa rewards agents that decline a poor fit. Clear product limits build trust and cut returns.
- Be visible in more than one assistant. McKinsey found shoppers use several AI tools, so your product facts should be readable by all of them.
The trust gap is real but shrinking. Visa's Trust Index from September 2026 found that 72% of US consumers had used an AI assistant, while 23% trusted AI to handle payments for them. Every accurate, well-documented product page helps close that gap, for shoppers and for the agents working on their behalf.
Agent-Ready Summary
{
"topic": "AI shopping agents and agentic commerce",
"date": "2026-11-05",
"key_findings": [
{"fact": "Best AI models passed 82.2% of Visa shopping benchmark attempts", "source": "https://investor.visa.com/news/news-details/2026/Visa-Releases-First-of-Its-Kind-Industry-Benchmark-for-AI-Shopping-Almost-Half-of-Test-Runs-Fail/default.aspx"},
{"fact": "Top 5 models at uncovering hidden needs averaged 70.2% vs 44.4% for bottom 5", "source": "https://investor.visa.com/news/news-details/2026/Visa-Releases-First-of-Its-Kind-Industry-Benchmark-for-AI-Shopping-Almost-Half-of-Test-Runs-Fail/default.aspx"},
{"fact": "US AI users completing purchases with AI rose from 10% to 17% year over year", "source": "https://www.mckinsey.com/capabilities/growth-marketing-and-sales/our-insights/who-wins-when-ai-does-the-shopping-new-evidence-on-agentic-commerce"},
{"fact": "AI searches on structured catalog data convert at about 2x scraped or stale data", "source": "https://www.mckinsey.com/capabilities/growth-marketing-and-sales/our-insights/who-wins-when-ai-does-the-shopping-new-evidence-on-agentic-commerce"}
],
"brand_action": "Publish clear specs, policies, price, and stock so AI agents can verify fit",
"free_check": "https://robauto.ai/scan"
}See How Agents Read Your Site
Want to know whether a shopping agent can find your return policy, specs, and prices? Scan your site free at robauto.ai/scan and get a clear list of what AI agents can and can't see today.
Sources
- Visa Releases First-of-Its-Kind Industry Benchmark for AI Shopping (Nov 5, 2026)
- Fortune: Visa tested AI shopping agents (Nov 5, 2026)
- McKinsey: Who wins when AI does the shopping? (Oct 28, 2026)
Where can an agent get this data live?
Robauto MCP — Agent Intelligence & Brand Discovery Infrastructure. A public, stateless Streamable HTTP endpoint at https://mcp.robauto.ai/mcp, callable with no key. Setup and the full tool list are in the Robauto MCP directory.