# Meta AI Is 32% of Classified Traffic. GPTBot Is About 2%.

Robauto’s Sep 24 engine mix: Meta AI 32.1%, ChatGPT 2.3%. Meta External Agent leads visits as other_bot — use Meta’s separate robots.txt tokens, not one “AI bot” switch.

Canonical: https://robauto.ai/blog/meta-ai-32-percent-gptbot-2-percent-engine-mix

Robauto Team · Published Sep 24, 2026 · Last updated Sep 24, 2026

Robauto’s Sep 24 engine mix: Meta AI 32.1%, ChatGPT 2.3%. Meta External Agent leads visits as other_bot — use Meta’s separate robots.txt tokens, not one “AI bot” switch.

Most AEO playbooks still treat **GPTBot** as the center of gravity. Robauto’s live engine mix on September 24, 2026 says otherwise: **Meta AI** accounts for **32.1%** of classified bot visits across the network, while **ChatGPT** sits at **2.3%**. That is not a rounding error. It is a different map of who is actually reading your pages.

The headline AI share of page views is **20.89%** over 90 days (264,884 of 1,267,777 views). Inside the classified bot pool, only the `llm_crawler` class — **143,039** visits, **54%** of bot traffic — counts as AI search alongside `ai_referral` (currently **0%**). That is about **11.3%** of total page views. The rest of the headline is classic SEO crawlers, headless automation, and other non-search agents — including Meta’s own External Agent on the visit leaderboard.

## **Engine mix: Meta AI dwarfs ChatGPT and Claude**

Robauto’s engine breakdown ranks every classified visitor by share of the 264,884-visit pool. Meta AI is first. ChatGPT is eleventh. Claude is tenth. Microsoft Copilot is ninth. If your robots.txt and content budget only name OpenAI tokens, you are optimizing for a minority of the observed mix.

EngineVisits (90d classified)ShareSource

Meta AI
85,051
32.1%
[Robauto AI Search Data](https://robauto.ai/ai-search-data)

grok_bot
27,008
10.2%
[Robauto AI Search Data](https://robauto.ai/ai-search-data)

Microsoft Copilot
9,646
3.6%
[Robauto AI Search Data](https://robauto.ai/ai-search-data)

Claude
8,543
3.2%
[Robauto AI Search Data](https://robauto.ai/ai-search-data)

ChatGPT
6,090
2.3%
[Robauto AI Search Data](https://robauto.ai/ai-search-data)

Meta also shows up as smaller sibling engines in the same mix: `meta_externalads` at **1.0%** and `meta_webindexer` at **0.9%**. Those are not duplicates of “Meta AI” in the ranking above — they are separate keys in Robauto’s engine table, which matches how Meta documents distinct crawler tokens.

## **Meta’s robots.txt tokens are not one bot**

Meta’s developer docs (updated May 21, 2026) list separate crawlers with different jobs. Treating “Meta” as a single Allow/Disallow is how publishers accidentally block link previews, ads quality crawls, or Meta AI citations while still letting training traffic through — or the reverse.

TokenMeta’s stated purposerobots.txt noteSource

`Meta-WebIndexer`
Improve Meta AI search result quality; helps cite and link your content in Meta AI responses
Honored as a distinct UA group
[Meta Web Crawlers](https://developers.facebook.com/docs/sharing/webmasters/web-crawlers/)

`Meta-ExternalAgent`
Training foundation models or indexing content directly for products
Separate Disallow from WebIndexer
[Meta Web Crawlers](https://developers.facebook.com/docs/sharing/webmasters/web-crawlers/)

`Meta-ExternalAds`
Improve advertising and business products
Separate token
[Meta Web Crawlers](https://developers.facebook.com/docs/sharing/webmasters/web-crawlers/)

`Meta-ExternalFetcher`
User-requested fetches; agentic AI navigation / task completion
May bypass robots.txt
[Meta Web Crawlers](https://developers.facebook.com/docs/sharing/webmasters/web-crawlers/)

`facebookexternalhit`
Link previews when content is shared on Meta apps
May bypass robots.txt for security/integrity checks
[Meta Web Crawlers](https://developers.facebook.com/docs/sharing/webmasters/web-crawlers/)

Cloudflare’s September 15 Search / Training / Agent split is the policy-layer version of the same idea: one “AI bot” switch is too blunt. Meta’s documented tokens already give you per-purpose levers if you use them.

## **The visit leaderboard is mostly not llm_crawler**

Robauto’s public Agent Leaderboard ranks by visits, not by “how AI” a visitor is. On September 24, the top of that board is dominated by Meta External Agent (`other_bot`) and classic SEO crawlers. The first clear `llm_crawler` in the top five is Bingbot (Copilot) at rank 5. GPTBot (OpenAI) is rank **10** with **724** visits in 7 days.

RankAgentBot classVisits 7dSource

1
Meta External Agent
other_bot
10,585
[Robauto Agent Leaderboard](https://robauto.ai/agent-leaderboard)

2
AhrefsBot
search_bot
8,583
[Robauto Agent Leaderboard](https://robauto.ai/agent-leaderboard)

3
Yandex Render Bot
search_bot
6,032
[Robauto Agent Leaderboard](https://robauto.ai/agent-leaderboard)

4
SemrushBot
search_bot
3,198
[Robauto Agent Leaderboard](https://robauto.ai/agent-leaderboard)

5
Bingbot (Copilot)
llm_crawler
2,533
[Robauto Agent Leaderboard](https://robauto.ai/agent-leaderboard)

10
GPTBot (OpenAI)
llm_crawler
724
[Robauto Agent Leaderboard](https://robauto.ai/agent-leaderboard)

That does not make Ahrefs or Semrush “AI search.” It means raw visit rankings mix SEO ops traffic with model and agent traffic. Robauto’s taxonomy note is explicit: only `llm_crawler` + `ai_referral` are AI search; `search_bot` and `headless` are classic crawlers and automation.

## **What to do this week**

- **Name Meta tokens separately** in robots.txt — at minimum `meta-webindexer` (Meta AI citations) vs `meta-externalagent` (training / direct indexing). Do not assume one Disallow covers the family.

- **Stop treating GPTBot volume as your AEO north star.** On this network snapshot, ChatGPT is 2.3% of classified engines and GPTBot is 10th on 7-day visits.

- **Report two KPIs:** headline non-human share if you must, and a strict AI-search line that keeps only `llm_crawler` (+ `ai_referral`). Today that compresses **20.89%** down to about **11.3%** of page views.

- **On Cloudflare,** map Training vs Agent preferences to the Meta tokens that match each job — Disallow AI Training is not the same decision as allowing Meta AI search citations.

- **Read the leaderboard with bot_class on.** Rank 1 visits can be `other_bot`; that is still useful operations data, not proof you are “winning AEO.”

```
`{
"as_of": "2026-09-24T15:00:08.742+00:00",
"network": {
"total_page_views_90d": 1267777,
"ai_agent_views_90d": 264884,
"headline_ai_traffic_pct": 20.89,
"llm_crawler_visits": 143039,
"llm_crawler_share_of_bot_traffic_pct": 54,
"approx_llm_crawler_share_of_page_views_pct": 11.3,
"ai_referral_visits": 0
},
"engine_highlights": {
"Meta AI": {"visits": 85051, "share_pct": 32.1},
"ChatGPT": {"visits": 6090, "share_pct": 2.3},
"Claude": {"visits": 8543, "share_pct": 3.2},
"Microsoft Copilot": {"visits": 9646, "share_pct": 3.6}
},
"leaderboard_highlights_7d": {
"1_Meta_External_Agent": {"bot_class": "other_bot", "visits_7d": 10585},
"10_GPTBot_OpenAI": {"bot_class": "llm_crawler", "visits_7d": 724}
},
"sources": [
"https://robauto.ai/ai-search-data",
"https://robauto.ai/agent-leaderboard",
"https://developers.facebook.com/docs/sharing/webmasters/web-crawlers/",
"https://blog.cloudflare.com/accountable-mixed-use-ai-crawlers/"
]
}`
```

## **What this means for builders**

If your content and crawl policy were written for a GPTBot-first web, update the map. On Robauto’s September 24 snapshot, **Meta AI is 32.1%** of the classified engine mix and **ChatGPT is 2.3%**. Meta External Agent leads the visit leaderboard as `other_bot`, while GPTBot is tenth. Split Meta’s tokens the way Meta documents them, split your KPIs the way Robauto classifies them, and optimize for citation, training refusal, or live agent action — not a single bot name that no longer describes the traffic.

## Sources

- [Robauto AI Search Data](https://robauto.ai/ai-search-data)
- [Robauto Agent Leaderboard](https://robauto.ai/agent-leaderboard)
- [Meta Web Crawlers documentation](https://developers.facebook.com/docs/sharing/webmasters/web-crawlers/)
- [Cloudflare: Disallow AI Training / Search-Training-Agent controls](https://blog.cloudflare.com/accountable-mixed-use-ai-crawlers/)

## Where can an agent get this data live?

**Robauto MCP — Agent Intelligence & Brand Discovery Infrastructure.** A public, stateless Streamable HTTP endpoint at `https://mcp.robauto.ai/mcp`, callable with no key. Setup and the full tool list are in the [Robauto MCP directory](https://robauto.ai/mcp-directory).

[All blog posts](/blog)

---

(c) 2026 Robauto, Inc. — support@robauto.ai
Machine surfaces: https://robauto.ai/llms.txt · https://robauto.ai/llms-full.txt · https://robauto.ai/.well-known/api-catalog