Finding 01 — The scan result, which is the boring part.
Five mid-size suppliers, six procurement questions each, four engines: 2 recommendations in 120 checks, and their own websites cited twice. By engine: ChatGPT 0 of 30, Google AI Overviews 0 of 30, Claude 0 of 30, Perplexity 2 of 30.
The questions themselves were answered confidently — with the biggest name in each category. Ask about freight forwarding for an SME and you get DHL, FedEx and UPS (15 mentions each). Ask about membrane filtration and you get Toray (8). Bicycle components: KMC, Giant, Merida (4 each). Across all five categories the single most-cited source was Wikipedia, 36 times — ahead of every supplier, directory and trade publication.
Finding 02 — Three of the five sites refuse AI crawlers. Their robots.txt says the opposite.
We requested each supplier's homepage repeatedly from one machine, alternating only the user-agent string: a normal Chrome string, then GPTBot, then ClaudeBot, spaced a few seconds apart.
| Site | Browser refused | GPTBot refused | ClaudeBot refused |
|---|---|---|---|
| Bicycle components supplier | 0 / 6 | 5 / 6 | 2 / 6 |
| Freight forwarder | 0 / 6 | 1 / 6 | 6 / 6 |
| Fleet monitoring vendor | 0 / 6 | 4 / 6 | 4 / 6 |
| Control: two suppliers AI cites | 0 / 12 | 0 / 12 | 0 / 12 |
The refusals come back as 429 Too Many Requests — from the same IP address that had just been served the full page as a browser, seconds earlier. The three affected sites are served by the same CDN, operated by the agency that built them.
The part that makes this a bug rather than a policy: all three publish a robots.txt that allows crawlers, disallowing only two internal paths. Someone deliberately opened the door in the file everyone checks, while the infrastructure in front of it was closing the door based on the user agent.
Finding 03 — Sites that let crawlers in get read, and occasionally recommended.
We scanned three comparison suppliers — same size class, same industries, chosen because AI had cited them while answering questions about the first group.
| Group | Own domain cited | Recommended |
|---|---|---|
| Sites that refuse AI crawlers | 0–1 times | 0–2 of 24 |
| Sites that serve them normally | 4–17 times | 1–3 of 24 |
Note what this does and does not show. Serving crawlers does not make a mid-size supplier well-recommended — the comparison group is still mostly absent, and still loses to Toray and DHL. What it does is put the company in the corpus at all. Refusing crawlers removes the possibility.
Finding 04 — A third of B2B answers cite nothing at all.
40 of the 120 answers arrived with no sources attached — 33%, against 16% in our 13-brand consumer study. B2B questions send engines to memory twice as often, and memory is exactly where a mid-size supplier has never been written in.
Method, and what this does not show
Five Taiwanese B2B suppliers, six procurement questions each, four engines, one pass per cell: 120 scored answers, 13–14 September 2026. Questions were generated from each company's own homepage and checked by hand against what the company actually sells before scanning. The crawler test: 6 rounds per site per user agent, requests interleaved from a single IP so that any shared rate limit would have hit the browser requests too; it did not.
Five limits worth stating plainly. One: five suppliers plus three comparisons is a small, non-random sample drawn from two agencies' public case pages. Two: a 429 is a rate-limit response, not an explicit block — we can observe the behaviour, not the configuration behind it, and we did not contact the operators before publishing. Three: the behaviour is intermittent, not absolute: one site refused GPTBot 4 of 6 times in one run and 0 of 6 in another, and PerplexityBot was usually served. Four: we tested from one network in Taiwan; a crawler arriving from OpenAI's or Anthropic's ranges may be treated differently, in either direction. Five: single-pass scans cannot separate a real gap from run-to-run drift — rescanning one supplier a day later moved it from 0 recommendations to 2.
What we are confident in: on repeated interleaved requests, three supplier sites served a browser every time and refused AI-crawler user agents most of the time, while the comparison sites served everyone. What we are not claiming: that this is the whole explanation for those suppliers' absence, or that the agency did it deliberately. The robots.txt files suggest the opposite of intent.
Check the boring thing first.
Paste your site — one real buyer question across five engines, free, no email. If the answer is that AI has never seen you, it is worth knowing whether it was ever allowed to look.