Agent Exchange x402 census and directory

x402 census · x402.agentharbor.agency · cf5ef7301c07 · JSON

GET /ai-crawlers

live

Given a website domain, reports which AI crawlers may fetch its home page under its robots.txt (RFC 9309): GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, PerplexityBot, Google-Extended, Applebot-Extended, CCBot and more, each allowed or blocked, plus Googlebot, listed sitemaps, whether /llms.txt is published and page-level noindex/nosnippet, with fixes. Static check, no browser…

Verdict: live. The unpaid GET reached a valid 402 (confirmed by the re-probe of 2026-11-11). The live amount equals the catalog price.

Facts from the catalogs

FieldValue
URLhttps://x402.agentharbor.agency/ai-crawlers
MethodGET
Price$0.01 USDC = 10000 atomic units of 0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913
Networkeip155:8453 (base)
payTo0x840C829530DF7B4CCF700Fa3A0060cF2C80b014e
Sellerx402.agentharbor.agency
Catalogscdp (row updated 2026-10-25)
Listed since (lower bound)2026-10-25 (earliest catalog timestamp; catalogs report last-update times only)
Last catalog update2026-10-25
CDP quality counters1 calls and 1 unique payers in 30 days, last call 2026-10-25 (catalog-reported, not verified on chain)
Category (keyword rule)inference/llm
Templated pathno
x402Version in catalog2
Service name / tagsah-ai-crawlers · ai-crawlers, robots-txt, llms-txt, gptbot, seo

Unpaid probe (census of 2026-10-22)

FieldValue
Probed at2026-10-22T00:30:10.994Z
HEAD405
GET402
Verdictlive live: valid 402 to an unpaid GET
402 carried inheader
x402Version in the 4022
accepts[0]{"scheme":"exact","network":"eip155:8453","amount":"10000","payTo":"0x840C829530DF7B4CCF700Fa3A0060cF2C80b014e","asset":"0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913","maxTimeoutSeconds":300,"extra":{"name":"USD Coin","version":"2"}}
Live amount equals catalog priceyes
Live payTo equals catalog payToyes
extensions.bazaar in the 402yes
PAYMENT-REQUIRED headeryes
WWW-Authenticate offerPayment id="1A5yQt7BQyDXjS3RaxMsQGNfElpZ
Content type / server / CORSapplication/json / cloudflare / none
Latency168 ms
ErrorPayment required

Latest re-probe

FieldValue
Re-probed at2026-11-11T19:13:26.064Z (cron)
Verdict nowlive unchanged
HEAD / GET402 / 402
accepts[0]{"scheme":"exact","network":"eip155:8453","amount":"10000","payTo":"0x840C829530DF7B4CCF700Fa3A0060cF2C80b014e","asset":"0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913","maxTimeoutSeconds":300,"extra":{"name":"USD Coin","version":"2"}}
Live amount equals catalog priceyes
Latency958 ms
Errornone

On-chain facts for the payTo (public address)

FieldValue
Settlement historyunresolved: payTo not seen in the payer forensics (180 wallets' outbound history) or the Blockscout sample

Declared input and output (extensions.bazaar)

{
 "input": {
  "method": "GET",
  "queryParams": {
   "domain": "agentharbor.agency"
  },
  "type": "http"
 },
 "output": {
  "type": "json",
  "example_truncated": true,
  "chars": 951
 }
}

Try it (unpaid: shows the 402)

curl -si -X GET 'https://x402.agentharbor.agency/ai-crawlers' -H 'accept: application/json'

The catalog declares GET. A 402 answer carries the payment requirements in the JSON body and, for x402 v2, base64 in the PAYMENT-REQUIRED header. This page never sends a payment.

Paid checks (x402, USDC on Base)

GET https://bazaar.agentexchange.work/r/cf5ef7301c07/probe.json re-runs this probe right now for $0.01 and writes the result here. POST https://bazaar.agentexchange.work/featured with {"id":"cf5ef7301c07"} places this resource at the top of the report and search pages for 30 days for $1.00, labelled. Unpaid requests answer 402 with the terms. Pricing.

Catalog references: agentic.market (CDP Bazaar front end) · this record as JSON · all resources on x402.agentharbor.agency.