Agent Exchange x402 census and directory

x402 census · api.crawlwitness.com · 23d34b52f25b · JSON

GET /v1/can-use

live

Use when a crawler, scraper or RAG pipeline must decide whether it may use a website for AI: yes/no verdict for a domain, AI bot and use (train, rag, search) from robots.txt, IETF AIPREF, TDMRep, noai and RSL licenses, with a signed receipt. Errors: 400 invalid domain or unknown bot (not charged).

Verdict: live. The unpaid GET reached a valid 402. The live amount equals the catalog price.

Facts from the catalogs

FieldValue
URLhttps://api.crawlwitness.com/v1/can-use
MethodGET
Price$0.001 USDC = 1000 atomic units of 0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913
Networkeip155:8453 (base); also accepts solana:5eykt4UsFv8P8NJdTREpY1vzqKqZKvdp
payTo0xa20F0b8ADf20Ed17a8d27B369B3058afdB71a5C5
Sellerapi.crawlwitness.com
Catalogscdp (row updated 2026-11-18), payai (row updated 2026-11-12)
Listed since (lower bound)2026-11-12 (earliest catalog timestamp; catalogs report last-update times only)
Last catalog update2026-11-18
CDP quality counters38 calls and 12 unique payers in 30 days, last call 2026-11-17 (catalog-reported, not verified on chain)
Category (keyword rule)trading-signals
Templated pathno
x402Version in catalog2
Service name / tagsCrawlWitness · ai-consent, robots-txt, crawler-compliance, data-licensing, batch

Unpaid probe (census of 2026-11-18)

FieldValue
Probed at2026-11-18T02:08:12.026Z
HEAD402
GET402
Verdictlive live: valid 402 to an unpaid GET
402 carried inheader+body
x402Version in the 4022
accepts[0]{"scheme":"exact","network":"eip155:8453","amount":"1000","payTo":"0xa20F0b8ADf20Ed17a8d27B369B3058afdB71a5C5","asset":"0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913","maxTimeoutSeconds":300,"extra":{"name":"USD Coin","version":"2"}}
Live amount equals catalog priceyes
Live payTo equals catalog payToyes
extensions.bazaar in the 402yes
PAYMENT-REQUIRED headeryes
WWW-Authenticate offernone
Content type / server / CORSapplication/json / n/a / none
Latency579 ms
Errornone

No hourly re-probe has reached this resource yet; the cron covers a rotating slice of 150 per hour.

On-chain facts for the payTo (public address)

FieldValue
Settlement historyunresolved: payTo not seen in the payer forensics (180 wallets' outbound history) or the Blockscout sample

Declared input and output (extensions.bazaar)

{
 "input": {
  "method": "GET",
  "queryParams": {
   "bot": "GPTBot",
   "domain": "nytimes.com",
   "use": "train"
  },
  "type": "http"
 },
 "output": {
  "type": "json",
  "example": {
   "allowed": false,
   "bot": "GPTBot",
   "bot_role": "training",
   "decision": "not_allowed",
   "domain": "nytimes.com",
   "method": "can-use/1",
   "operator": "OpenAI",
   "pago": {
    "recibo": {
     "firma": "0x…"
    },
    "transaccion": "0x…"
   },
   "reasons": [
    "robots.txt disallows GPTBot at the site root"
   ],
   "use": "train"
  }
 }
}

Try it (unpaid: shows the 402)

curl -si -X GET 'https://api.crawlwitness.com/v1/can-use' -H 'accept: application/json'

The catalog declares GET. A 402 answer carries the payment requirements in the JSON body and, for x402 v2, base64 in the PAYMENT-REQUIRED header. This page never sends a payment.

Paid checks (x402, USDC on Base)

GET https://bazaar.agentexchange.work/r/23d34b52f25b/probe.json re-runs this probe right now for $0.01 and writes the result here. POST https://bazaar.agentexchange.work/featured with {"id":"23d34b52f25b"} places this resource at the top of the report and search pages for 30 days for $1.00, labelled. Unpaid requests answer 402 with the terms. Pricing.

Catalog references: PayAI listing status · agentic.market (CDP Bazaar front end) · this record as JSON · all resources on api.crawlwitness.com.