x402 census · api.crawlwitness.com · 23d34b52f25b · JSON
GET /v1/can-use
live
Use when a crawler, scraper or RAG pipeline must decide whether it may use a website for AI: yes/no verdict for a domain, AI bot and use (train, rag, search) from robots.txt, IETF AIPREF, TDMRep, noai and RSL licenses, with a signed receipt. Errors: 400 invalid domain or unknown bot (not charged).
Verdict: live. The unpaid GET reached a valid 402. The live amount equals the catalog price.
Facts from the catalogs
| Field | Value |
|---|---|
| URL | https://api.crawlwitness.com/v1/can-use |
| Method | GET |
| Price | $0.001 USDC = 1000 atomic units of 0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913 |
| Network | eip155:8453 (base); also accepts solana:5eykt4UsFv8P8NJdTREpY1vzqKqZKvdp |
| payTo | 0xa20F0b8ADf20Ed17a8d27B369B3058afdB71a5C5 |
| Seller | api.crawlwitness.com |
| Catalogs | cdp (row updated 2026-11-18), payai (row updated 2026-11-12) |
| Listed since (lower bound) | 2026-11-12 (earliest catalog timestamp; catalogs report last-update times only) |
| Last catalog update | 2026-11-18 |
| CDP quality counters | 38 calls and 12 unique payers in 30 days, last call 2026-11-17 (catalog-reported, not verified on chain) |
| Category (keyword rule) | trading-signals |
| Templated path | no |
| x402Version in catalog | 2 |
| Service name / tags | CrawlWitness · ai-consent, robots-txt, crawler-compliance, data-licensing, batch |
Unpaid probe (census of 2026-11-18)
| Field | Value |
|---|---|
| Probed at | 2026-11-18T02:08:12.026Z |
| HEAD | 402 |
| GET | 402 |
| Verdict | live live: valid 402 to an unpaid GET |
| 402 carried in | header+body |
| x402Version in the 402 | 2 |
| accepts[0] | {"scheme":"exact","network":"eip155:8453","amount":"1000","payTo":"0xa20F0b8ADf20Ed17a8d27B369B3058afdB71a5C5","asset":"0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913","maxTimeoutSeconds":300,"extra":{"name":"USD Coin","version":"2"}} |
| Live amount equals catalog price | yes |
| Live payTo equals catalog payTo | yes |
| extensions.bazaar in the 402 | yes |
| PAYMENT-REQUIRED header | yes |
| WWW-Authenticate offer | none |
| Content type / server / CORS | application/json / n/a / none |
| Latency | 579 ms |
| Error | none |
No hourly re-probe has reached this resource yet; the cron covers a rotating slice of 150 per hour.
On-chain facts for the payTo (public address)
| Field | Value |
|---|---|
| Settlement history | unresolved: payTo not seen in the payer forensics (180 wallets' outbound history) or the Blockscout sample |
Declared input and output (extensions.bazaar)
{
"input": {
"method": "GET",
"queryParams": {
"bot": "GPTBot",
"domain": "nytimes.com",
"use": "train"
},
"type": "http"
},
"output": {
"type": "json",
"example": {
"allowed": false,
"bot": "GPTBot",
"bot_role": "training",
"decision": "not_allowed",
"domain": "nytimes.com",
"method": "can-use/1",
"operator": "OpenAI",
"pago": {
"recibo": {
"firma": "0x…"
},
"transaccion": "0x…"
},
"reasons": [
"robots.txt disallows GPTBot at the site root"
],
"use": "train"
}
}
}
Try it (unpaid: shows the 402)
curl -si -X GET 'https://api.crawlwitness.com/v1/can-use' -H 'accept: application/json'
The catalog declares GET. A 402 answer carries the payment requirements in the JSON body and, for x402 v2, base64 in the PAYMENT-REQUIRED header. This page never sends a payment.
Paid checks (x402, USDC on Base)
GET https://bazaar.agentexchange.work/r/23d34b52f25b/probe.json re-runs this probe right now for $0.01 and writes the result here. POST https://bazaar.agentexchange.work/featured with {"id":"23d34b52f25b"} places this resource at the top of the report and search pages for 30 days for $1.00, labelled. Unpaid requests answer 402 with the terms. Pricing.
Catalog references: PayAI listing status · agentic.market (CDP Bazaar front end) · this record as JSON · all resources on api.crawlwitness.com.