x402 census · x402-agent-tools.nerolabs.workers.dev · f500748361b6 · JSON
POST /ai-crawlers
gone
Checks if AI crawlers can read up to 20 websites: GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot and Meta-ExternalAgent by default (four more optional). Reads robots.txt, fetches each site as each bot vs a browser to catch firewall blocks, and checks llms.txt. Returns allowed or blocked per bot and why. $0.005 per website. Unreachable sites are not charged. Runs nerolabs/ai-crawler-access-checke…
Verdict: dead listing. 404/410: gone.
Facts from the catalogs
| Field | Value |
|---|---|
| URL | https://x402-agent-tools.nerolabs.workers.dev/ai-crawlers |
| Method | POST |
| Price | $0.01 USDC = 10000 atomic units of 0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913 |
| Network | eip155:8453 (base) |
| payTo | 0xA969a3127DcC74B2989a4b4e45Ae61B81093164A |
| Seller | x402-agent-tools.nerolabs.workers.dev |
| Catalogs | cdp (row updated 2026-11-18) |
| Listed since (lower bound) | 2026-11-18 (earliest catalog timestamp; catalogs report last-update times only) |
| Last catalog update | 2026-11-18 |
| CDP quality counters | 1 calls and 1 unique payers in 30 days, last call 2026-11-01 (catalog-reported, not verified on chain) |
| Category (keyword rule) | inference/llm |
| Templated path | no |
| x402Version in catalog | 2 |
| Service name / tags | Nero Labs Agent Tools · ai crawlers, gptbot, robots.txt, llms.txt, seo |
Unpaid probe (census of 2026-11-18)
| Field | Value |
|---|---|
| Probed at | 2026-11-18T02:08:02.260Z |
| HEAD | 404 |
| GET | 404 |
| Verdict | gone 404/410: gone |
| 402 carried in | n/a |
| x402Version in the 402 | n/a |
| accepts[0] | none |
| Live amount equals catalog price | not comparable |
| Live payTo equals catalog payTo | not comparable |
| extensions.bazaar in the 402 | no |
| PAYMENT-REQUIRED header | no |
| WWW-Authenticate offer | none |
| Content type / server / CORS | application/json / cloudflare / * |
| Latency | 620 ms |
| Error | none |
No hourly re-probe has reached this resource yet; the cron covers a rotating slice of 150 per hour.
On-chain facts for the payTo (public address)
| Field | Value |
|---|---|
| Settlement history | unresolved: payTo not seen in the payer forensics (180 wallets' outbound history) or the Blockscout sample |
Declared input and output (extensions.bazaar)
{
"input": {
"body": {
"websites": [
"nytimes.com",
"wikipedia.org"
]
},
"bodyType": "json",
"method": "POST",
"type": "http"
},
"output": {
"type": "json",
"example_truncated": true,
"chars": 1566
}
}
Try it (unpaid: shows the 402)
curl -si -X POST 'https://x402-agent-tools.nerolabs.workers.dev/ai-crawlers' -H 'accept: application/json' -H 'content-type: application/json' --data '{}'
The catalog declares POST. A 402 answer carries the payment requirements in the JSON body and, for x402 v2, base64 in the PAYMENT-REQUIRED header. This page never sends a payment.
Paid checks (x402, USDC on Base)
GET https://bazaar.agentexchange.work/r/f500748361b6/probe.json re-runs this probe right now for $0.01 and writes the result here. POST https://bazaar.agentexchange.work/featured with {"id":"f500748361b6"} places this resource at the top of the report and search pages for 30 days for $1.00, labelled. Unpaid requests answer 402 with the terms. Pricing.
Catalog references: agentic.market (CDP Bazaar front end) · this record as JSON · all resources on x402-agent-tools.nerolabs.workers.dev.