Our crawler.

Agent402 publishes an index of services that advertise machine payments over x402 or MPP. To build it we read the discovery documents a service publishes for exactly this purpose: /.well-known/x402, /openapi.json, /llms.txt, and the HTTP 402 challenge a paid route returns to an unpaid request.

That is all. We do not log in, we do not submit forms, we do not read anything behind a paywall, and we do not pay to see more. A 402 challenge is a public advertisement of price and payment rails - answering one with no credential is the ordinary way to read it.

The crawler identifies itself on every request as Mozilla/5.0 (compatible; Agent402/1.0; +https://agent402.tools/crawler), so it can be recognised, rate-limited or blocked by name.

It honours robots.txt, including for the discovery paths above. It re-reads an origin about every thirty minutes, backs off on errors, sends conditional requests so an unchanged document costs a single 304, and caps what it will read from any one response.

If our crawler is causing you a problem, a Disallow for that User-Agent stops it and needs no message from us. Email works too.

A crawl and a purchase are different acts and this page is linked from both, so it should say so. When our router buys from a service to resell the result to a caller it sends a real HTTP 402 payment, which is not a crawl: it is a customer.

Those requests identify themselves as Mozilla/5.0 (compatible; Agent402-Router/1.0; +https://agent402.tools/crawler) and carry X-Agent402-Via: router. Either one separates a purchase from a probe in your logs. Until 2026-09-21 they carried neither and arrived as node, which we fixed when a seller pointed out they could not tell our traffic apart.

robots.txt does not govern these. A purchase is a request for a service you advertised a price for, and declining one is a matter for your own endpoint, not for a crawl directive. If you would rather we did not buy from you at all, say so and we will stop.

Email mike@agent402.tools from an address at the domain, or open an issue, and we will remove the origin from the index and stop crawling it. We do not require a reason and we do not argue about it.

Blocking the User-Agent in robots.txt has the same effect and takes immediate effect on the next cycle without waiting for us.

The public index carries what a service advertises about itself: its endpoints, the prices and payment networks it declares, whether our own probe could reach it, and the payout addresses it names in its own challenges. Payout addresses are public infrastructure - they appear in every challenge the service hands to every buyer.

We publish counts of settled payments per payee, read from public chains, and buyer figures in the index are counts. Payments on public chains are public: some of our pages show individual payments as the chain records them, for example the live view on our home page, which shows shortened payer addresses with links to the public transactions. We do not sell payer data.

Measurements made by third parties are not redistributed. Where our pages display someone else's figures we label them as theirs, and they are excluded by name from any dataset we distribute.

If a row about your service is wrong, tell us and we will fix it. Several improvements to how prices are read came from operators who checked their own row and found it stale - that feedback loop is the main reason the index is accurate, and it is why the index is free to read.

If you index this catalog, three documents carry every route and its price: /.well-known/x402, /openapi.json and /api/pricing. Read those rather than requesting each priced route to learn what it costs. /openapi.json is served with an ETag, so a repeat read that sends If-None-Match is a 304 with no body.

Listed prices change when we ship a release, not from one request to the next, so a full walk of every priced route more than a few times an hour is not needed.

Unpaid requests to priced routes are budgeted per client, meaning one address and one User-Agent product token. Past 3,000 in an hour, the answer is a 429 with a Retry-After header that counts down to the top of the hour. A request that carries a payment, a credits key or a proof-of-work solution is never counted, and neither are the three documents above, /mcp or any page. Every purchase opens with one bare request, so buying is exempt from the first settlement onward: once a client settles a payment, nothing it sends is counted for the rest of that hour.

Named indexers, search engines and AI crawlers are not budgeted. If your index needs more than this and we do not recognize it by name, email mike@agent402.tools with the User-Agent it sends and we will add it to the named list.