This page describes Marketfauna's automated requests, so that any site or bot manager that sees them can tell what they are and who to contact. There are two kinds: a weekly collector for the Bot Barometer, and occasional access readings.
| User-Agent | MarketfaunaBot/1.0 (+https://marketfauna.com/bot.html; [email protected]) |
| Purpose | Collects a small set of public counts for a weekly index of agent-economy activity, published on this site. |
| What it fetches | Public JSON APIs: the GitHub search API and the Hacker News API. It does not crawl web pages, does not follow links, and does not fetch pages behind a login. |
| How often | Once a week, Mondays around 09:00 UTC, from GitHub Actions. At most one request every two seconds to any host. |
| Operator | Marketfauna. Contact [email protected]; replies come from the operator's agents. |
| Source | github.com/marketfauna/bot-barometer (the collector is tools/collect_index.py). |
| User-Agent | Either the collector's User-Agent above, or an operator's own User-Agent followed by MarketfaunaReading/1.0 (+https://marketfauna.com/bot.html; reading for <operator>). One control request per site uses python-urllib (generic HTTP client; control request for an access reading; +https://marketfauna.com/bot.html). We never present as a browser. |
| Purpose | A dated reading of how a named public URL answers an identified, non-browser request, for the operator of a feed reader, link-preview service or similar agent. Described on the readings page. |
| What it fetches | /robots.txt, then one public URL per site, only if robots.txt allows it for both the operator's agent name and ours. If robots.txt is refused or unreachable, the page is not requested. No links are followed, nothing behind a login is requested, and no challenge is attempted. |
| How much | A handful of requests per site (robots.txt, one control, then up to three unsigned and three signed), at least two seconds apart and slower where robots.txt asks for a crawl delay. A reading is run once, not on a schedule. Readings to date: 15 and 18 September 2026. |
| From where | Our own network. For comparison, the same reading is sometimes repeated once from a GitHub-hosted Actions runner, and the report says which vantage each result came from. |
| What is kept | Status, response headers without cookie values, and a hash of the body. Response bodies are not stored. |
If you run a site either of these has reached and would rather it did not, email the address above and it will be excluded.
The collector implements Web Bot Auth (RFC 9421 HTTP Message Signatures, Ed25519). Production requests are currently unsigned. After the signed key directory is deployed and bot verification is complete, signing can be enabled so a receiving site can check the operator's identity. The public key:
{"kty":"OKP","crv":"Ed25519","x":"XmPA2ES_rjqaqSrHB2iSR-hpYsnSAMjGsAIEO73fp3c"}
keyid (RFC 7638 thumbprint): F1L6QDTZZUbg8vKM1PIxrvsrkychEG9PUGz76RJgP3w
The signing and verification code is published as web-bot-auth-python (MIT) for any operator who needs the same path.
Status on 2026-09-15: the signed key directory is served at marketfauna-wba-directory.marketfauna.workers.dev/.well-known/http-message-signatures-directory (a Cloudflare Worker; this site's static host cannot add the signed response headers the format requires). An application to Cloudflare's Bots and Agents Directory was submitted on 2026-09-15 and is waiting for review. Production requests stay unsigned until that review completes. No verified-bot status is claimed.
For operators: access readings and Web Bot Auth setup, priced and scoped on their own page.
Marketfauna, [email protected]