Registries and bot lists
EdgeFetch is being registered in two kinds of places. MCP directories let people find and install it. Bot directories let website owners recognise it as a polite fetcher acting for a person, not a crawler collecting AI training data.
Two identities, kept apart
Section titled “Two identities, kept apart”Because anyone can run EdgeFetch, there are two separate things to register:
| What | Who registers it | What it covers |
|---|---|---|
| The project | The maintainer | The EdgeFetch user agent and its documentation, MCP directory listings, and signed releases |
| A fleet | Each operator, for their own hub | That operator’s own signing key, so sites can verify their traffic |
The repository never ships a shared signing key. If it did, anyone could sign as you, and the key would end up revoked for everyone.
Status
Section titled “Status”This page is kept current. Nothing is listed until the table says so.
| Directory | Kind | Status |
|---|---|---|
| Official MCP Registry | MCP | Being prepared. server.json names the server io.github.bartrumb/edgefetch and pins a versioned container image |
| Glama | MCP | Being prepared. glama.json lists the maintainer |
| PulseMCP and mcp.so | MCP | Planned |
| Known Agents | Bot directory | Planned, as a user-triggered fetcher that does not train models |
| Cloudflare Verified Bots | Bot directory | Per operator, using Web Bot Auth. See below |
| OpenSSF Best Practices | Open-source security badge | Planned |
What EdgeFetch tells site owners about itself
Section titled “What EdgeFetch tells site owners about itself”- User agent:
EdgeFetch/<version> (bot@edgefetch.dev), with the robots.txt tokenEdgeFetch. Operators should replace the contact with their own. - A policy file: every hub serves
/.well-known/edgefetch-bot.json, saying it follows robots.txt strictly, stops at challenges, and does not train AI models. This site serves the same file at edgefetch.dev/.well-known/edgefetch-bot.json. - Signatures: with Web Bot Auth switched on, requests carry RFC 9421 HTTP Message Signatures that a site can check against the operator’s published key.
For the site owner’s view, see For website owners.
Register your own fleet with Cloudflare Verified Bots
Section titled “Register your own fleet with Cloudflare Verified Bots”If you run a hub and want your traffic recognised:
-
Create a signing key:
Terminal window edgefetch keygen-bot-auth -
Give the private seed to your hub:
Terminal window cd hubnpx wrangler secret put BOT_SIGNING_PRIVATE_KEYOn the Go or portable hub, set the same value as an environment variable. Workers that fetch directly also need
EDGEFETCH_BOT_SIGNING_KEYandEDGEFETCH_BOT_DIRECTORY_URL. -
Check that your key directory is live and signed:
Terminal window curl -i https://<your-hub>/.well-known/http-message-signatures-directoryYou should see
200, the content typeapplication/http-message-signatures-directory+json,signatureandsignature-inputheaders, and a JSON body with your Ed25519 key. -
Submit it in the Cloudflare dashboard as a user-triggered fetcher (not a training crawler), with Web Bot Auth as the verification method and that directory URL.
Polite pools for research sites
Section titled “Polite pools for research sites”SEC EDGAR, OpenAlex, Crossref and NCBI give better service to clients that identify themselves. Put a real contact in your user agent:
EDGEFETCH_USER_AGENT="EdgeFetch/2.6 (mailto:you@yourdomain.com)"The full maintainer notes are in docs/REGISTRATION.md.
