Deploy a hub
The hub is the only part of EdgeFetch that faces the internet. It serves /mcp, the REST API,
the /admin console and OAuth 2.1, keeps per-host pacing and cooldowns, and hands work to
connected workers. Worker nodes never accept inbound connections.
Pick one place to run it. For the click-and-go versions, see One-click deploys.
| Option | Runtime | Storage | Idle cost | Workers built in? |
|---|---|---|---|---|
| Cloudflare Workers | TypeScript hub on a hibernating Durable Object | Durable Object SQLite, optional R2 and D1 | $0 on the free plan | No. Cloudflare egress is a fallback only |
| All-in-one container | Node hub + Go worker + Chromium | SQLite with FTS5 under /data |
Whatever the host charges | Yes |
| Portable hub on Deno or Node | hub/portable/ |
Turso (libSQL over HTTP) and any S3-compatible bucket | Low | No |
Go binary, --mode=hub |
One static Go binary | Local | One small VPS | Add ,agent to run both |
Cloudflare Workers
Section titled “Cloudflare Workers”git clone https://github.com/bartrumb/edgefetch.gitcd edgefetch/hubnpm cinpx wrangler secret put CALLER_TOKENnpx wrangler secret put AGENT_TOKENnpm run deployThe Durable Object hibernates between requests, and idle worker pings are answered by
Cloudflare’s auto-response without waking it, so a quiet hub costs nothing. R2 (artifacts) and
two D1 databases (crawl metadata and full-text search) are optional bindings in
hub/wrangler.toml. Native Workers logs and traces are switched on in the same file.
All-in-one container
Section titled “All-in-one container”The simplest self-hosted setup: one container with the hub, a Go worker with headless Chromium, and SQLite full-text search. No Redis, no Postgres.
git clone https://github.com/bartrumb/edgefetch.git && cd edgefetchCALLER_TOKEN="$(openssl rand -hex 24)" docker compose up --build -ddocker-compose.yml binds to 127.0.0.1 only. Put TLS in front of it before you expose it.
The same image runs on Render (render.yaml), Railway (railway.json), Fly.io (fly.toml) and
Coolify. Give it a persistent volume at /data.
Portable hub on Deno Deploy or Node
Section titled “Portable hub on Deno Deploy or Node”hub/portable/ runs the same hub code outside Cloudflare. It swaps D1 for Turso over HTTP and R2
for any S3-compatible store (AWS S3, R2, Tigris, MinIO or Backblaze B2), using WebCrypto SigV4
with no extra dependencies.
cd hubnpm cinpm run start:portable # builds dist/portable-server.mjs and starts it on Node 22.5+Point it at your storage with TURSO_DATABASE_URL and TURSO_AUTH_TOKEN, plus S3_ENDPOINT,
S3_BUCKET, S3_ACCESS_KEY_ID and S3_SECRET_ACCESS_KEY. Without them it keeps everything under
EDGEFETCH_DATA_DIR on local disk.
On Deno Deploy, use the button on One-click deploys or point a project at
hub/portable/deno-server.ts.
Go binary on a VPS
Section titled “Go binary on a VPS”The Go binary can be the hub itself, serving /agent/ws, /v1/scrape, /fetch and /mcp:
EDGEFETCH_CALLER_TOKEN="<32-char-secret>" \EDGEFETCH_AGENT_TOKEN="<32-char-secret>" \ edgefetch --mode=hub --addr=0.0.0.0:8080Use --mode=hub,agent to make one box both the hub and a worker, for example a mini PC on your
tailnet.
High availability
Section titled “High availability”These layers stack, and none needs an external broker:
- Several hubs per worker. Give a worker
EDGEFETCH_HUB_URLS="wss://cf-hub.example.com,wss://standby.fly.dev"and it holds a connection to each, sharing one per-host politeness state. - Durable Object shards. Set
HUB_SHARDS="global,standby"on the Cloudflare hub and connect workers to?shard=globaland?shard=standby. - Client failover. The TypeScript and Python SDKs take a list of
baseUrlsand move to the next hub on network errors or502,503and504.
Tracing
Section titled “Tracing”The hub and workers export OpenTelemetry over OTLP/HTTP and pass W3C traceparent across the
WebSocket, so the hub’s dispatch and the worker’s DNS, connect, TLS and first-byte timings show
up as one trace in Honeycomb, Grafana Tempo, Axiom, SigNoz or Jaeger.
OTEL_EXPORTER_OTLP_ENDPOINT="https://api.honeycomb.io"OTEL_EXPORTER_OTLP_HEADERS="x-honeycomb-team=<YOUR_API_KEY>"OTEL_SERVICE_NAME="edgefetch-hub"See also Troubleshooting and traces.
