Skip to content

Contributing

EdgeFetch is MIT licensed, and issues and pull requests are triaged weekly. The full guide is CONTRIBUTING.md, and everyone follows the Code of Conduct.

  • Adapters. A new site, job board or open-data portal is one file and one fixture. Start with Write an adapter.
  • False-positive fixtures. If EdgeFetch flagged a normal page as a bot challenge, or missed a real one, an HTML sample makes the classifier better for everyone (internal/classify/waf_test.go and hub/src/waf.ts).
  • Hardware reports. Tell us EdgeFetch runs on your router, NAS, Raspberry Pi, FreeBSD box or Mac, so a platform can move from “build verified” to “live tested”.
  • Docs. This site lives in bartrumb/edgefetch-site. Every page has its source there.
  1. Politeness can’t get weaker. No change may loosen robots.txt handling, Retry-After and Crawl-delay, or the dial-time IP checks.
  2. No ways around site defences (permanently out of scope). Anti-bot fingerprinting, TLS/HTTP2 fingerprint spoofing, stealth browser patches, CAPTCHA solvers, and rotating residential proxies are permanently out of scope.
  3. Every outbound call has a deadline. Every network fetch carries a timeout and cancels or drains unread bodies; contributed adapters run inside a zero-secret, zero-network bulkhead.
  4. DCO sign-off & contributor vouching. Sign commits with git commit -s; first-time external PRs are gated by .github/VOUCHED.td before untrusted workflows run.
  5. The supply chain stays small. The Go module keeps exactly one external dependency, and the hub keeps zero npm runtime dependencies. CI checks both.
  6. Built-in adapters stay keyless. Paid services belong in operator settings, not in the adapter library.
  7. Tests come with changes. Every fix or adapter includes a fixture or a unit test.
  8. Release files stay out of PRs. Leave .github/workflows/release.yml and keys/ alone unless the maintainer asks.

AI-assisted pull requests are welcome if you’ve read every line yourself, checked robots.txt on the live site, and run the tests below.

By opening a pull request you confirm you can license your work under the MIT licence.

Go worker, CLI and Go hub (Go 1.26 or later):

Terminal window
gofmt -s -w .
go vet ./...
go test -race ./...

TypeScript hub (Node.js 22.5 or later):

Terminal window
cd hub
npm ci
npm run type-check
npm test
npm run deploy:dry

Extraction benchmark:

Terminal window
node bench/run-bench.mjs

Please don’t open a public issue. Report privately through GitHub Security Advisories.