Contributing
EdgeFetch is MIT licensed, and issues and pull requests are triaged weekly. The full guide is CONTRIBUTING.md, and everyone follows the Code of Conduct.
Where help matters most
Section titled “Where help matters most”- Adapters. A new site, job board or open-data portal is one file and one fixture. Start with Write an adapter.
- False-positive fixtures. If EdgeFetch flagged a normal page as a bot challenge, or missed a
real one, an HTML sample makes the classifier better for everyone
(
internal/classify/waf_test.goandhub/src/waf.ts). - Hardware reports. Tell us EdgeFetch runs on your router, NAS, Raspberry Pi, FreeBSD box or Mac, so a platform can move from “build verified” to “live tested”.
- Docs. This site lives in bartrumb/edgefetch-site. Every page has its source there.
Rules every pull request keeps
Section titled “Rules every pull request keeps”- Politeness can’t get weaker. No change may loosen robots.txt handling,
Retry-AfterandCrawl-delay, or the dial-time IP checks. - No ways around site defences (permanently out of scope). Anti-bot fingerprinting, TLS/HTTP2 fingerprint spoofing, stealth browser patches, CAPTCHA solvers, and rotating residential proxies are permanently out of scope.
- Every outbound call has a deadline. Every network fetch carries a timeout and cancels or drains unread bodies; contributed adapters run inside a zero-secret, zero-network bulkhead.
- DCO sign-off & contributor vouching. Sign commits with
git commit -s; first-time external PRs are gated by.github/VOUCHED.tdbefore untrusted workflows run. - The supply chain stays small. The Go module keeps exactly one external dependency, and the hub keeps zero npm runtime dependencies. CI checks both.
- Built-in adapters stay keyless. Paid services belong in operator settings, not in the adapter library.
- Tests come with changes. Every fix or adapter includes a fixture or a unit test.
- Release files stay out of PRs. Leave
.github/workflows/release.ymlandkeys/alone unless the maintainer asks.
AI-assisted pull requests are welcome if you’ve read every line yourself, checked robots.txt on the live site, and run the tests below.
By opening a pull request you confirm you can license your work under the MIT licence.
Check your work locally
Section titled “Check your work locally”Go worker, CLI and Go hub (Go 1.26 or later):
gofmt -s -w .go vet ./...go test -race ./...TypeScript hub (Node.js 22.5 or later):
cd hubnpm cinpm run type-checknpm testnpm run deploy:dryExtraction benchmark:
node bench/run-bench.mjsSecurity issues
Section titled “Security issues”Please don’t open a public issue. Report privately through GitHub Security Advisories.
