Gentoo's Bugzilla vs. Thousands of IPs Nobody Can Block

Gentoo Bugzilla went dark because one maintainer got tired of subsidizing AI training runs with his own server bill. That's the real headline, even if the polite version says "scraper overload."

Michał Górny didn't mince words. He took the tracker offline because it was "unusable anyway" and he had zero interest in feeding LLM scrapers more free data. The traffic wasn't coming from a handful of misbehaving bots you could firewall in an afternoon. It was coming from thousands of different IPv4 addresses, with no discernible pattern. That's not a scraper. That's a botnet-scale distributed harvesting operation wearing a scraper costume.

Here's the part that should worry every open-source maintainer running public infrastructure:

You can't rate-limit your way out of that. You can't fail2ban a moving target with no fixed origin. The entire defensive playbook built for the last twenty years of web abuse — IP blocklists, basic rate limiting, robots.txt — assumes an adversary that behaves like a single actor or a small botnet. AI scraping infrastructure in 2025 looks more like a distributed denial-of-service attack that happens to also want your bug reports for training data.

The Real Story

Everyone arguing on Hacker News about whether this was a "real" shutdown or just a temporary mitigation is missing the point entirely. One commenter noted the tracker came back up, which triggered the usual pile-on of "see, it wasn't that serious." That's exactly backwards.

The fact that Górny could take it down, breathe, and bring it back up is the story. This is now a lever every maintainer of public infrastructure has to consider pulling. Bugzilla instances, wikis, forums, changelogs — anything text-heavy and structured is catnip for model training pipelines. GCC's own Bugzilla already runs Anubis, a proof-of-work challenge system, explicitly to fend off the same kind of aggressive AI crawling. That's not paranoia. That's the new baseline cost of running open infrastructure.

A LinkedIn post floating around claimed AI crawlers hit Gentoo's tracker at 40,000 requests per second, ignoring robots.txt entirely. That number isn't corroborated anywhere in the primary source or the HN thread, so treat it as unverified noise — but the underlying claim, that crawlers ignore robots.txt, tracks with what every other project dealing with this has reported.

What's actually happening:

• robots.txt is decorative. It was a gentleman's agreement. AI scraping outfits apparently didn't get the memo, or don't care.
• IP-based blocking is obsolete against distributed harvesting. You need behavioral fingerprinting, proof-of-work, or segmentation — not blocklists.
• Volunteer projects eat the cost. Gentoo isn't a venture-backed company with a platform team. It's volunteers paying for bandwidth and CPU to train someone else's commercial model, for free, without consent.
• Segmentation works, sort of. One HN commenter suggested routing bot traffic to a dedicated server via Cloudflare load balancing, and claimed it's kept their own infrastructure stable. That's a real mitigation, not a theory.

The uncomfortable truth is that this incident is a preview, not an anomaly. Expect more proof-of-work walls, more CAPTCHA gates, more friction between legitimate researchers and the sites they need to access — all because AI companies decided crawl etiquette was optional. Open source built its reputation on being open. That openness is now a line item on someone's AWS bill, and nobody's writing the check except the maintainers.