March 15, 20265 min read
The End of the Scraping Arms Race: Unpacking Cloudflare’s New /crawl Endpoint
For the past few years, the AI boom has been fueled by a notoriously fragile infrastructure: web scraping. In March 2026, Cloudflare effectively made that legacy workflow obsolete for the cooperative web.