Tech »  Topic »  Cloudflare will now block AI crawlers on your website - and even force them to pay you

Cloudflare will now block AI crawlers on your website - and even force them to pay you


(Image credit: Thomas Fuller/SOPA Images/LightRocket via Getty Images)
  • Cloudflare is introducing a way to charge AI web scrapers
  • Content creators can protect their sites from unwanted scrapers
  • Specific crawlers can be granted free access, charged, or blocked

Online creators often have very little control over the types of crawlers that can access their content, but Cloudflare may have a solution.

The company has revived HTTP response code 402 as a neat way to block or charge AI crawlers to access your site in a new feature it calls ‘pay per crawl’.

The best part is, it’s not a block or charge all control - users will be able to allow specific crawlers to access their site for free, charge others for access, and block the ones you don’t want trawling your content.

GenAI bots could well be scraping your web apps, researchers warnForget Chrome and Edge ...
Copyright of this story solely belongs to techradar.com . To see the full text click HERE