Reddit Blocks Internet Archive Amid AI Data Scraping Concerns

Reddit has announced it will restrict the Internet Archive’s Wayback Machine from accessing most of its content, citing concerns about AI companies exploiting the digital preservation service to scrape data in violation of platform policies.

The move significantly limits what portions of Reddit can be archived for future reference.

Major Access Restrictions Implemented

The social media giant will now block the Internet Archive from indexing post detail pages, comments, and user profiles.

Only Reddit’s homepage will remain accessible to the Wayback Machine, effectively limiting the archive to capturing which headlines and posts were trending on specific dates rather than preserving the full context of discussions and user interactions, as per a report by TheVerge.

“Internet Archive provides a service to the open web, but we’ve been made aware of instances where AI companies violate platform policies, including ours, and scrape data from the Wayback Machine,” Reddit spokesperson ...