Back to News
Cloudflare’s new policy pushes AI companies to pay for publishers’ content
BusinessWednesday, July 1, 2026 at 06:00 PM

Cloudflare’s new policy pushes AI companies to pay for publishers’ content

Source: TechCrunch AI
Curated Summary

Cloudflare has announced a new policy requiring AI companies to separate web crawlers used for search from those used for AI training and agents.

Starting September 15, 2026, default settings will block mixed-use crawlers from pages hosting ads unless site owners adjust their preferences.

This move aims to protect publishers' intellectual property and create a sustainable ecosystem where content creators are compensated.

Cloudflare co-founder Matthew Prince emphasized the need for faster action as bot traffic now surpasses human traffic online.

The company is evolving its Pay Per Crawl tool into Pay Per Use, allowing publishers to charge AI firms based on content value.

Google has previously defended its practices, noting that opting out of AI training does not affect search visibility.

This policy shift highlights growing tensions between AI developers and content publishers over data usage rights.

Want the full story?

Read Original Article

via TechCrunch AI

#ai-policy#content-scraping#cloudflare#publishers#web-crawlers

Related Articles

OpenAI proposed donating 5% of its equity to a US sovereign wealth fund
TechCrunch AI
Business2d ago

OpenAI proposed donating 5% of its equity to a US sovereign wealth fund

OpenAI CEO Sam Altman has proposed donating 5% of the company's equity to a U.S. sovereign wealth fund, a move reported by the Financial Times. This initiative aims to secure good relations with the administration and address potential political blowback regarding AI's rapid growth. The proposal aligns with recent discussions involving President Trump about the American public becoming a partner in AI companies. OpenAI has previously outlined similar concepts in a policy paper titled 'Industrial Policy for the Intelligence Age,' suggesting returns could be distributed directly to citizens. While similar ideas have been proposed by Senator Bernie Sanders, including a 50% tax on AI stock, those legislative efforts have not yet advanced. The current OpenAI proposal remains preliminary and would likely require complex congressional approval to become reality. This development highlights the growing intersection between AI corporate strategy and national economic policy.

1 min readRead
Anthropic is discussing a new custom chip with Samsung
TechCrunch AI
Business2d ago

Anthropic is discussing a new custom chip with Samsung

Anthropic is reportedly in discussions with Samsung to develop a custom AI chip, signaling a serious move towards hardware independence. This potential partnership aims to diversify Anthropic's compute strategy beyond its current reliance on Google, Amazon, and Nvidia. The move comes amidst a broader industry trend where major AI firms seek to reduce dependency on Nvidia's dominant market position. Notably, this development follows OpenAI's recent announcement of its own custom inference processor, 'Jalapeño,' developed with Broadcom. While Anthropic has not yet disclosed specific technical details or use cases for the Samsung collaboration, the strategic alignment highlights the growing importance of specialized hardware. Samsung, already a key Nvidia partner, brings significant manufacturing capabilities to such high-stakes AI infrastructure projects. This shift underscores the competitive race among AI leaders to optimize performance and efficiency through proprietary silicon.

1 min readRead
Microsoft launches its own AI deployment company with $2.5 billion commitment
TechCrunch AI
Business2d ago

Microsoft launches its own AI deployment company with $2.5 billion commitment

Microsoft has launched a new operating business named Microsoft Frontier, dedicated to facilitating enterprise AI deployments. The initiative is backed by a substantial $2.5 billion investment and leverages 6,000 industry and engineering experts. Microsoft’s Commercial Business CEO Judson Althoff emphasizes that this venture exceeds traditional Forward Deployed Engineering models. This move positions Microsoft directly against competitors like AWS, which recently announced a similar $1 billion commitment. Early partnerships include major entities such as the London Stock Exchange Group, Unilever, and Accenture. The company aims to utilize its existing Fortune 500 client base to accelerate adoption of its AI tools. This strategic shift highlights the growing focus on practical implementation over mere model development.

1 min readRead