newsJuly 13, 2026·AI News Daily

AI agent crawlers now need permission. Here’s how to get it

AI agent crawlers, the bots that fetch pages in real time on behalf of a person waiting for an answer, will be blocked by default on a slice of the web from September 15 onwards. Cloudflare announced the change on July 1, and most of the coverage since then has focused on Google. The more useful par

Key Takeaways

•AI agent crawlers, the bots that fetch pages in real time on behalf of a person waiting for an answer, will be blocked by default on a slice of the web from September 15 onwards
•This story was reported by AI News Daily, covering developments in the news space.
•AI advancements continue to reshape industries — read the full article on AI News Daily for complete coverage.

📖 Continue reading the full article:

Read Full Article on AI News Daily →

Share this article

X Facebook Reddit ☕ Support

TechCrunch AI

Already rich, already successful, why the last wave of tech winners is grinding again

They're rolling up their sleeves again, seemingly out of fear of missing AI's defining moment and, presumably, the irresistible allure of making even more money -- potentially a lot more.

TechCrunch AI

Uber’s product chief on hotels, robotaxis, and why the company doesn’t want to be “everything for everyone”

Uber Chief Product Officer Sachin Kansal walks TechCrunch through the company's financial-services ambitions, its increasingly complicated relationship with Waymo, its new AV Labs data operation, and how AI is starting to show up in ways riders and drivers will actually notice.

TechCrunch AI

Hermes agent maker Nous Research in talks for new funding at $1.5B valuation

The company is raising at least $75 million, led by Robot Ventures, with significant participation from USV and other prominent investors.

TechCrunch AI

Satya Nadella has issued a shocking warning to companies using AI

Of all the debates raging about the potential downsides of AI, there is one worry causing the most hand-wringing among AI enthusiasts in Silicon Valley — that the giant AI labs that sell proprietary models are somehow acting like Trojan horses.

Discussion

Loading articles...

AI agent crawlers now need permission. Here’s how to get it

Key Takeaways

Related Articles

Already rich, already successful, why the last wave of tech winners is grinding again

Uber’s product chief on hotels, robotaxis, and why the company doesn’t want to be “everything for everyone”

Hermes agent maker Nous Research in talks for new funding at $1.5B valuation

Satya Nadella has issued a shocking warning to companies using AI

Discussion

AI agent crawlers now need permission. Here’s how to get it

Key Takeaways

Related Articles

Already rich, already successful, why the last wave of tech winners is grinding again

Uber’s product chief on hotels, robotaxis, and why the company doesn’t want to be “everything for everyone”

Hermes agent maker Nous Research in talks for new funding at $1.5B valuation

Satya Nadella has issued a shocking warning to companies using AI

Discussion

Related Articles

TechCrunch AI
Already rich, already successful, why the last wave of tech winners is grinding again
They're rolling up their sleeves again, seemingly out of fear of missing AI's defining moment and, presumably, the irresistible allure of making even more money -- potentially a lot more.

TechCrunch AI
Uber’s product chief on hotels, robotaxis, and why the company doesn’t want to be “everything for everyone”
Uber Chief Product Officer Sachin Kansal walks TechCrunch through the company's financial-services ambitions, its increasingly complicated relationship with Waymo, its new AV Labs data operation, and how AI is starting to show up in ways riders and drivers will actually notice.

TechCrunch AI
Hermes agent maker Nous Research in talks for new funding at $1.5B valuation
The company is raising at least $75 million, led by Robot Ventures, with significant participation from USV and other prominent investors.

TechCrunch AI
Satya Nadella has issued a shocking warning to companies using AI
Of all the debates raging about the potential downsides of AI, there is one worry causing the most hand-wringing among AI enthusiasts in Silicon Valley — that the giant AI labs that sell proprietary models are somehow acting like Trojan horses.