最新报道:Cloudflare has partnered with AI startup Perplexity to block stealth web crawlers that scrape content without permission. The collaboration aims to identify and stop unauthorized AI bots that bypass traditional detection methods. Perplexity's AI-powered crawler, which indexes web pages for its answer engine, will now respect Cloudflare's new "AI bots" category in its firewall settings. Websites can choose to block all AI crawlers or allow specific ones like Perplexity's. This move comes amid growing concerns over AI companies using scraped data without proper attribution or compensation.