Blocking of AI web crawlers in US & UK leads to calls for fair revenue share in India

Wait 5 sec.

The global move has major ramifications for India. Cloudflare’s announcement that it will now block AI scrapers by default on all new domains signals a significant industry shift.As international momentum builds against the unauthorised scraping of journalistic content by artificial intelligence (AI) companies, Indian digital news publishers have renewed calls for fair revenue sharing in India as well.This comes in response to a landmark development in the US and UK, where major publishers have begun to block AI web crawlers by default, backed by a new initiative from Cloudflare, one of the world’s largest internet infrastructure companies. More than a dozen global news organisations – including Associated Press, The Atlantic, Sky News, Time, Buzzfeed, Conde Nast, and DMGT – are part of this effort to protect original journalism from unauthorised AI exploitation.Indian publishers, represented by industry groups such as the Digital News Publishers Association (DNPA), have been actively protesting against what they term “unauthorised data theft” by AI developers. Many publishers in India report that their news content, painstakingly created by human editors and journalists, is being used without consent or compensation to train commercial AI models. Some have already taken steps to join forces and explore legal challenges to such practices.“The situation in India is becoming increasingly untenable,” said a spokesperson for DNPA. “While global players are waking up to importance of permission and fair compensation, Indian news content continues to be freely exploited without dialogue or safeguards. We urge the Government of India to take immediate steps to take necessary measures against such unauthorised and rampant data scraping,” he said.The global move has major ramifications for India. Cloudflare’s announcement that it will now block AI scrapers by default on all new domains signals a significant industry shift. Website owners, including publishers, can now explicitly choose which AI crawlers to allow – and under what terms.DNPA and other digital publishers have called on the Ministry of Electronics and IT (MeitY) and the I&B Ministry to recognise unauthorised AI scraping as a violation of copyright and mandate consent-based access for AI training. It also sought to support the creation of an Indian licensing framework, possibly modeled on those now emerging in the West, along with providing technological tools, in partnership with firms like Cloudflare, to empower smaller publishers. “India has the potential to be a global AI leader – but not by trampling on the rights of its own creators,” said a senior editor from a national digital daily.Stay updated with the latest - Click here to follow us on Instagram© The Indian Express Pvt LtdTags:artificial intelligence