Rewiring the Web: Data Access, AI Agents & the Next Digital Revolution | RAISE Summit 2026

Core Drivers of Demand:

• The AI era has made live, fresh web data essential for model training, as most large language models possess fixed knowledge cutoff dates without continuous web access.
• Beyond AI, web scraping remains critical for e-commerce analytics, price monitoring, and regulatory compliance.
• Agentic commerce is emerging as a dominant trend, where AI agents autonomously gather information, compare prices, and complete purchases on behalf of users.

Shift in Data Acquisition Dynamics:

• Data acquisition has become significantly more difficult due to the proliferation of advanced anti-bot technologies.
• Defense mechanisms have evolved from basic CAPTCHAs to sophisticated fingerprinting techniques, including analysis of operating systems, browser configurations, and mouse movement patterns to distinguish humans from bots.
• Successful scraping now requires high-level skills in reverse engineering and cybersecurity, effectively shifting the landscape from simple data collection to complex bot engineering.

Legal, Ethical, and Economic Constraints:

• Distinct "red lines" exist regarding data acquisition, specifically the prohibition of creating fake accounts to bypass login walls and purchasing access via dark web tokens.
• Regulatory environments vary geographically: the US and China are relatively permissive for AI training, while the EU imposes stricter regulations, and China simultaneously restricts access for data building.
• The traditional web economics are disrupted; AI models now answer queries directly without driving referral traffic to source sites, threatening the ad-revenue model for publishers.
• Content producers face a crisis in monetization as subscription and ad models may fail if data is freely scraped by AI aggregators without revenue sharing.

Strategic Recommendations for the Industry:

• Website owners must transition from building "for search engines" to building "for AI agents," potentially adopting standardized data formats similar to Cloudflare's LLM-markdown version.
• The industry is expected to undergo a paradigm shift where blocking data is replaced by strategic open data access, mirroring the historical evolution of search engine optimization in the late 1990s.
• Arthur Mensch (Mistral.ai) argues that over-regulation will stifle the new AI economy, as data owners must participate in data creation to secure a share of the resulting value.
• Long-term competition will likely rely on a balance of price, product quality, and reviews rather than price alone, though competitors like Amazon have already begun restricting review access behind login walls.