AI Agent Readiness Across 100,000 Websites
Year: 2026
Type: article
Abstract: This study evaluates web readiness for autonomous AI agents across the top 100,000 domains in the Tranco global ranking using 120 deterministic technical indicators across six operational layers. While direct requests retrieve readable HTML on nearly half of all websites (48.66%), dedicated machine interfaces such as llms.txt (7.16%), OpenAPI schemas (0.23%), and Model Context Protocol servers (0.05%) remain rare. Domains satisfy an average of 21.33% of evaluated criteria when counting verified capabilities, reflecting substantial operational uncertainty caused by live-web network drops, timeouts, and bot defenses. These empirical observations establish a baseline for web agent infrastructure, indicating that autonomous systems must continue relying on browser rendering, DOM heuristics, and fallback scraping until structured machine contracts achieve widespread adoption.
Author: Vatsa, Vedang Ratan
Language: English
Topic: Web Data Mining and Analysis
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225729891
FWCI:
Cites:
Cited by:
Related to: