Work

Seconds Against a Day: Certificate Transparency, Wildcard Certificates and the Discovery of New Hostnames by Scanners and AI Crawlers in a Six-Arm Honeypot

Year: 2026
Type: preprint
Abstract: Every publicly trusted TLS certificate is recorded in the Certificate Transparency (CT) logs, which makes the logs a real-time feed of new hostnames. The usual advice for keeping a hostname out of the feed is a wildcard certificate, but that advice has not been tested with the served certificate held constant. We created 18 empty hostnames on three established domains in a two-by-three design: nine were named in their own certificate, nine were covered only by a wildcard, all 18 served the same wildcard certificate, and each listed 500 URLs in one of three places. Within 24 hours all 9 logged hostnames were requested by third parties (95% CI 70.1 to 100.0%) against 1 of 9 unlogged (2.0 to 43.5%; Fisher’s exact p = 0.00041). The first request reached a logged host 1.2 to 4.4 s after the log’s timestamp. The unlogged host was reached after 23.9 hours by one client; DNS analytics show the domains under constant subdomain guessing, so a wildcard certificate does not hide a dictionary label. ClaudeBot reached every logged host within 161 to 283 s and fetched all 500 URLs on each, finding an unannounced /sitemap.xml on its own. GPTBot arrived within 1,361 to 5,498 s and fetched all 500 only where the root page listed them as HTML links. Neither reached an unlogged host. An estimated 92.0% of first-hour scanner requests were for unlisted paths, mostly environment, credential and configuration files; one scanner sent 41.0% of all traffic. One client sent 14 AI-crawler names from a single cloud network, so we counted crawlers only from their operators’ networks. The pseudonymised request-level dataset, the honeypot source, the analysis plan and a script that recomputes every number are released under CC BY 4.0. Pre-print, not peer reviewed. Dataset and analysis script: 10.5281/zenodo.24203362. AI assistance: a large language model (Claude, Anthropic) wrote the honeypot and analysis code and drafted the manuscript under the author’s direction; the paper’s AI-assistance statement gives the details. The vendor of that model operates one of the crawlers measured.
Language: English
Open Access status: green
OpenAlex ID: W7226028654
Cites:
Cited by:
Related to: 10