Pop the Hood - See robots.txt, security.txt, humans.txt & More

Description

Reads robots.txt, security.txt, humans.txt, ads.txt, sitemap.xml and more - the layer of the web most people never see. Every website you visit publishes a set of files about itself - how it wants to be crawled, who to contact about security issues, who built it, and how its advertising works. These files sit in plain sight. Anyone can read them. Most people just never know to look. Pop the Hood shows you everything. WHAT IT DOES Visit any website and Pop the Hood silently checks for all publicly available meta files in the background. Click the icon to open the side panel and see exactly what the site has chosen to publish about itself - and what it hasn't. Files found are shown with a plain English summary of what they contain. You choose exactly which files to check, from a minimal core set up to the full list. WHAT IT READS robots.txt - crawler instructions, AI training blocks, crawl delays and duplicate rules security.txt - who to contact about vulnerabilities, expiry status, PGP keys humans.txt - who built the site ads.txt - authorised advertising sellers sellers.json - the full ad supply chain app-ads.txt - mobile advertising sellers sitemap.xml - the site's own map of itself and how many pages it has manifest.json - Progressive Web App metadata mta-sts.txt - email transport security policy pgp-key.txt - public encryption key dnt-policy.txt - Do Not Track commitment llms.txt - AI content index (emerging standard) llms-full.txt - full AI content index gpc.json - Global Privacy Control policy *LATEST ADDED* ai.txt - AI usage policy (still a contested format, shown as found/not found) WHAT MAKES IT DIFFERENT Most tools look at one file. Pop the Hood reads them together and presents a complete picture of what a site chooses to be transparent about - in plain English, not jargon. You don't need to know what "Disallow: /" means to understand "Blocks all crawlers by default." Both audiences are served - the curious newcomer and the experienced developer who wants the raw file too. STANDOUT FEATURES Detects which AI crawlers are blocked or allowed, and whether they're blocked from training or just live browsing Flags security.txt files that have expired, are expiring soon, or have no expiry date set Detects easter eggs hidden in comment lines - ASCII art, hiring messages, greetings left by developers Identifies placeholder ads.txt entries (sites that publish the file but run no programmatic advertising) Flags duplicate rules in robots.txt - a sign the file hasn't been reviewed recently Detects cryptographically signed security.txt files Copy raw file or URLs to clipboard with one click Retry timed-out files without re-fetching everything PLAIN ENGLISH EXPLAINER Not sure what any of these files are? The built-in explainer covers all of them in plain language - what they are, why they exist, who reads them, and which formal internet standards they follow. PRIVACY Pop the Hood reads only publicly accessible files - the same files any browser can access by typing the URL directly. File selection preferences are stored locally in your browser. Fetched results are cleared automatically when the browser closes. Nothing is sent to the developer. The extension makes one additional request to Google's favicon service to display site icons. What's new: file selection settings, 15 supported files (added llms.txt, gpc.json, ai.txt most recently), an option to hide files a site hasn't published, and dozens of accuracy fixes including corrected AI crawler detection and easter egg false-positive cleanup.

Details & SEO