Fake AI Crawlers Are Knocking on Your Servers — And They Want Your .env Files
Threat actors are impersonating OpenAI, Anthropic, Google, DeepSeek and Perplexity crawlers to scan for exposed secrets. Here is how the forgery works, why allow-listing AI bots made it worse, and how to tell real crawlers from impostors.
The Friendliest Bot on Your Logs Might Be a Thief
Over the past year, thousands of site operators added AI crawlers — GPTBot, ClaudeBot, Google's AI agents, PerplexityBot — to their allow-lists, either to appear in AI answers or simply to reduce alert noise. GreyNoise's 28 August research shows attackers noticed. Scanners are now spoofing those exact user-agent strings to probe for .env files, AWS credential stores, private keys and configuration backups on misconfigured hosts.
The brand does the social engineering. The request does the stealing.
How the Forgery Gives Itself Away
GreyNoise's telemetry identifies three reliable tells:
- Wrong origin. The traffic does not come from the IP ranges OpenAI, Anthropic, Google and others publish for their crawlers.
- No manners. Legitimate crawlers honor
robots.txt; the impostors largely ignore it. - Wrong appetite. Real crawlers want indexable content. These scanners want secrets.
Why This Matters More Than a Typical Scan
Credential-hunting scans are nothing new. What changed is the trust surface: AI crawlers are now a category many defenders deliberately wave through. A user-agent that once triggered a rate-limit or block is now a free pass — unless verification goes deeper than a string match.
Practical Defenses
- Verify, don't trust. Match AI-crawler traffic against vendor-published IP ranges or reverse-DNS validation. A user-agent header is a claim, not proof.
- Block secret paths universally. No legitimate visitor — crawler or human — needs
/.env,/.aws/credentials,/.git/or*.pem. Deny them at the edge and alert on attempts. - Audit your web roots. Search for accidentally deployed secret files; rotate anything found, because you should assume it has already been read.
- Instrument the impostors. Forged-crawler traffic is high-volume and consistent — ideal for honeypot and WAF signatures.
The Takeaway
This campaign does not compromise any AI system. It exploits the reputation of AI systems to lower defenders' guard. As AI crawlers become permanent fixtures of the web, verifying who is actually knocking is now a baseline control, not an optimization.
Sources: GreyNoise, https://www.greynoise.io/blog/threat-actors-posing-as-ai-crawlers; Cyber Security News; Help Net Security.
