Bot mitigation
Telling automated traffic apart from real people, then blocking, slowing, or challenging the bad half without punishing everyone else.
See it
What it is
Bot mitigation is two jobs stapled together: deciding whether a request came from a person, then deciding what to do about it. Detection reads signals a script struggles to fake: TLS fingerprints (JA3, JA4), header order and casing, missing Accept-Language, headless browser tells, IP reputation and ASN (datacenter ranges are not where customers live), and behavior over time (300 product pages in 40 seconds, no mouse movement, signup completed in 0.4s). Responses range from silent block, to a challenge (Cloudflare Turnstile, hCaptcha), to tarpitting, to quietly serving a cached or slightly wrong response so the scraper never learns it was caught.
Reach for it on the endpoints where automation actually costs you: signup (fake accounts, free-tier abuse), login (credential stuffing against leaked password lists), password reset and OTP send (SMS pumping fraud), checkout (carding), search and pricing pages (scrapers), and any expensive AI endpoint. Not on your marketing pages, where you want Googlebot and preview crawlers to sail through, which is why any real setup keeps a verified good-bot allowlist.
Gotchas: CAPTCHAs tax your real users hardest, break for screen reader users, and cost attackers about a tenth of a cent per solve at a solver farm, so treat them as friction, not a wall. Blocking by user agent string is theater, since it is one line to change. robots.txt is a polite request, not enforcement, and the current wave of AI crawlers is the reason that stopped being funny. Watch your false positive rate as closely as your block rate: a blocked signup is a lost customer who never tells you.
Ask AI for it
Add layered bot mitigation to the signup, login, and password reset endpoints of this app. Layer 1: rate limit per IP and per account, with exponential backoff after failures. Layer 2: an invisible Cloudflare Turnstile challenge that only escalates to interactive when the risk score is high. Layer 3: server-side heuristics scoring datacenter ASNs, missing Accept-Language, form-fill time under one second, and disposable email domains. Route high scores to a challenge rather than a hard block, log every decision with the reason so I can audit false positives, and allowlist verified Googlebot and our uptime monitor by reverse DNS, not by user agent string.