Crawlability
Whether a search bot can find a page, request it, and get useful content back. A page behind the wrong block is effectively invisible.
See it
What it is
Crawlability is whether a search bot can discover a URL, request it, and receive the page's useful content. The path starts with a real <a href> link or an XML sitemap, then runs through robots.txt, DNS, the CDN, authentication, redirects, and the final HTTP response.
Reach for it when an important page never appears in crawl logs or Google Search Console says the URL is blocked or unreachable. Check the exact production URL as Googlebot, not just the page in your logged-in browser. A staging Disallow, a 403 from a bot rule, an infinite redirect, or navigation built only from click handlers can stop the trip.
Gotcha: crawlable does not mean indexable. A bot can fetch a page that has noindex and correctly leave it out of search. The reverse is stranger: blocking a URL in robots.txt can prevent Google from seeing the noindex instruction on that page.
Ask AI for it
Test this site's crawlability. Use Google Search Console URL Inspection, curl with a Googlebot User-Agent, and server access logs to verify that every important URL is discoverable through a real <a href> link, permitted by RFC 9309 robots.txt rules, and returns a final 200 response without a redirect loop, login wall, 403, or soft 404. Compare raw HTML with the rendered DOM and list every resource or route that blocks the main content.