Faceted navigation
Catalog filters that can turn one category into thousands of crawlable URLs, most of them duplicate or useless to searchers.
See it
What it is
Faceted navigation is the filter system on a large catalog: size, colour, brand, price, availability. Each choice narrows the same inventory, but when every combination gets its own crawlable URL, ten filters can multiply into millions of pages. Ecommerce sites are the classic case, though job boards, property listings, and documentation search can build the same trap.
Keep a filter URL indexable when it answers a real search need with enough distinct results, such as /shoes/womens/running/. Keep sorting, view mode, stock toggles, and empty combinations out of the index. The job is an allowlist: decide which combinations deserve stable links, titles, canonicals, and sitemap entries, then stop crawlers wandering through everything else.
Gotcha: noindex does not save crawl budget, because a bot must fetch the page to see the directive. A canonical is also a hint, not a force field, and pointing every useful filtered page at the root category throws away pages people may actually search for. Start with URL design and real <a href> links, then use robots.txt only for patterns you are certain should never be crawled.
Ask AI for it
Audit this catalog's faceted navigation and implement an indexability allowlist. Inventory every query parameter and path segment produced by the filters, group combinations by result count and distinct search intent, and keep only valuable landing pages crawlable and indexable. Give those pages stable paths, self-referencing <link rel='canonical'> tags, unique title tags, and real <a href> links. Remove links to sort, view, session, empty-result, and near-duplicate combinations, then add precise Google-compatible robots.txt Disallow patterns for URL spaces that should never be fetched. Do not use noindex as a crawl-budget fix, and list every allowed filter URL that belongs in the XML sitemap.