Index bloat is when a large number of thin, duplicate, or low-value URLs end up in Google's index. On a big ecommerce site it happens quietly and at scale: a few filter parameters combine into hundreds of thousands of crawlable pages, and before long the index holds ten times the URLs you actually care about. The fix is consolidation, done carefully so you keep the pages that rank.
What index bloat is
Bloat is not just having many pages. A large catalogue with genuinely distinct products is fine. Bloat is having many pages that are near-identical or offer nothing a searcher wants: sorted views of the same list, tracking-parameter copies, empty filter results, thin tag pages. These dilute the signals that should concentrate on your real product and category pages, and they waste crawl.
Where it comes from
On ecommerce, the usual suspects are predictable.
| Source | Example | Typical volume |
|---|---|---|
| Faceted navigation | /shoes?color=red&size=9&sort=price | Very high |
| Tracking and session parameters | /product?utm_source=... or ?sessionid= | High |
| Pagination and view variants | ?page=2, ?view=grid, ?per_page=60 | Medium |
| Thin or empty results | Filter combinations with zero products | Medium to high |
Scroll table horizontally →
How to measure it
Start by quantifying the gap. Compare the number of URLs Google has indexed against the number you actually want indexed. The Pages report in Search Console, the Indexed, not submitted in sitemap bucket in particular, is the fastest way to see bloat. Then group the offending URLs by pattern so you can act on rules rather than individual pages.
The logs matter here too, because they show whether Google is spending real crawl on these patterns. Our guide to log file analysis covers how to pull that quickly.
A useful way to size the problem is to crawl the site and group the results by URL template, then compare that against your product feed or database count. If the crawl finds ten times more indexable URLs than you have real products, the ratio alone tells you how much of the crawl and index budget is being spent on near-duplicates. That single number is often what gets an index bloat project onto the roadmap, because it translates a vague sense of mess into a figure a stakeholder can act on.
How to consolidate safely
For each pattern, pick one of four actions. The right choice depends on whether the URL has any value and whether people search for it.
- Keep and index the filter combinations that have search demand, for example a genuine category like waterproof running shoes.
- Canonicalise parameter and sort variants to the clean base URL, so signals consolidate.
- Noindex pages that must stay accessible for users but should not be in the index, like some paginated or filtered views.
- Block in robots.txt only the genuine crawl traps that have no user value and no rankings to lose.
<!-- On /shoes?sort=price -->
<link rel="canonical" href="https://example.com/shoes" />
<!-- Keep the demand-having facet self-canonical -->
<!-- On /shoes/waterproof -->
<link rel="canonical" href="https://example.com/shoes/waterproof" />The order matters. Identify and protect the valuable filtered pages first, then apply consolidation to everything else. Do it the other way round and you will de-index pages that were quietly earning traffic. For the deeper version of this, see our post on faceted navigation and SEO.
A worked example
Picture a mid-sized fashion retailer with 12,000 real products. In Search Console the site shows 380,000 indexed URLs. That gap is the whole story. A quick group by pattern reveals where the extra 368,000 came from: colour and size filter combinations, a sort parameter on every listing, and a print-friendly version of each product page. None of it was created on purpose. It accumulated as features shipped.
The plan writes itself once the patterns are visible. The print versions get a canonical pointing to the main product page. The sort parameter gets canonicalised to the clean listing. The deep filter combinations get controlled, keeping the handful with real search demand, such as a specific colour that people actually search for, and removing the rest from indexing. The team checks Search Console first and finds that about 200 filtered URLs earn clicks, so those are protected and made self-canonical before anything else changes.
The result is not instant. Over the following six to eight weeks the indexed count drifts down toward the real catalogue size as Google recrawls and reprocesses. Crawl, freed from chasing hundreds of thousands of near-duplicates, starts reaching new products faster. The illustrative point is that the fix was not clever, it was disciplined: measure the gap, group by pattern, protect what earns traffic, and consolidate the rest.
Mistakes to avoid
Two errors show up again and again. The first is blocking bloat in robots.txt without canonicalising or removing it. Robots.txt stops crawling, not indexing, so a blocked-but-linked URL can still sit in the index, now uncrawlable and stuck. The second is mass-noindexing without checking traffic, which quietly kills filtered pages that ranked.
Also resist the urge to fix everything at once. Roll out the rules by pattern, watch index coverage move toward the intended set over a few weeks, and confirm no valuable pages dropped. Reprocessing at this scale takes time.
The takeaway: index bloat is one of the most common and most fixable problems on large ecommerce sites. Quantify it, group by pattern, decide per pattern, and protect what ranks before you cut. If the catalogue is large and the facets are complex, a crawl and indexation engagement will get the rules right without collateral damage.