Pagination and infinite scroll are two of the most common ways that large sites accidentally hide their own content from search engines and AI crawlers. The short answer: if a crawler cannot reach a URL by following a real link, that content effectively does not exist for search or for AI. Blog archives, product listings, and forum threads that live only behind "Load more" buttons or scroll-triggered JavaScript are routinely invisible. Here is how to keep every paginated page reachable and indexable.
The core problem with infinite scroll
Infinite scroll feels modern, but it is built on an event that crawlers do not fire: scrolling. A bot loads the page, sees the first batch of items, and stops. It never triggers the JavaScript that fetches page two, three, or thirty. Everything below the first load is unreachable. This is the same class of problem that makes JavaScript-only websites invisible to AI assistants — content that exists for humans but not for machines.
The fix is not to remove infinite scroll but to back it with real, crawlable pagination. Each "page" of results should exist at its own indexable URL such as /blog/page/2, and those URLs should be linked in the HTML. Progressive enhancement is the principle: serve working paginated links in the markup, then layer infinite scroll on top for people who have JavaScript. Bots follow the links; humans get the smooth scroll.
How to paginate so crawlers follow every page
Give every paginated page a unique, static URL and make sure page-to-page navigation uses ordinary anchor links, not buttons that only respond to clicks in JavaScript. A crawler reads href attributes; it does not press buttons. If your "next" control is a div with an onclick handler, the next page is invisible.
A few rules keep the sequence healthy. Each page in the series should self-canonicalize — page two canonicals to page two, not back to page one, because collapsing everything onto page one tells engines the deeper pages do not exist. Every paginated URL should return a proper 200 status and be free to crawl. Do not noindex deep pages reflexively; if page five holds the only link to a product, noindexing it can strand that product. For the mechanics of status codes and canonicals, see our guide to canonical URLs and duplicate content.
What happened to rel=prev and rel=next
For years the advice was to add rel="prev" and rel="next" link tags to signal a paginated sequence. Google confirmed it stopped using those signals for indexing, so they are no longer required for Google. That does not make pagination markup pointless — Bing and other engines, which feed AI systems like Copilot, may still use these hints, and they cost nothing to include. The real work, though, is structural: crawlable links and unique URLs matter far more than any link-relation tag.
The practical takeaway is that you cannot rely on a meta hint to rescue a broken structure. If the pages are not linked and addressable, no annotation will surface them. Fix the plumbing first.
Pagination, crawl budget, and AI retrieval
On a large site, endless paginated URLs can drain crawl budget, especially when combined with filters and sort parameters that multiply into thousands of near-duplicate pages. Keep the paginated series clean and separate it from faceted filter URLs, which usually should not be indexed at all — our crawl budget guide covers how to stop bots wasting attention on junk URLs.
For AI retrieval, the goal is that any single item — a specific article, product, or answer — sits on its own stable, directly linkable URL, reachable within a couple of clicks from the homepage. Deep, well-linked pages get chunked and retrieved; orphaned ones do not. If you want to see which of your pages crawlers and AI can actually reach right now, run the CheckMy.site scanner and find the content that is hiding behind scroll.