SEO

Caffeine

Also called Google Caffeine

The 2010 rebuild of Google's indexing infrastructure, which replaced batch index refreshes with continuous updating and much fresher results.

Quick facts: Caffeine

Category
SEO
Also called
Google Caffeine
Level
Intermediate
Affects
Index freshness, how quickly new pages appear, crawl behaviour
Where to see it
Search Console URL Inspection and Page Indexing report; server log files
In this article4
  1. How Caffeine works
  2. Why Caffeine matters
  3. Common mistakes with Caffeine
  4. How to act on it

How Caffeine works

Caffeine, which Google completed in 2010, was a change to indexing rather than to ranking. Before it, Google built its index in layers and refreshed each layer on a schedule, so a page could sit crawled but unpublished until its layer’s turn came round. Caffeine replaced that with a system which analyses small portions of the web continuously and adds them to the index as it goes.

The practical result is that crawling and indexing became a stream rather than an event. A page discovered this morning can appear in results the same day, and a page updated today does not wait for the whole index to be rebuilt around it. Google was explicit at the time that this made results fresher and the index larger, and that it was not a change to how pages were ranked.

Why Caffeine matters

It matters mostly as context for two things people still get wrong. The first is the belief that rankings shift on a monthly cycle — a leftover from the era Caffeine ended, when the index really did rebuild in batches and results really did lurch. The second is the assumption that publishing speed is irrelevant; once indexing became continuous, being early to cover something turned into a real advantage on time-sensitive topics.

It also set the pattern everything since has followed. Google’s systems now update continuously by default: core updates get announced, many other changes do not, and the idea of waiting for a scheduled refresh has gone entirely.

Common mistakes with Caffeine

The usual one is citing it as a ranking update. Caffeine changed the plumbing. It did not decide who wins a search result, and any explanation of a traffic drop that blames it is confused about what it was.

The second is assuming that continuous indexing means guaranteed indexing. Google still decides what is worth storing, and a page can be crawled and left out of the index anyway. Speed of discovery is not the same thing as an invitation.

How to act on it

Take the lesson rather than the update. Because indexing runs continuously, the things that slow you down are nearly all on your side: a page nothing links to, a sitemap that has not been regenerated since launch, a server that answers crawling slowly, or a template that hides new content behind scripts.

When something new needs to be found quickly, link to it from pages that are already crawled often, make sure it appears in the XML sitemap, and use URL Inspection in Search Console to request indexing for that single page. If pages routinely take a long time to show up, the cause is usually structural and belongs with your technical SEO rather than with your writing.

Do and do not

Do

  • Link new pages from pages already crawled often
  • Keep the XML sitemap regenerating automatically
  • Use URL Inspection when a page must appear quickly

Do not

  • Blame a ranking drop on an indexing rebuild
  • Assume crawled always means indexed
  • Wait for a monthly refresh that no longer exists

Questions people ask about this

Was Caffeine a ranking update?

No. It was a rebuild of how Google crawls, stores and updates its index, completed in 2010. Google said at the time that it changed how quickly content could be found and how large the index could grow, not how pages were ordered in results. Blaming a ranking drop on Caffeine confuses infrastructure with the algorithm.

Does Caffeine still exist?

The name is historical, but the principle it introduced is still how Google works: the index updates continuously instead of being rebuilt on a schedule. Google's indexing systems have been rewritten since and it no longer refers to Caffeine by name. What matters for a site owner is that there is no publishing cycle left to wait for.

Why does my new page take days to appear?

Continuous indexing means Google can index quickly, not that it must. Discovery depends on links pointing at the page, on your sitemap being current, and on the server answering crawl requests promptly. Selection depends on whether the page looks worth storing. Inspect the URL in Search Console to see which of the two is holding it up.

Related terms

Found this useful?

Share it, or ask an AI to summarise it

Back to the glossary

Knowing the term is the easy part

Applying it to your own site and budget is the work. Book a call and I will tell you what actually applies to you.