Indexing in search engines is the step where a search engine like Google stores what it found on your web page in its index: a huge database of pages it can show in results. First a crawler visits the page, then Google works out what it’s about and decides whether to keep it. If a page isn’t indexed, it can’t appear in search results or in AI Overviews, however good it is.
If you’ve seen “Crawled – currently not indexed” in Search Console, or someone told you your site “isn’t indexed”, this guide is for you. I’m a freelance web designer and SEO consultant in London, and checking indexing is the first thing I do on every new site and every SEO client. People ask me “what is indexing in search engines?” all the time, usually because a page is missing from Google. On 13 November 2026 I checked everything below against Google’s own guide to how Google Search works, its Search Console Help pages and its developer docs. Most articles on page one were written for developers or for people building a site search box. This one is for business owners who just want their pages found.
Key takeaways
- Search engine indexing means a search engine has stored your page in its database so it can show it in results.
- Google works in three stages: crawling, indexing and serving results. Not every page makes it through all three.
- Google says plainly that indexing isn’t guaranteed, and it doesn’t accept payment to crawl or index a site.
- You can check any page for free with the URL Inspection tool in Google Search Console.
- A page has to be indexed before it can appear as a link in Google’s AI Overviews or AI Mode.
What is indexing in search engines?
Search engine indexing is how Google, Bing and the rest file away what they learned about your page, so they can pull it out in a split second when someone searches. Google describes its index as “a large database hosted on thousands of computers”.
Think of a library. A book can sit in the building, but if it was never added to the catalogue, nobody can find it. Search engine indexing is the catalogue. When Google indexes a page, it reads the text, the title, the images and their alt text, works out the language and the area the content is aimed at, and stores all of that against the page’s address.
Google also groups pages that look the same and picks one to keep. Google calls the chosen one the canonical: “The canonical is the page that may be shown in search results.” So if your site has three near-identical versions of a page, only one is likely to be indexed and shown.
How does search engine indexing work?
Search engine indexing is the middle step of three. Google’s own guide says it “works in three stages, and not all pages make it through each stage”: crawling, indexing and serving search results.
Here’s what each stage means for a small business website, and which parts you can actually influence:
| Stage | What happens | What you control |
|---|---|---|
| 1. Crawling | Google’s crawler (Googlebot) finds your page through links from other pages or your sitemap, and downloads it. | A sitemap, links between your own pages, a site that loads and isn’t blocked. |
| 2. Indexing | Google reads the page, groups duplicates, picks the canonical and decides whether to store it in the index. | Useful, original content, one clear version of each page, no accidental “noindex”. |
| 3. Serving | When someone searches, Google pulls matching pages from the index and ranks them. | Relevance and quality: this is where normal SEO work pays off. |
Google finds most pages by following links. In its words, pages are discovered “when Google extracts a link from a known page to a new page”, or when you submit a sitemap. That’s why a brand new page with no links pointing to it can sit unnoticed for weeks.
Is being indexed the same as ranking?
No. Being indexed only means your page is allowed into the race. Ranking is where it finishes, and that depends on how useful and relevant Google thinks it is compared with every other indexed page.
Even indexing itself is never promised. Google’s guide says “Google doesn’t guarantee that it will crawl, index, or serve your page”, even if you follow its rules, and adds that “Indexing isn’t guaranteed; not every page that Google processes will be indexed.” It also says something every business owner should hear before paying anyone for “fast indexing”: “Google doesn’t accept payment to crawl a site more frequently, or rank it higher.”
So if your page is indexed but nowhere to be seen, that’s a ranking problem, not an indexing one. My guide to why a website isn’t showing on Google walks through both, and how long SEO takes explains what happens after a page is in the index.
Why isn’t Google indexing my page?
Usually for one of a handful of reasons, and Search Console tells you which. Google lists the common ones as low content quality, robots meta rules that block indexing, and website designs that make indexing difficult.
The Page indexing report in Search Console gives each non-indexed page a reason. These are the ones I see most on small business sites:
| What Search Console says | What it means | Do you need to act? |
|---|---|---|
| Discovered – currently not indexed | Google knows the page exists but hasn’t crawled it yet. | Usually just wait. Add internal links if it lasts weeks. |
| Crawled – currently not indexed | Google visited the page and chose not to index it, for now. | Often not on its own. If it’s an important page, make it more useful and less like your other pages. |
| URL marked “noindex” | A tag on the page tells Google not to index it. | Yes, if you want the page found. This often gets left on after a site launch. |
| Duplicate without user-selected canonical | Google thinks it’s a copy of another page and picked the other one. | Only if Google picked the wrong page. |
| Alternate page with proper canonical tag | The page correctly points to the main version. | No. This is working as intended. |
| Page with redirect | The old address forwards to a new one. | No, as long as the new page is indexed. |
| Not found (404) | The page doesn’t exist any more. | Only if it moved. Then add a 301 redirect. |
| Blocked by robots.txt | Your robots.txt file stops Google crawling it. | Yes, if the page should be in Google. |
| Server error (5xx) | Your hosting failed when Google tried to visit. | Yes. Talk to your host if it keeps happening. |
| Soft 404 | The page looks empty or like an error, but doesn’t return a 404 code. | Yes. Add real content or remove the page properly. |
One status catches people out. A robots.txt block doesn’t keep a page out of Google. Google’s robots.txt guide says it “is not a mechanism for keeping a web page out of Google”: the address can still show up, just without a description. If you want a page out of search results, a “noindex” tag is the right tool, and my guide to meta tags in SEO explains how it works.
Pages missing from Google and not sure why? I’m a freelance SEO consultant, not an agency, so you deal with me from the first call. I’ll check your Search Console, find out which pages aren’t indexed and why, and tell you honestly what’s worth fixing, even if that’s “you can do this yourself”. See my SEO consultant services in London or book a free consultation.
How do I check if my website is indexed?
The quickest rough check is a Google search, and the reliable one is Search Console’s URL Inspection tool. Here’s the order I do it in:
- Search Google for site:yourdomain.co.uk. It gives a rough list of indexed pages. If nothing comes up at all, something is seriously wrong.
- Open Google Search Console. If you don’t have it, set it up first. It’s free, and my guide on what Google Search Console is shows how.
- Paste a page address into the search bar at the top to run the URL Inspection tool.
- Read the verdict. “URL is on Google” means the page is eligible to appear. “URL is not on Google” means it won’t appear, and the Page indexing section says why.
- Click “Test live URL” if you’ve just fixed something. It checks the page as it is now, not the version Google stored.
- Open Indexing, then Pages to see the whole site: how many pages are indexed and the reason for each one that isn’t.
- Fix, then click “Validate fix” on that issue. Google says validation “typically takes up to about two weeks”.
One 2026 quirk worth knowing. The Page indexing report itself sometimes runs late. In summer 2026 it stalled at 11 June data for over three weeks, until Google updated it on 3 July, as Search Engine Land reported. When people later noticed the June gap in their charts, Google’s John Mueller replied that Google doesn’t “back-fill indexing data”. So if the report looks frozen, check single pages with URL Inspection before you panic.
How long does indexing take?
Anything from a few hours to a few weeks. Google’s page on asking Google to recrawl your URLs says “Crawling can take anywhere from a few days to a few weeks.”
Here’s real data from my own blog. Every article below showed “URL is unknown to Google” on the day I published it. Two weeks later I checked each one with URL Inspection:
| Article on my blog | Published | Crawl date shown in Search Console | Status at the 14-day check |
|---|---|---|---|
| How to add your business to Google Maps | 19 Oct 2026 | 22 Oct 2026 | Submitted and indexed |
| How long Google reviews take to show up | 21 Oct 2026 | 22 Oct 2026 | Submitted and indexed |
| How to add your business to Apple Maps | 24 Oct 2026 | 29 Oct 2026 | Submitted and indexed |
| How to remove Google reviews | 25 Oct 2026 | 28 Oct 2026 | Submitted and indexed |
| How to respond to Google reviews | 26 Oct 2026 | 28 Oct 2026 | Submitted and indexed |
| How to change business information on Google | 28 Oct 2026 | 28 Oct 2026 | Submitted and indexed |
| What is Google Search Console? | 29 Oct 2026 | 30 Oct 2026 | Submitted and indexed |
That’s an established site with a sitemap and plenty of internal links. A brand new site with no links pointing to it will usually take longer, and indexed doesn’t mean ranking: most of those articles were still climbing weeks later.
Can I make Google index my site faster?
A little, yes, but nobody can force it. The honest, free options are a sitemap, good internal links and the “Request indexing” button in Search Console.
For one or two pages, inspect the address in Search Console and click Request indexing. Google’s URL Inspection Help page says “There is a daily limit to how many index requests you can submit” and that “Submitting a request does not guarantee that the page will appear in the Google Index.” Pressing it ten times won’t help either: Google’s recrawl page says asking again for the same address won’t speed things up. For lots of pages, submit a sitemap instead.
Watch out for two shortcuts that don’t do what they promise:
- Google’s Indexing API. Some plugins and “instant indexing” services use it for normal pages. Google’s Indexing API quickstart says it “can only be used to crawl pages with either JobPosting or BroadcastEvent embedded in a VideoObject”, in other words job adverts and livestreams, and warns that abuse “may result in access being revoked”.
- IndexNow. It’s a real, free way to tell search engines about new pages, and I use it. But it’s for Bing and other search engines: the IndexNow FAQ lists Amazon, Bing, Naver, Seznam.cz, Yandex and Yep, not Google, and says “Submitting a URL does not guarantee immediate indexing.”
Should every page on my website be indexed?
No, and that’s normal. Google’s Page indexing report help says “Don’t expect every URL on your site to be indexed”, and only the main (canonical) version of each page needs to be.
The pages that should be in Google are the ones a customer might search for: your homepage, service pages, area pages, contact page and useful articles. Pages that can happily stay out include thank-you pages after a form, cart and checkout pages, login pages, tag archives and test pages. For those, a “noindex” tag is fine. Just never block a page in robots.txt and add noindex at the same time: Google’s noindex guide warns that if robots.txt blocks the page, “the crawler will never see the noindex rule”.
If you’re moving to a new site, this matters even more. Old addresses need 301 redirects to the new ones, or Google ends up with a pile of 404s. My guide to a website redesign without losing SEO covers it.
Does indexing affect AI Overviews?
Yes. A page that isn’t indexed can’t be linked from Google’s AI Overviews or AI Mode. Google’s page on AI features and your website says that to be a supporting link, “a page must be indexed and eligible to be shown in Google Search with a snippet”.
The same page adds: “There are no additional requirements to appear in AI Overviews or AI Mode, nor other special optimizations necessary.” So search engine indexing is the boring basic that comes first: get indexed, then be the most useful answer. My guide on how to rank in Google AI Overviews picks it up from there.
Which situation are you in?
The right next step depends on what you’re seeing. Find your row:
| What you see | Likely reason | What I’d do |
|---|---|---|
| A new page isn’t in Google after a day or two | Google hasn’t crawled it yet | Link to it from another page, check it’s in the sitemap, request indexing once. |
| The whole site is missing from site: searches | A sitewide noindex, a robots.txt block or a brand new domain | Run URL Inspection on the homepage straight away. |
| “Crawled – currently not indexed” on key pages | Google doesn’t see enough value, or pages look too similar | Make each page more useful and clearly different. |
| Lots of “Not found (404)” after a new website | Old addresses weren’t redirected | Add 301 redirects from old to new pages. |
| Pages are indexed but get no clicks | A ranking problem, not indexing | Work on content, titles and local SEO. |
| You’ve never opened Search Console | No data to judge | Set it up and check your homepage today. |
Can I sort out indexing myself?
Often, yes. Setting up Search Console, submitting a sitemap and inspecting your key pages is free and takes an afternoon. If it’s just a new page waiting its turn, patience and a couple of internal links usually do the job.
It’s worth getting help when important pages stay out for weeks, when the report is full of errors you don’t understand, or when traffic dropped after a redesign. One honest note: I don’t edit websites someone else built, because you never know how they were put together. On an existing site I can run local SEO, including indexing checks and a monthly report, from £495 a month (minimum 6 months, then month to month). If the site itself is the problem, every website I build launches with Google Search Console and Analytics set up and a sitemap submitted. Prices are on my pricing page.
Search engine indexing FAQs
What is indexing in search engines?
Indexing is when a search engine stores what it learned about a web page in its index, a huge database of pages, so it can show that page in search results. A page that isn’t indexed can’t appear in Google.
What is the difference between crawling and indexing?
Crawling is when a search engine’s bot visits and downloads your page. Indexing is the next step, when the search engine analyses the page and decides whether to store it. A page can be crawled and still not be indexed.
How do I know if Google has indexed my website?
Search Google for site:yourdomain.co.uk for a rough check, then use the URL Inspection tool in Google Search Console. URL is on Google means the page is eligible to appear in results.
How long does it take Google to index a new page?
Google says crawling can take anywhere from a few days to a few weeks. On my own established blog in October 2026, new articles were crawled between the same day and five days after publishing.
Can I pay Google to index my website?
No. Google says it doesn’t accept payment to crawl a site more frequently or rank it higher. Be wary of anyone selling guaranteed or instant Google indexing.
Does a page need to be indexed to appear in AI Overviews?
Yes. Google says a page must be indexed and eligible to be shown in Google Search with a snippet to appear as a supporting link in AI Overviews or AI Mode.
Want to know if Google can find your pages?
Now you know what indexing in search engines means, why some pages never make it in and how to check yours in a few minutes. If you’d rather have someone go through your Search Console, find the pages Google is ignoring and tell you what’s worth fixing, tell me about your business. You’ll get an honest answer, even if it’s “you can do this yourself”. You can also read what an SEO company actually does, or see my SEO consultant services in London.
Book a free consultation, or call or WhatsApp me on 07404 145722. No jargon, just a clear answer about where your website stands with Google.
Freelance web designer and SEO in Clapham, London, and two-time Prestige Awards Web Designer of the Year. I design, build and rank websites for small businesses. More about me
