Blog

How to check the Google indexing status of every page on your site

Inspect every URL in your sitemap with the Search Console URL Inspection API from Claude, ChatGPT, or Cursor, then fix the pages Google has not indexed.

Search Console can tell you whether Google has indexed a page, and if it has not, why. In the Search Console interface you check one URL at a time. The URL Inspection API returns the same data, and an AI assistant connected to it through an MCP server can inspect every URL in your sitemap in one conversation and sort the results for you.

This post walks through that check in Claude, ChatGPT, Cursor, or Claude Code, with CalmSEO as the MCP server. The example is our own site, calmseo.com, which we inspected on November 11 and 12, 2026.

What indexing status means

For each URL, the URL Inspection API returns a verdict and a coverage state. The coverage state is the same label you see in the Page indexing report in Search Console. These are the ones you are most likely to see:

  • Submitted and indexed. The page is in Google's index and listed in a sitemap Google knows about.
  • URL is unknown to Google. Google has no record of the URL yet.
  • Discovered - currently not indexed. Google knows the URL exists but has not crawled it.
  • Crawled - currently not indexed. Google crawled the page and decided not to index it for now.
  • Excluded by "noindex" tag, Blocked by robots.txt, and the canonical states such as Alternate page with proper canonical tag. These usually point to something on your own site.

The result describes Google's indexed version from its last crawl. The API does not run a live test of the page.

Why Search Console is slow for a whole site

The URL Inspection tool in Search Console takes one URL, shows the result, and then you paste the next one. That works for a single page after a launch. For a sitemap with dozens or hundreds of URLs it turns into an afternoon of copying and pasting. The Page indexing report groups URLs by reason, but you still click into each reason to see which pages it covers.

The URL Inspection API has the same per-URL data. Google allows 2,000 inspections per day and 600 per minute for each property, which is enough to check a site with a few hundred pages in one session. Bulk index checker tools can look up any URL. The URL Inspection API only works for properties you can access in Search Console, and in return it gives you Google's own reason for each page.

What you need

  • Access to the site's property in Google Search Console.
  • An AI client with CalmSEO connected. CalmSEO has setup guides for Claude, ChatGPT, Cursor, and Claude Code.
  • A way for the assistant to read your sitemap. If it can fetch web pages, it can read the sitemap directly. In Cursor or Claude Code with your repo open, it can also build the list from your routes or sitemap config. Otherwise, paste the URLs into the chat.

The inspect_gsc_urls tool is free on every CalmSEO plan, like the other Search Console tools. CalmSEO connects to Google with the read-only webmasters.readonly scope.

Step 1. Connect Search Console

Sign up for CalmSEO, open the Google Search Console card in the dashboard, and approve read-only access with the Google account that can see your property. Then add the MCP endpoint https://mcp.calmseo.com/mcp to your client using the guide for that client. To check the connection, ask for your properties:

Use CalmSEO to list my Search Console properties.

This calls list_gsc_sites. Domain properties look like sc-domain:example.com and URL-prefix properties look like https://www.example.com/. Use the exact ID in the next prompts. Every URL you inspect has to be inside that property.

Step 2. Give the assistant your sitemap

Start by asking which sitemaps Google has for the property. The list_gsc_sitemaps tool returns each submitted sitemap with its last submitted and last downloaded times and its error and warning counts. A sitemap Google has not read recently is worth knowing about before you look at single pages.

Use CalmSEO to list the sitemaps for sc-domain:example.com and tell me when Google last read each one.
Then read https://example.com/sitemap.xml and list every URL in it.

CalmSEO does not download the sitemap file itself, so the URL list comes from the assistant's own web fetch, from your repo, or from a list you paste in.

Step 3. Run the inspection

The inspect_gsc_urls tool takes up to 25 URLs per call, and the assistant repeats the call for the rest of the list. For each URL it returns:

  • verdict, coverage_state, and indexing_state
  • robots_txt_state and page_fetch_state
  • last_crawl_time and crawled_as
  • google_canonical and user_canonical
  • in_sitemap, plus the sitemaps Google associates with the URL
  • inspection_result_link, which opens the same inspection in Search Console

Fields Google does not report are left out. A URL that fails comes back as a failed row, and the rest of the batch still returns. If the quota runs out or Google responds too slowly, the result keeps the URLs it already inspected and adds stopped_early with the remaining_urls to send again.

One detail on in_sitemap. It reflects the sitemaps Google knows list the URL, so a false value does not prove the URL is missing from every sitemap. Check the file before you change it.

Step 4. Sort the results by coverage state

Every call also returns counts by coverage state. Ask the assistant to add them up across calls and to list the URLs that are not indexed, grouped by state. Put the pages that matter most to you at the top: the home page, product and pricing pages, and anything you published recently.

Worked example: calmseo.com

On November 11, 2026, we inspected all 35 URLs in the calmseo.com sitemap. 19 were indexed. We had published 14 of those pages since October 12, and 4 of them were indexed: /blog, the post on SEO in Cursor, the keyword volume checker, and /openseo-alternative, which Google crawled a day after it went live. The other 10 were not indexed.

InspectionURLsIndexedNot indexed
Nov 11, full sitemap351916
Nov 11, pages published since Oct 1214410
Nov 12, recently published pages11011

The 10 unindexed new pages were either "URL is unknown to Google" or "Discovered - currently not indexed". All 10 were in the sitemap and linked from the footer and from at least two other pages, so internal linking did not explain why some new pages were indexed and others were not.

The next day we re-checked 11 recently published pages:

Coverage state on Nov 12URLs
URL is unknown to Google8
Discovered - currently not indexed3
Indexed0

What we took from it:

  • The split between "Discovered" and "URL is unknown" changed from one day to the next, so we would not read much into which of the two a new page shows.
  • On a young domain, new pages can take weeks to get indexed, even when they are in the sitemap and linked from other pages.
  • The API cannot request indexing. For the pages that matter most, the inspection_result_link in each result opens the page in Search Console, where you can click Request indexing yourself.

What to do for each state

URL is unknown to Google

Make sure the URL is in your sitemap, that Google has read the sitemap recently, and that at least one page Google already indexed links to it. If all of that is true, as it was for us, give it time and request indexing for the most important pages.

Discovered - currently not indexed

Google has the URL in its queue but has not crawled it. Add links from pages that already get search traffic, keep the URL in the sitemap, and check that the server responds quickly. On a new site this often clears up with time.

Crawled - currently not indexed

Google fetched the page and passed on it. This usually means the page is thin or close to another page. Make it clearly more useful, merge it into a stronger page, or link to it from pages that get traffic.

Excluded by noindex, robots.txt, or canonical

These come from your own site, so they are often the quickest fixes.

  • Excluded by "noindex" tag: find the noindex in your templates or CMS and remove it if it is there by mistake.
  • Blocked by robots.txt: find the rule that matches the URL. The robots.txt checker shows whether Googlebot can access a URL under your current rules.
  • Google chose a different canonical: compare google_canonical with user_canonical. Google sees the pages as duplicates, so merge them, make them clearly different, or point your canonical at the version Google picked.
  • Page with redirect or Not found (404) in the sitemap: list the final URL in the sitemap, or fix or remove the page.

A prompt you can copy

Replace the property and sitemap URL with your own. On a large site, add a line that limits the run to the most important URLs, since each inspection counts against the daily quota of 2,000.

Use CalmSEO for my Search Console property sc-domain:example.com.
1. Read the sitemap at https://example.com/sitemap.xml and list every URL in it.
2. Inspect all of them with inspect_gsc_urls, 25 URLs per call. If a call stops early, send its remaining_urls in the next call.
3. Give me a table with the counts per coverage state.
4. For every URL that is not indexed, list the coverage state, the last crawl time, the Google-selected and declared canonical, and the inspection_result_link.
5. Group the unindexed URLs by likely cause and suggest one fix per group.

Run it as a skill

If you use an agent that reads SKILL.md files, the indexing-check skill in CalmSEO's open-source SEO skills repo runs this workflow with the free CalmSEO Search Console tools. It builds the URL list from your sitemaps and from Search Analytics, treating sitemap URLs with no impressions in the last 90 days as the main suspects. It inspects up to about 100 URLs, starting with pages you name, money pages, and recently published pages. It then groups the unindexed URLs by cause, writes a short report, and, if your source code is open, offers to fix robots.txt, meta robots tags, canonical tags, sitemap generation, redirects, and internal links once you agree. To install only this skill:

npx skills add calm-seo/seo-skills --skill indexing-check

Frequently asked questions

Can the URL Inspection API request indexing?

No. The API only reads the inspection result. CalmSEO also connects to Search Console with read-only access. Each result includes an inspection_result_link that opens the URL in Search Console, where you can click Request indexing by hand.

How many URLs can I check per day?

Google allows 2,000 URL inspections per day and 600 per minute for each property. CalmSEO's inspect_gsc_urls tool takes up to 25 URLs per call. If the quota runs out, the result lists the remaining URLs so you can continue the next day.

Does checking indexing status cost credits?

No. inspect_gsc_urls is one of CalmSEO's Search Console tools, and those are free on every plan.

Is the inspection result live?

The result shows Google's indexed version of the page from its last crawl, including the last crawl time. CalmSEO caches each result for one hour, so re-inspecting the same URL right away returns the same data.

How long does it take Google to index a new page?

It varies. On calmseo.com, Google crawled one new page a day after it went live, while 10 pages published in the month before November 11 were still not indexed on that date. Re-inspect important pages a week or two after you publish or fix them.

Check which of your pages Google has indexed

30 free credits + free Google Search Console tools. No card required.