Crawl lifecycle

How Crawl Insight walks your site

The audit follows the same lifecycle search engines do — so the report mirrors how Google actually experiences your pages.

1. Discovery

Sitemap parsing, internal-link traversal and orphan-page detection to map every URL that should be reachable.

2. Crawl

Robots, status codes, redirect chains and broken links — anything that interrupts a clean fetch.

3. Render

Mobile-first rendering of HTML, CSS and JS to capture what Google actually sees, not just the source.

4. Index signals

Meta robots, canonicals, hreflang and indexability conflicts that prevent eligible pages from being included.

5. Performance

Core Web Vitals measured under realistic mobile conditions — LCP, INP and CLS scored per page.

6. Trust & structure

HTTPS, mixed content, structured data validity and sitemap integrity rolled into a single health score.

Health scoring

Site-wide technical scorecard

Animated indicators give you a fast read on where the site stands today.

Crawlability92%
Indexability88%
Mobile friendliness96%
Core Web Vitals81%
Structured data validity74%
HTTPS & security99%
Technical checks

Every layer of the technical stack, audited

CR

Crawlability

Robots rules, disallow patterns, login walls and pages search bots simply cannot reach.

IX

Indexability

Meta robots, canonicals and conflicts that quietly remove eligible pages from the index.

BL

Broken links

Internal 404s, dead outbound links and patterns that erode crawl budget over time.

RD

Redirect issues

Loops, chains, mixed protocols and 302s where 301s belong.

CV

Core Web Vitals

LCP, INP and CLS measured on a realistic mobile profile with actionable causes per page.

MO

Mobile friendliness

Viewport, tap targets, font scaling and rendering verified mobile-first by default.

SD

Structured data

Schema validation, eligibility for rich results and conflicts across templates.

XM

XML sitemap

Coverage, freshness, orphan vs. indexed mismatches and invalid entries.

HT

HTTPS checks

Certificate validity, mixed content and protocol consistency across the site.

Crawl budget

Where your crawl budget is actually going

For large sites, every wasted crawl is a missed indexing opportunity. Crawl Insight shows you which URLs are absorbing crawl attention — redirects, near-duplicates, faceted noise — so you can free that budget for pages that should rank.

Fix the technical foundations holding your rankings back.

Get a full Crawl Insight technical audit and a prioritised remediation plan for crawl, index, performance and structured data.

Request technical audit