Technical SEO is about making sure search engines can reach, understand and choose the right version of each page. For most business websites that means six checks: crawl access, indexing, canonical URLs, a sitemap, correct status codes and acceptable page experience. Fix those in order, because a problem early in the chain, such as blocked crawling, makes later improvements irrelevant.
1. Crawl access
Google’s robots.txt introduction explains that robots.txt tells crawlers which parts of a site they may access and mainly helps manage crawler traffic. It is not a way to keep a page out of Google: a blocked URL can still appear in results, often without a description, if other sites link to it. To keep a page out of Search use noindex or password protection.
Checklist:
- robots.txt exists, loads and does not block important sections, scripts or styles the page needs to render
- staging or private areas are protected by login, not only by robots.txt
- no accidental “disallow everything” left over from development
- your CDN or host is not blocking search crawlers
2. Indexing
- Important pages return a successful response and are not marked noindex.
- Each key page is reachable through internal links, not only the sitemap; see internal linking strategy.
- Search Console shows pages as indexed, and you understand why others are excluded.
- Do not combine robots.txt blocking with noindex on the same URL: if the page is blocked, Google cannot see the noindex.
Google notes that eligibility does not guarantee indexing, so aim for useful, distinct pages rather than volume.
3. Canonical URLs and duplicates
Google’s guide to consolidating duplicate URLs ranks the methods from strongest to weakest: redirects, rel=“canonical” annotations, then sitemap inclusion. Recommendations include:
- add a self-referential rel=“canonical” to canonical pages
- use absolute URLs in canonical tags
- link internally to the canonical URL, not to duplicates
- prefer HTTPS, redirecting HTTP to HTTPS
- on JavaScript-rendered sites, put the canonical in the HTML source
Things to avoid: using robots.txt to select a canonical, using noindex to choose a canonical within your own site, and sending conflicting signals, such as a sitemap URL that differs from the page’s canonical tag.
Common duplicate sources: www versus non-www, trailing slashes, tracking parameters, filtered or sorted lists and printer versions. See URL structure and redirects.
4. Sitemaps
Google’s sitemap overview says you may need a sitemap if your site is large, new with few external links, or has many video or image files. It says a site of about 500 pages or fewer may not need one if it is well linked internally and light on media or news content, and that a sitemap helps discovery without guaranteeing every URL is crawled or indexed.
Checklist:
- list only canonical, indexable, successful URLs
- keep it updated automatically
- reference it in robots.txt and submit it in Search Console
- remove redirected, noindexed and error URLs
5. Status codes and redirects
Google’s page on HTTP status codes describes how each is handled:
| Status | How Google treats it |
|---|---|
| 301 | Followed, a strong signal that the target should be processed |
| 302 / 307 | Followed, a weaker signal |
| 404 / 410 | Content does not exist; indexed URLs are eventually removed |
| 5xx / 429 | Google slows crawling; persistent errors lead to URLs being dropped |
| 200 with error content | A soft 404, reported in Search Console |
Checklist:
- old URLs redirect with 301 to the closest relevant page, not all to the home page
- avoid redirect chains
- genuine missing pages return a real 404 or 410 status with a helpful page for visitors; see broken links and 404 pages
- server errors are monitored and fixed quickly
6. Page experience and speed
Google’s Core Web Vitals documentation lists three metrics and what counts as good: Largest Contentful Paint within 2.5 seconds, Interaction to Next Paint under 200 milliseconds and Cumulative Layout Shift below 0.1. Google recommends achieving good scores, while describing them as one part of page experience rather than a sole ranking determinant. Our Core Web Vitals guide explains measurement and fixes.
Quick audit order
- Can Google fetch the page? (robots.txt, status code)
- Is it allowed to index? (noindex, canonical)
- Is it the preferred version? (redirects, canonical, internal links)
- Is it discoverable? (sitemap, internal links)
- Is it usable? (mobile, speed, Core Web Vitals)
- Is it monitored? (Search Console, alerts after releases)
Mistakes to watch after a redesign
Most technical SEO damage happens at launch: a leftover noindex, a robots.txt copied from staging, URLs changed without redirects, canonical tags pointing at the staging domain. Add these to your launch checklist and re-check a week later.
Limits
This list covers fundamentals. It does not guarantee rankings, which depend on many factors including content quality and competition. Rules and tools change, so verify against Google Search Central periodically.
Where Radiance Impact fits
Radiance Impact’s Growth team works on search visibility and its Technology team on website builds and maintenance. See Growth or Technology, or contact us.
Sources
- Introduction to robots.txt, Google Search Central
- How to specify a canonical URL, Google Search Central
- Sitemaps overview, Google Search Central
- HTTP status codes and network errors, Google Search Central
- Understanding Core Web Vitals, Google Search Central
