Field notes
Technical SEO fix guides
These are not ranking tips. They are cases where your site tells a crawler two contradictory things and the contradiction is invisible from a browser: a sitemap recommending pages that say noindex, a robots.txt blocking the noindex it was meant to enforce, a missing page answering 200. Each note explains the mechanism and gives you a script that checks it.
Every problem here is invisible in a browser. Status codes are not displayed, canonical tags are not rendered, and robots.txt is a file nobody opens. A page can be excluded from indexing in one place and recommended in another for years, because the two sources are written by different systems and nothing joins them up. The only way to see it is to ask the way a crawler asks.
Technical SEO
Your sitemap lists URLs that redirect, 404 or say noindex
A sitemap is a recommendation. Listing a noindex or 404 URL contradicts itself, and Search Console reports it as an error.
Robots.txt blocks the noindex it was meant to enforce
Disallow stops the crawl, so the noindex on the page is never read. The URL can still be indexed, without a snippet.
A canonical tag pointing at staging, a redirect or nothing
Canonicals are generated from a base URL. When that base is wrong, every page on the site quietly nominates the wrong URL.
A missing page that returns 200 instead of 404
Client-side routing and catch-all handlers answer every URL with 200. Google calls it a soft 404 and it fills the index with nothing.
These are the ones I keep hitting. If yours is broken in a way none of them describes, tell me on LinkedIn — it is usually how the next note gets written.