SEO for Beginners: A Complete Guide
SEO gets your pages to rank in Google without paying for ads. This guide covers how search engines work, the three pillars of SEO, and where to start.
In-depth articles on technical SEO, crawling, and getting the most from Crawly.
Claude Code MCP Setup
Connect Crawly's MCP server to Claude Code, Cursor, Codex or Windsurf.
CrawlingCrawl Configuration Guide
Understanding depth, concurrency, rate limits and user agents for accurate crawls.
FeaturesUsing Crawl Comparison
How to diff two crawls and catch regressions before your clients do.
MigrationSEO Site Migration Checklist
Use Crawly to verify redirects, check indexability and protect rankings during a migration.
SEOHow to Run a Technical SEO Audit
A step-by-step walkthrough using Crawly to identify and fix technical SEO issues.
Internationalhreflang: The Complete Guide
Everything you need to know about implementing hreflang for international SEO.
SEO gets your pages to rank in Google without paying for ads. This guide covers how search engines work, the three pillars of SEO, and where to start.
Improve Google rankings by diagnosing what is holding your pages back. This guide covers technical fixes, on-page SEO, internal links, and backlinks.
Free keyword research tools cover most of what beginners and small teams need. Here is how to use Google Search Console, autocomplete, and free tools.
A 301 redirect permanently moves a URL to a new address. Here is how it works, when to use it, and how redirect chains affect SEO and crawl budget.
Indexability determines whether Google can discover, crawl, and add a page to its index. Here is every factor that can prevent a page from being indexed.
Internal links distribute ranking signals and help Google discover pages. Here is how to find orphan pages, fix broken links, and improve anchor text.
A noindex tag tells search engines not to include a page in their index. Here is how it works, when to use it, and the mistakes that cause problems.
Local SEO determines which businesses appear in the Google Local Pack. Here is how it works, what Google's three ranking factors are, and how to improve.
Ecommerce SEO covers category pages, product descriptions, faceted navigation, and crawl budget. Here is how to optimise an online shop that scales.
International SEO ensures search engines serve the right language version to users in each country. Here is how hreflang works and the common mistakes to avoid.
Keyword research identifies the phrases your audience searches for. Here is how to do it step by step, the metrics that matter, and the free tools available.
Mobile-first indexing means Google ranks your mobile page, not desktop. Here is what that means in practice, the common problems, and how to check compliance.
A content audit reviews every page on your site to decide whether to keep, improve, consolidate, or remove it. Here is the full process, step by step.
Topical authority is your site's demonstrated depth of expertise on a subject. Here is how it affects rankings, how to build it, and how content clusters work.
Site architecture is how pages on a website are organised and connected through internal links. Here is why it matters for crawling, ranking, and link equity.
Orphan pages have no inbound internal links. Search engines rarely crawl them and they receive no link equity. Here is how to find and fix them.
E-E-A-T stands for Experience, Expertise, Authoritativeness, and Trust. Here is what each component means, how it affects rankings, and how to demonstrate it.
JavaScript SEO is about whether search engines can access JS-rendered content. Here is how rendering approaches differ and the most common problems to fix.
On-page SEO covers the elements you control on each page: title tags, headings, content, images, and schema. Here is what each one does and how to audit them.
A crawl trap causes search engine crawlers to follow an infinite number of URLs, wasting crawl budget. Here is every trap type and how to detect and fix them.
URL parameters are a major source of duplicate content and crawl waste. Here is how different parameter types affect SEO and how to handle each one correctly.
Link equity is the ranking value passed between pages through hyperlinks. Here is how it flows through a site, how redirects affect it, and how to audit it.
Anchor text is the clickable text of a hyperlink. Here is how Google uses it, the types that matter, and how to audit your internal anchor text distribution.
Thin content fails to satisfy search intent regardless of word count. Here is what Google considers thin, what to do about it, and how to find it.
Duplicate content is a consolidation problem, not a penalty. Here is how Google handles it, what causes it, and how to fix it with canonicals and redirects.
Technical SEO is the foundation every other SEO effort builds on. Here is what it covers, why it matters, and how to audit it properly.
Checking backlinks one at a time is slow. Here is how to check referring domains, backlink counts and authority scores for up to 100 domains simultaneously.
Crawly's free SEO API returns domain authority, referring domains, backlink counts and spam scores for any domain. 100 requests per day, no credit card.
Crawly's free backlink API returns referring domains, individual backlinks with quality ratings, and outbound links for any domain. No credit card required.
Manual SEO reporting is slow. Here is how to automate DA tracking, link prospecting, and backlink audits using Crawly's free SEO API with Python and JavaScript.
GEO is the practice of getting cited by AI search tools: Google AI Overviews, ChatGPT, Perplexity, and Claude. Here is how it works and how to do it.
A SERP is the page Google shows after a search. Here is what it contains, how different result types affect click-through rates, and how to preview yours.
Clean URLs are short, readable, and keyword-focused. Here is how to structure them correctly, what to avoid, and how to change URLs without losing rankings.
Open Graph tags control how your pages look when shared on social media. Here is what the tags are, how to write them, and how to preview your social cards.
UTM parameters track where your traffic comes from. Here is how the five parameters work, how to name them consistently, and how to build them without errors.
Meta descriptions do not affect rankings but directly affect click-through rate. Here is how to write them, what length to aim for, and what Google rewrites.
The title tag is the clickable headline in search results. It is one of the most important on-page SEO signals. Here is how to write one that ranks.
Canonical tags tell search engines which version of a page to index. Here is how rel=canonical works, when to use it, and the mistakes that cause problems.
Core Web Vitals are Google's three page experience metrics: LCP, INP, and CLS. Here is what each one measures, the thresholds, and how to fix common problems.
Schema markup tells search engines what your content means. Here is how it works, which types matter most, and how to implement and validate it correctly.
An XML sitemap lists every URL you want search engines to index. Here is what to include, what to leave out, how to submit it, and the mistakes to avoid.
Spam score is a Moz metric that estimates how likely a domain is to be penalised by Google. What it measures, the thresholds, and what to do about it.
DA is from Moz, DR is from Ahrefs, and Authority Score is from Semrush. They measure the same thing using different data. Here is how to use each one correctly.
Domain authority rises when your backlink profile improves. Here is the order of operations, what moves the score fastest, and how to track your progress.
Domain authority is a score that predicts how well a site ranks in search engines. Here is how it works, what counts as good, and how to check yours.
llms.txt is a plain-text file that tells AI crawlers what your site is about. Here is what it is, whether it actually helps, and how to generate one.
A robots.txt file tells search engine crawlers which pages to visit and which to skip. Here is how it works, common mistakes, and how to check yours.
HTTP status codes tell you exactly what happened when a browser or crawler requested a URL. Here is what each code means and how they affect your SEO.
Checking your backlinks takes minutes with the right tool. Here is a step-by-step guide to seeing every domain that links to your site, for free.
Referring domains is the most important metric in backlink analysis. Here is what they are, how they differ from backlinks, and why the count matters.
A backlink audit tells you which links are helping your site, which are hurting it, and what to do about each. Here is the full process, step by step.
Competitor backlink analysis is one of the fastest ways to find link building opportunities. Here is how to do it properly.
A disavow file tells Google to ignore specific backlinks when assessing your site. Here is when to use one and how to build it correctly.
Link building is the process of earning backlinks from other websites. Here is why it matters, how it works, and the most effective approaches in 2026.
A backlink profile tells you who links to a site, how good those links are, and where risks exist. Here is how to interpret what you see.
Paid backlink tools are good. Free tools are good enough for most audits. Here are the best free backlink checkers available in 2026.
Paid SEO tools are good. Free tools, used correctly, are just as effective for most teams. The best free SEO tools in 2026, organised by what they actually do.
Missing title tags, broken links, redirect chains - these problems accumulate silently. Here is how to do a thorough free SEO website check, step by step.
An SEO crawler visits every page on your site and records what a search engine sees. Here is how they work, what they find, and how to get the most from one.
A canonical tag tells search engines which page to index. Get it wrong and you suppress pages from Google. Here is how to audit canonical tags properly.
SEO spider software crawls your site like Googlebot, recording titles, status codes, and issues across every page. Here's how it works and what to look for.
Crawly's built-in MCP server connects your crawl data directly to Claude Code. Here's how to set it up and what you can ask it.
A step-by-step walkthrough of Crawly's audit workflow: issues, response codes, security headers, hreflang, images, and crawl comparison.
Broken links waste crawl budget, harm user experience, and lose link equity. Here's how to find them all — and track down which pages are causing the problem.
Redirect chains waste crawl budget, slow page load, and dilute link equity. Here's how to find every chain on your site and fix them efficiently.
Crawl budget determines how much of your site Google actually crawls. Here's when it matters, what wastes it, and how to audit it.