Custom Extraction
Scrape any data from any page using CSS selectors.
Standard SEO data tells you most of what you need - but not everything. Crawly's custom extraction lets you define your own data points using CSS selectors or regex patterns, and run them across every page in the crawl.
Want to extract a custom schema value, a price, a publication date, or a specific element that your CMS outputs? Define a selector, name it, and it appears as a column in the pages table.
How it works
- 1
Before starting a crawl, open the Custom Extraction settings.
- 2
Define one or more CSS selectors (to extract text or attribute values) or regex patterns (to search raw HTML).
- 3
Name each extraction. Run the crawl.
- 4
Each extraction appears as its own column in the pages table - with the extracted value for every URL.
Why it matters
CSS selectors
Extract any text content or attribute value from any element using standard CSS selectors. Results are stored per URL as named columns.
Regex source search
Define regex patterns matched against the raw HTML of every crawled page. Useful for finding specific code patterns, tracking pixels, or script inclusions.
Multiple extractions at once
Stack multiple named selectors in a single crawl. Each becomes its own column in the results - no need to re-crawl for each new data point.
Use cases
Schema validation
Extract specific JSON-LD values to verify that structured data is correctly implemented across every page - without opening each page individually.
Price or stock audits
For e-commerce sites, extract price or stock elements from every product page and spot anomalies or missing data at scale.
Tracking pixel audits
Use regex patterns to verify that analytics tags, consent scripts, or conversion pixels are present on every page that should have them.
Related features
Start crawling smarter
Download Crawly for free. Connect to Claude Code via MCP and start auditing your site in minutes.
Download free