How to Automate SEO Reporting with an API

Manual SEO reporting takes hours every month: pulling data from multiple tools, copying numbers into spreadsheets, reformatting for clients. Most of it is mechanical work that can be scripted once and run automatically.

This guide covers how to automate the most common SEO reporting tasks using APIs and basic scripting, with Crawly's free SEO API as the data source for domain authority, backlinks, and spam scores.

What can you automate?

Most recurring SEO reporting tasks follow a simple pattern: fetch data, format it, output it somewhere. The data fetching is the part most people do manually. Automating it requires an API that returns structured data you can process in code.

Tasks that are straightforward to automate:

• Monthly domain authority tracking: query DA and referring domain count for your site and competitors, log the results, chart the trend
• Link prospect filtering: take a list of domains, filter by authority threshold, output a qualified prospect list
• Backlink monitoring: compare your referring domain count month-on-month, flag significant changes
• Spam score auditing: check each referring domain's spam score and flag anything above your threshold
• Client rank reporting: pull keyword rankings from an API, combine with on-site metrics, build a client-ready report

Setting up the Crawly SEO API

Get your free API key at getcrawly.com/api. The key is generated instantly. You get 100 requests per day on the free tier, which covers most monthly reporting workflows.

Three endpoints are available:

Endpoint	Returns
/api/v1/domain-authority	DA score, referring domains, backlink count
/api/v1/backlinks	Full backlink profile with individual referring domains
/api/v1/spam-score	Spam risk score (0–100)

Automating monthly DA tracking

The simplest automated report: query your domain and a set of competitors each month, store the results, and track the trend over time.

import requests
import csv
from datetime import date

KEY = "crawly_live_your_key"
DOMAINS = ["yourdomain.com", "competitor-a.com", "competitor-b.com"]

def get_da(domain):
    res = requests.get(
        "https://www.getcrawly.com/api/v1/domain-authority",
        params={"domain": domain},
        headers={"Authorization": f"Bearer {KEY}"},
    )
    data = res.json()
    return {
        "domain": domain,
        "date": date.today().isoformat(),
        "referring_domains": data["summary"]["referring_domains"],
        "total_links": data["summary"]["total_links"],
    }

results = [get_da(d) for d in DOMAINS]

with open("da-report.csv", "a", newline="") as f:
    writer = csv.DictWriter(f, fieldnames=results[0].keys())
    writer.writerows(results)

print(f"Logged {len(results)} domains for {date.today()}")

Run this script on a schedule (cron on Linux/Mac, Task Scheduler on Windows, or a free service like GitHub Actions). After a few months, the CSV gives you a clean month-by-month comparison.

Automating link prospect filtering

You have a list of 200 domains from a search. You want to keep only those with more than 50 referring domains and a spam score below 30. Doing this manually in a tool takes an hour. Scripted, it takes two minutes to run.

import requests
import time
import csv

KEY = "crawly_live_your_key"
MIN_REFERRING_DOMAINS = 50
MAX_SPAM_SCORE = 30

with open("prospects-raw.txt") as f:
    domains = [line.strip() for line in f if line.strip()]

qualified = []

for domain in domains:
    da_res = requests.get(
        "https://www.getcrawly.com/api/v1/domain-authority",
        params={"domain": domain},
        headers={"Authorization": f"Bearer {KEY}"},
    ).json()

    spam_res = requests.get(
        "https://www.getcrawly.com/api/v1/spam-score",
        params={"domain": domain},
        headers={"Authorization": f"Bearer {KEY}"},
    ).json()

    rd = da_res["summary"]["referring_domains"]
    spam = spam_res["spam_score"]

    if rd >= MIN_REFERRING_DOMAINS and spam <= MAX_SPAM_SCORE:
        qualified.append({
            "domain": domain,
            "referring_domains": rd,
            "spam_score": spam,
        })

    time.sleep(0.3)  # stay within rate limits

with open("prospects-qualified.csv", "w", newline="") as f:
    writer = csv.DictWriter(f, fieldnames=["domain", "referring_domains", "spam_score"])
    writer.writeheader()
    writer.writerows(qualified)

print(f"{len(qualified)} domains qualified from {len(domains)} checked")

Automating client reports with JavaScript

If you are building a client-facing dashboard or a reporting tool in Node.js, the API fits into any async workflow:

const KEY = "crawly_live_your_key";

async function getDomainReport(domain) {
  const [daRes, spamRes] = await Promise.all([
    fetch(`https://www.getcrawly.com/api/v1/domain-authority?domain=${domain}`, {
      headers: { Authorization: `Bearer ${KEY}` },
    }),
    fetch(`https://www.getcrawly.com/api/v1/spam-score?domain=${domain}`, {
      headers: { Authorization: `Bearer ${KEY}` },
    }),
  ]);

  const da = await daRes.json();
  const spam = await spamRes.json();

  return {
    domain,
    referringDomains: da.summary.referring_domains,
    totalLinks: da.summary.total_links,
    spamScore: spam.spam_score,
    spamRisk: spam.risk,
  };
}

const report = await getDomainReport("example.com");
console.log(report);

Scheduling automated reports

GitHub Actions (free, no server required)

Create a .github/workflows/seo-report.yml file in any repository:

name: Monthly SEO Report
on:
  schedule:
    - cron: "0 9 1 * *"  # 9am on the 1st of every month

jobs:
  report:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4
        with:
          python-version: "3.11"
      - run: pip install requests
      - run: python report.py
        env:
          CRAWLY_API_KEY: ${{ secrets.CRAWLY_API_KEY }}

Store your API key as a GitHub secret. The workflow runs your script monthly with no server required.

cron (Mac/Linux)

Add a line to your crontab:

0 9 1 * * /usr/bin/python3 /path/to/report.py

This runs at 9am on the first of every month.

Handling errors and rate limits

The API returns standard HTTP status codes. Build basic error handling into any script that runs unattended:

def safe_get(url, params, headers):
    try:
        res = requests.get(url, params=params, headers=headers, timeout=10)
        res.raise_for_status()
        return res.json()
    except requests.exceptions.HTTPError as e:
        if res.status_code == 429:
            print(f"Rate limit hit. Waiting 60 seconds.")
            time.sleep(60)
            return safe_get(url, params, headers)
        print(f"HTTP error for {params.get('domain')}: {e}")
        return None
    except requests.exceptions.RequestException as e:
        print(f"Request failed: {e}")
        return None

With 100 requests per day, a script checking 50 domains uses the daily allowance in one run. Space out larger jobs across multiple days, or run them at the start of the month when the counter resets.

What to build next

Once you have the data pipeline working, common next steps are:

• Alerting: send a Slack or email notification if a competitor's referring domain count jumps by more than 10% month-on-month
• Trend visualisation: pipe the CSV data into a Google Sheet and use charts to visualise DA trends over time
• CRM integration: push domain metrics into your sales CRM to enrich prospect records automatically

The full API reference covers all three endpoints, response schemas, and error codes.

---

Get your free API key at getcrawly.com/api and start automating your SEO reporting today. No credit card required.