Magento 2 llms.txt Generator
Automatically serve llms.txt, llms-full.txt, and agents.md from your Magento store. Multi-store aware, Composer installable, free.
Install via Composer
The module is available on Packagist. Run these three commands in your Magento root.
$composer require get-crawly/magento2-crawly$bin/magento module:enable Limely_Crawly$bin/magento setup:upgrade && bin/magento cache:flushRequires Magento 2.4+ and PHP 8.1+. Compatible with Luma and Hyvä themes. Once installed, your store automatically serves /llms.txt, /llms-full.txt, and /agents.md.
What it generates
The module serves three AI discovery files, each with a distinct purpose. All are generated dynamically from live store data.
/llms.txtStructured store summary
A plain-text summary of your store for AI assistants. Lists active CMS pages, categories, and optionally products. Each section is independently toggleable from the admin.
/llms-full.txtFull content listing
Always includes all content sections. Products are ordered by best-seller, falling back to the 100 newest products if no sales data exists. Intended for deep indexing by AI crawlers.
/agents.mdAgent instructions
A markdown file for AI agents and automated tools. Auto-detects your platform (including Hyva), lists REST and GraphQL endpoints, links to your sitemap if it exists, and includes guest checkout steps. REST and checkout sections are hidden if anonymous REST access is disabled in your store config.
How it works
All three files are generated dynamically on every request using live store data. No cron jobs, no static file generation.
Multi-store aware
A static file at pub/llms.txt would be identical across all stores. This module uses Magento's store resolver to serve the correct content per domain.
Configurable per store view
Enable or disable each file independently. Control which sections appear in llms.txt, add custom intro text, and append custom markdown to agents.md.
Always up to date
Reflects your live content automatically. Add a category, publish a CMS page, or update your sitemap and the output updates on the next request.
Admin configuration
All settings live at Stores › Configuration › Limely › Crawly. Each file has its own config group and can be enabled independently per store view.
llms.txt Generator
| Setting | Default | Description |
|---|---|---|
| Enabled | Yes | Enable or disable the /llms.txt and /llms-full.txt routes |
| Include CMS Pages | Yes | Add active CMS pages to llms.txt |
| Include Categories | Yes | Add active categories to llms.txt |
| Include Products | No | Add visible, enabled products to llms.txt |
| Custom Introduction | - | Optional text shown below the store name in llms.txt |
| Include Attribution | Yes | Adds an AI Discovery section to all three files crediting Crawly and Limely |
agents.md Generator
| Setting | Default | Description |
|---|---|---|
| Enabled | Yes | Enable or disable the /agents.md route |
| Custom Introduction | - | Optional text inserted after the store name, before the auto-generated content |
| Custom Content | - | Appended at the end of agents.md. Use markdown - good for contact details, brand notes, or agent-specific instructions |
Example output
A sample of what each file looks like with default settings on a typical Magento store.
/llms.txt
# My Magento Store ## Pages - [About Us](https://example.com/about-us) - [Contact](https://example.com/contact) - [Privacy Policy](https://example.com/privacy-policy) ## Categories - [Mens](https://example.com/mens) - [Womens](https://example.com/womens) - [Accessories](https://example.com/accessories) ## AI Discovery This website uses the Crawly Magento 2 Module for AI discovery and structured content indexing. Generated resources may include: - `llms.txt` - `llms-full.txt` - `agents.md` Crawly Magento 2 Module: https://www.getcrawly.com/extensions/magento-2-llms-txt-generator Developed by Limely (https://www.limely.co.uk), a UK ecommerce agency established in 2015, specialising in Magento, Hyva and Shopify.
/agents.md
# My Magento Store
## Platform
- Platform: Adobe Commerce (Magento 2) with Hyva theme
- Version: 2.4.x
## REST API
Base URL: https://example.com/rest/V1
Key endpoints:
- GET /products - list products
- GET /products/{sku} - get product by SKU
- GET /categories - list categories
- POST /carts/mine - create customer cart
- GET /store/storeConfigs - store configuration
Authentication: Bearer token required (obtain via POST /integration/customer/token).
## GraphQL
Endpoint: https://example.com/graphql
Supports: products, categories, cart, customer, CMS pages and blocks.
## Store Resources
Sitemap: https://example.com/sitemap.xml
## Transacting
To place an order as a guest:
1. POST /guest-carts - create a guest cart
2. POST /guest-carts/{cartId}/items - add item
3. POST /guest-carts/{cartId}/shipping-information - set shipping
4. POST /guest-carts/{cartId}/payment-information - place orderAbout
Built by Limely
Limely is a UK ecommerce agency established in 2015, specialising in Magento, Hyvä and Shopify. We build performant, SEO-ready stores for ambitious brands.
limely.co.ukPowered by Crawly
Crawly is a native macOS SEO crawler with Claude Code MCP integration. Free forever, no page cap, no Java required.
getcrawly.comWhat is llms.txt?
llms.txt is a plain-text file placed at the root of your website, at /llms.txt. It gives AI language models and agents a structured, human-readable summary of your site - your brand, what each section covers, and links to key pages.
Unlike a sitemap, which lists URLs for crawlers, llms.txt is written for AI to read and understand. It helps AI assistants, coding agents, and AI-powered search tools give more accurate answers about your content.
Generate an llms.txt for any site