What this task does
Crawls up to 200 pages of a docs site, limited to the platform and API sections, and produces a single llms-full.txt with every page as clean Markdown. Point AI coding assistants and support bots at one file instead of hundreds of pages. Schedule weekly so it never goes stale. It runs on LLMs.txt & llms-full.txt Generator for Any Website, costs $0.004 per result, and exports to CSV, JSON or Excel.
⚙️ The input this task runs with
These are the exact settings saved in the task. Open it on Apify and they are already filled in, so you can run it unchanged or edit a field first.
{
"websiteUrl": "https://docs.apify.com",
"maxPages": 200,
"includeFullText": true,
"includePatterns": [
"/platform/",
"/api/"
]
}📦 What every run returns
One record per result, with these fields, exportable as CSV, JSON or Excel.
🚀 How to run this task
- Open the task on Apify
Use the button above. New accounts get $5 of free credit and no credit card is required.
- Check the pre-filled input
Every setting shown above is already there. Change a city, a price range or the result limit if you want a different slice of the data.
- Click Run
Pagination, rotating proxies and anti-bot handling all happen in the cloud. The result limit is what controls your cost.
- Export or schedule it
Download CSV, JSON or Excel, or save the task in your own account and schedule it daily or weekly. Webhooks and the Apify API push results straight into your own tools.
💵 What this task costs
🧪 Other LLMs.txt & llms-full.txt Generator for Any Website tasks
Generate llms.txt and llms-full.txt for any website
Paste a homepage and the Actor finds the sitemap, picks the 50 most important pages, and writes a spec compliant llms.txt index plus an llms-full.txt with the complete Markdown content of every page, ready to upload to your site root so ChatGPT, Claude and Perplexity read what you want them to.
See this task →🔗 Related actors
Run it with your own settings
Open the task, change what you need, and save it as your own. The original stays as it is.