Free web scraper API

A read-only JSON API over every actor in the catalogue β€” 44 production web scrapers with live pricing, usage stats, output fields and FAQ. No authentication, no API key, no sign-up.

What it is for

Ask β€œis there a scraper for X?” in a single request and get back the matching actors, what they cost, what fields they output, and where to run them. Built for AI agents and scripts that need that answer without parsing HTML. The API describes actors β€” it does not run them. Every actor object carries an apify_url; the actor itself executes on the Apify platform.

πŸ†“ No API key πŸ“˜ OpenAPI 3.1 πŸ“ Markdown on every page

Endpoints

Every endpoint is GET, public and unauthenticated.

EndpointReturns
/api Index of every endpoint (send Accept: application/json).
/api/actors All actors. ?q= free-text search, ?category= category slug, ?sort= usersΒ·runsΒ·ratingΒ·name, ?limit= 1–100.
/api/actors/{slug} One actor in full β€” output fields, use cases, steps, pricing detail, FAQ.
/api/categories The 8 category hubs with actor counts.
/api/categories/{slug} One category, its answer block, FAQ and every actor in it.
/api/posts Published guides. ?q=, ?category=, ?limit=.
/api/posts/{slug} One guide, including its full text as markdown.
/api/stats Catalogue totals: actors, runs, users, posts, categories.
/openapi.json OpenAPI 3.1 description of everything above.

Quick start

Find a scraper, then read its output schema

# "Is there a scraper for Amazon?"
curl "https://automationbyexperts.com/api/actors?q=amazon"

# Full detail for one actor
curl "https://automationbyexperts.com/api/actors/bulk-ai-image-generator"

# Everything in one category
curl "https://automationbyexperts.com/api/categories/vehicle-scrapers"

# Catalogue totals
curl "https://automationbyexperts.com/api/stats"

Python

import requests

r = requests.get("https://automationbyexperts.com/api/actors", params={"q": "amazon", "limit": 5})
for actor in r.json()["results"]:
    print(actor["name"], actor["price"], actor["price_unit"], actor["apify_url"])

Markdown for every page

Every public page has a markdown twin. Append .md to its URL, or send Accept: text/markdown on the normal URL β€” both return the same clean markdown with YAML front matter, which is far cheaper for a model to read than the rendered page.

curl "https://automationbyexperts.com/apify/bulk-ai-image-generator.md"
curl -H "Accept: text/markdown" "https://automationbyexperts.com/blog"
curl "https://automationbyexperts.com/index.md"

There is also /llms.txt (short index plus when-to-use guidance) and /llms-full.txt (the entire catalogue in one fetch).

Errors

Errors return application/problem+json with a stable machine-readable error code β€” on every API endpoint, and on any page requested with Accept: application/json.

{
  "type": "https://automationbyexperts.com/api#error-actor_not_found",
  "title": "No such actor",
  "status": 404,
  "error": "actor_not_found",
  "message": "No such actor",
  "detail": "No actor exists with the slug \"nope\".",
  "documentation_url": "https://automationbyexperts.com/api"
}

Codes in use: not_found, actor_not_found, category_not_found, post_not_found, method_not_allowed, gone, internal_error.

Terms of use

  • Free to use, including commercially. No key, no sign-up.
  • Attribution appreciated: β€œAutomationByExperts β€” https://automationbyexperts.com”.
  • Responses cache for 10 minutes; actor stats refresh from Apify every 6 hours.
  • Read-only. There are no write endpoints and no user data of any kind.
  • Questions or higher volume? contact@automationbyexperts.com.

API questions

Do I need an API key?

No. Every endpoint is public, unauthenticated and free.

Can I run a scraper through this API?

No β€” this API describes actors, it does not execute them. Use the apify_url on any actor object to run it on Apify, either from the Apify UI or the Apify API.

How fresh are the run counts and ratings?

They are live figures from the Apify Store, refreshed every six hours and cached for ten minutes at the API layer.

Is there a rate limit?

No hard limit. Please cache responses rather than polling β€” the data changes a few times a day at most.

What if there is no actor for the site I need?

Request one. It is free to submit and most requests are built within a few days.