Web Scraping API

Web Scraping API for
website discovery, scraping and extraction.

Map a site, choose source URLs and turn selected public pages into records you can inspect.

Try the Web Scraping API See a real response

150 free credits to start·No card required

GET/web/map

monocrawl / web

Recorded response

IANA website links from a Web map request

url: https://www.iana.org/
origin: https://www.iana.org
count: 10
limit: 10
first link: https://www.iana.org/about
second link: https://www.iana.org/domains
third link: https://www.iana.org/domains/root

Selected public fields2026-09-21

Recorded excerpt. Values may change; missing fields stay null.

Structured JSONData you can work with

One API keyYour Monocrawl integrations

cURL · TypeScript · PythonStart in your own stack

The data

Web Scraping data,
ready to work with.

A web map is a starting list of discovered URLs, not the text of those pages. The recorded request to iana.org returned ten links at a limit of ten, including the about and domains sections. Choose a URL before calling scrape for page content or extract for structured page metadata. Crawl and batch-scrape run larger jobs with their own status and continuation; their presence does not make every site accessible. The API also exposes sessions and job controls for longer workflows. Retain final URLs, page status and collection dates so downstream research can distinguish discovered links from content actually retrieved.

URL discovery

List returned links for a public site.

site URL → links

Page content

Read a selected page into the requested output format.

page URL → content

Page metadata

Extract returned headings, metadata and content signals.

page URL → fields

Site crawl

Start a broader crawl job with its own status.

site URL → job

Multiple pages

Submit a selected URL set as one batch job.

URLs → job

Job status

Check a previously submitted job by ID.

job ID → status

Request → response

From Web Scraping
to structured JSON.

Inspect selected fields from a recorded Web Scraping response for IANA website links from a Web map request. Choose a language and use your own key to request current data.

1
Get your Monocrawl key
Create an account and collect your key.
2
Choose your inputs
Replace the example inputs with your own target. Send your key in the x-api-key header.
3
Use the returned JSON
Read the returned record from data. Check credits_used for the call’s confirmed credit usage.

Your requestGET /v1/web/map

curl "https://www.monocrawl.com/v1/web/map?url=https%3A%2F%2Fwww.iana.org&limit=10&sitemap=skip" \
  -H "x-api-key: mn_your_key_here"

What comes backdata excerpt

{
  "url": "https://www.iana.org/",
  "origin": "https://www.iana.org",
  "count": 10,
  "limit": 10,
  "items": [
    "https://www.iana.org/about",
    "https://www.iana.org/domains",
    "https://www.iana.org/domains/root"
  ]
}

Recorded 21 Sep 2026 for IANA website links from a Web map request. Selected fields only; this is a historical snapshot.

Recorded response for the inputs shownTry the Web Scraping API

What you can build

What you can build
with Web Scraping data.

Use Web Scraping data in the products and workflows you’re already building.

Site URL
Discovered links
Your source list

Map a website

Discover returned URLs before deciding which pages deserve inspection.

Explore this workflow

Page URL
Page content
Your content record

Scrape a selected page

Request content from an individual public URL and inspect its status.

Explore this workflow

Page URL
Extracted metadata
Your structured record

Extract page metadata

Read returned headings and metadata for a selected URL.

Explore this workflow

Start with a working workflowIntegration guide

Web Scraping API pricing

Try the Web Scraping API for free.

Start with 150 free credits. Choose a paid plan when you need a monthly allowance.

Free

No card required

$0/ month

150 free credits to start

Try the endpoints your application needs, with your own API key.

Email verification required. Free usage limits apply. See limits

Starter

Monthly subscription

$29/ month

10,000 credits every month

More monthly credits for your Web Scraping integration and other supported platforms.

Explore the Starter plan

USD, before applicable tax. See pricing for subscription terms.

Compare all plans

Explore by data type

Web Scraping API endpoints

16 currently available operations, grouped by the data you need. Open an endpoint for parameters, response details and pricing.

Filter Web Scraping endpoints

16 of 16 endpoints

Discover URLs2

Async crawls & jobs6

Read pages3

Browser sessions5

Listed credit prices are per request. Metered or variable-cost operations can settle differently; check the endpoint reference when budgeting.

Web Scraping documentation Try the Web Scraping API

Before you build

Web Scraping API questions,
answered.

Need to check a particular workflow?
Talk to us

Comparing providers for Web Scraping data?
Web scraping APIs compared

Does map scrape every discovered page?

No. The IANA example only returns URLs. Call scrape or extract on selected pages to retrieve their content or metadata.

Will every website return content?

No. A saved scrape request in this benchmark was rejected by its target site. Handle blocked pages and inspect returned status rather than assuming universal access.

What Web Scraping operations can I use?

The catalogue shows 16 currently available operations with their individual prices. Open an endpoint for its accepted inputs, returned fields and limitations.

How much does the Web Scraping API cost?

Start with 150 free credits. Paid subscriptions start at $29 per month for 10,000 credits, before applicable tax. Each operation has its own price. Check the endpoint reference for variable charges. Compare plans

How do I authenticate requests?

Use your Monocrawl API key in the x-api-key header. The endpoint reference lists any additional inputs needed for the operation. Authentication guide

What request limits apply?

Limits depend on your Monocrawl plan. The Free account ceiling is 600 requests per minute and 50 simultaneous requests. Endpoint capacity and Free daily protections also apply; these ceilings do not guarantee source throughput. Read the limits guide

What should I know about Web Scraping coverage?

Mapping discovers links; it does not scrape every page or guarantee full-site coverage. Some sites reject requests or return incomplete content. Check status and warnings for each URL.