The request contract

Paths

How a path resolves

The first segment after /v1 is the platform; everything after it is the endpoint id. Some endpoints take their identifier in the path rather than the query string, and the router recognises both call styles.

A path segment is treated as an identifier rather than a route word when it contains a digit or an underscore, or runs past 24 characters — so job_9f3a and 12345 are ids while runs and checks are route words. An unknown path answers 404 ENDPOINT_NOT_FOUND.

Envelope

One success shape, one error shape

Both shapes always carry credits_used and request_id, for standard routed JSON responses. Error responses include credits_remaining only when the balance is known. Batch has a separate wrapper and per-item envelopes; streamed searches carry an envelope in their final result event.

{
  "success": true,
  "platform": "github",
  "endpoint": "/v1/github/profile",
  "data": { … },
  "credits_used": 1,
  "credits_remaining": 99,
  "request_id": "req_4f2b8c1d09ae37b562",
  "cached": false
}

{
  "success": false,
  "error": {
    "type": "RESOURCE_NOT_FOUND",
    "message": "GitHub returned 404 for this resource.",
    "status": 404,
    "doc_url": "https://www.monocrawl.com/docs/errors#resource-not-found"
  },
  "credits_used": 0,
  "credits_remaining": 99,
  "request_id": "req_71ac0e5b39d8f2c604"
}

error.doc_url points at the exact section of the errors page for that error type — a client can surface it verbatim.

Pagination

Cursors, not page maths

Many list endpoints return items, count and cursor inside data. Where supported, pass the cursor back with the original filters. A null cursor means no usable continuation was supplied, not proof that all history was retrieved. Cursors are opaque — some are page numbers, some are upstream tokens — so echo the value you were given rather than deriving the next one.

$ curl "https://www.monocrawl.com/v1/github/profile/repos?handle=torvalds&limit=50" \
    -H "x-api-key: mn_your_key_here"

"data": {
  "items": [ … 50 rows … ],
  "count": 50,
  "cursor": "2"
}

# continue with the cursor exactly as it was given to you
$ curl ".../v1/github/profile/repos?handle=torvalds&limit=50&cursor=2" -H "x-api-key: …"

Page-size parameters, defaults and caps vary by endpoint. Some routes use pageor do not support continuation. Each page is a separate request under that endpoint’s billing rules; a larger requested limit is not a guarantee of more results or a lower price.

Batch

POST /v1/batch — up to 20 calls at once

The batch endpoint takes a list of ordinary requests and runs each one through the full pipeline independently. Items may target different platforms. The wrapper itself costs nothing — you pay only for the calls inside it.

curl -X POST "https://www.monocrawl.com/v1/batch" \
  -H "x-api-key: mn_your_key_here" \
  -H "content-type: application/json" \
  -d '{
    "requests": [
      { "platform": "github",  "endpoint": "profile", "params": { "handle": "torvalds" } },
      { "platform": "bluesky", "endpoint": "profile", "params": { "handle": "bsky.app" } }
    ]
  }'

{
  "success": true,
  "count": 2,
  "succeeded": 2,
  "failed": 0,
  "credits_used": 2,
  "credits_remaining": 97,
  "results": [
    { "index": 0, "status": 200, "success": true, "platform": "github",
      "endpoint": "/v1/github/profile", "data": { … }, "credits_used": 1,
      "credits_remaining": 98, "request_id": "req_…", "cached": false },
    { "index": 1, "status": 200, "success": true, "platform": "bluesky",
      "endpoint": "/v1/bluesky/profile", "data": { … }, "credits_used": 1,
      "credits_remaining": 97, "request_id": "req_…", "cached": false }
  ]
}

The top-level credits_used is the sum across items; credits_remaining, when present, is the lowest balance any item observed while the batch ran. For an exact figure afterwards, call /v1/credits/balance. Batch items are admitted individually, so each one counts against the per-key rate limit. The wrapper does not return per-item HTTP headers or a top-level request_id. Idempotency-Key and Cache-Control headers are not forwarded to items.

Discovery

Read the catalogue at runtime

These endpoints describe the API to a client without any hard-coded list. They all cost 0 credits, so a service can refresh its view of the catalogue on every boot.

utility/llms is the fastest way to teach an agent the whole surface: one call returns the base URL, the auth header, both envelope shapes, the error type list and every endpoint with its cost and parameters.

For humans, the browsable endpoint reference lists every operation with its parameters and a ready-to-run curl. For tooling, /openapi.json (or /openapi.yaml) is the same registry as an OpenAPI 3.1 document — import it into Postman, Insomnia or a client generator. These surfaces share the registry but expose different subsets and metadata; registration and recorded proof are not a guarantee of current availability.

Headers

What comes back on the wire

Early authentication refusals, batch wrappers, SSE and idempotent replays may omit some headers. Rate and concurrency headroom are not duplicated in the body. Read credits_remaining when present, or call /v1/credits/balance for free.

Idempotency

Retry a lost request without paying twice

If a response is lost — a timeout, a dropped connection, a crash between the call and reading it — you cannot tell whether the request landed. Retrying blindly risks paying twice for one result. Send an Idempotency-Key header on the first direct GET request and reuse it on retries. A stored replay adds no charge. This header does not protect batch items or MCP calls. If the idempotency store is unavailable, the call proceeds without replay protection and returnsx-idempotency-status: unavailable; do not assume an automatic retry is deduplicated.

# first call: runs, and is charged
$ curl "https://www.monocrawl.com/v1/github/profile?handle=torvalds" \
    -H "x-api-key: mn_your_key_here" \
    -H "Idempotency-Key: 8f3a1c2e-order-4471"

x-credits-used: 1
{ "success": true, "data": { … }, "credits_used": 1, "request_id": "req_4f2b8c…" }

# the response was lost — retry with the SAME key
$ curl "https://www.monocrawl.com/v1/github/profile?handle=torvalds" \
    -H "x-api-key: mn_your_key_here" \
    -H "Idempotency-Key: 8f3a1c2e-order-4471"

x-idempotent-replay: true
{ "success": true, "data": { … }, "credits_used": 1, "request_id": "req_4f2b8c…" }
# ^ the stored original result. No new charge; its balance is historical.

If a second request arrives while the first is still in flight, the second gets 409 with details.reason = idempotency_in_progress — wait for the first rather than racing it. The header is entirely optional; without it nothing about a request changes.

Webhooks

Signed events pushed to your server

Register an https URL in the console (Dashboard → API → Integrations) and Monocrawl POSTs you a signed JSON event when something happens on your account — an async job finishing, a failed job and its refund status, a low balance or an auto-recharge outcome. Up to five endpoints per account, each with its own signing secret and its own event filter.

POST https://example.com/monocrawl/webhook
content-type: application/json
monocrawl-event: job.completed
monocrawl-delivery: whd_8c1d09ae37b562
monocrawl-signature: t=1767225600,v1=5f8a…e2c1

{
  "id": "whd_8c1d09ae37b562",
  "type": "job.completed",
  "created_at": "2026-01-01T00:00:00.000Z",
  "data": { "job_id": "job_9f3a", "kind": "crawl", "credits_charged": 40 }
}

Verify the signature

Every delivery carries monocrawl-signature: t=<unix seconds>,v1=<hex>, where v1 is HMAC-SHA256 over `${t}.${rawBody}` with your endpoint's secret. Verify before trusting anything in the body — anyone can POST to a public URL, a valid signature proves possession of the signing secret. Keep that secret private.

import crypto from 'node:crypto';

// rawBody is the original Buffer (or unmodified UTF-8 string), BEFORE JSON parsing.
export function verifyMonocrawlSignature(rawBody, header, secret, nowMs = Date.now()) {
  const match = /^t=(\d+),v1=([0-9a-f]{64})$/.exec(header ?? '');
  if (!match || !secret) return false;
  const [, timestamp, signature] = match;
  const seconds = Number(timestamp);
  if (!Number.isSafeInteger(seconds) || Math.abs(nowMs / 1000 - seconds) > 300) return false;
  const expected = crypto.createHmac('sha256', secret)
    .update(timestamp + '.').update(rawBody).digest();
  const received = Buffer.from(signature, 'hex');
  return received.length === expected.length && crypto.timingSafeEqual(expected, received);
}

Event types

Rotating a secret takes effect immediately — deliveries signed with the old secret stop verifying at that moment, so rotate when your receiver is ready for the new one. The "Test" button in the console sends a test.ping synchronously and shows you exactly what your endpoint answered.