Read as Markdown · Use with an AI agent
Runnable platform workflows
Collect a bounded set of records, open selected results, and retrieve their comments, replies or reviews. These scripts pass identifiers from the actual responses into subsequent calls.
Download and set a budget
Download monocrawl-client.mjs and platform-workflow.mjs into the same folder. They require Node.js 20 or later and no packages. Read the scripts before running them.
Create a dedicated API key with a credit limit in the dashboard. For a new key and a 20-credit job, set its limit to 20. For an existing key, the limit is cumulative: account for credits it has already used. Keep this key exclusive to this job if you need a predictable remaining allowance.
The script reads current availability and prices from the public OpenAPI catalogue, then checks a local budget before each request. This estimate can race with a price change. The key’s server-enforced credit limit supplies the hard spending ceiling. Direct REST does not expose MCP’s max_credits argument. Each new page or detail lookup can cost credits.
export MONOCRAWL_API_KEY='YOUR_DEDICATED_CAPPED_KEY' export MONOCRAWL_BUDGET=20 node platform-workflow.mjs tiktok nasa tiktok-results.json
$env:MONOCRAWL_API_KEY = 'YOUR_DEDICATED_CAPPED_KEY' $env:MONOCRAWL_BUDGET = '20' node platform-workflow.mjs reddit "standing desk" reddit-results.json
Use a new output filename for each run. The file contains collected public content and request recovery keys; keep it out of source control and serve neither it nor your API key to a browser.
Six connected journeys
| Command arguments | What flows between requests |
|---|---|
tiktok nasa | Profile → account videos → returned video URL → post details → comments. |
instagram nasa | Profile → account posts → returned post URL → post details → comments. |
reddit "standing desk" | Search → returned discussion URL → full post → bounded comment tree. |
youtube googledevelopers | Handle → resolved channel ID → uploads → returned video ID/URL → video statistics → comments. |
x nasa | Profile → account posts → returned post URL → post details → resolved post ID → replies. |
amazon "standing desk" | Search → selected ASIN → product → reviews, retaining the same marketplace throughout. Set MONOCRAWL_COUNTRY to change the GB default. |
These are separate runs. The script defaults to at most two feed pages and two detail records, then one comments/replies/reviews response per selected record. Amazon search is a single-page workflow. Change maxPages and maxDetails only after reviewing the increased spend; the exported runWorkflow function accepts both.
Results are deduplicated by ID, ASIN or URL within the run. Records without an identifier are retained; records without a usable lookup identifier are marked as skipped. Cursors are followed only on the selected pageable feeds, with all original filters retained. Repeated cursors stop the loop.
Read the saved result and stop reason
The output preserves the original envelopes, credit counts and warnings in profile, pages and details. It also stores deduplicated items, next_cursor, stop_reason and a request journal containing the exact parameters and idempotency key before each send. Each checkpoint replaces the output atomically so an interrupted write preserves the previous complete file.
| Value | Meaning |
|---|---|
page_limit | The requested page budget was reached. A next cursor may still be present. |
no_cursor | No usable continuation was supplied; this does not establish complete source history. |
source_exhausted | This endpoint explicitly reported has_more:false for the traversal. |
repeated_cursor | The source repeated a token; inspect the saved pages before attempting further retrieval. |
single_page_endpoint | The workflow deliberately makes one request to this non-pageable operation. |
status: stopped | A budget, availability, request or response check stopped the job. Earlier results remain in the output. |
status: finished means this bounded script finished. It does not mean all posts, comments or reviews were collected. Inspect Reddit’s unexpanded replies, YouTube reply previews and platform-specific warnings before interpreting coverage.
Recover an interrupted request deliberately
The client saves an idempotency key before each request and reuses it for its bounded retries. If billing is unresolved, it stops further spending. Keep the saved key, request ID and original path/parameters together. Check usage and retry only that same logical request when appropriate; an idempotent replay retains the original credits_used and historical balance.
Running the CLI again creates a new job and new keys. It is not an automatic resume. Before rerunning, reconcile any interrupted request using its saved checkpoint; for a durable production job, move those checkpoints and completed results into your job store. See the production guide and testing guide.