How it works
- Request the first page with no cursor.
- Read the items and the cursor from the response.
- Request again with
cursor=<that value>. - Repeat until the cursor comes back empty or the API says there are no more results.
The cursor is usually an encoded pointer to the last item you saw: an ID, a timestamp, or a token only the server understands. Treat it as opaque. Store it and send it back exactly as you received it; never build or edit one yourself.
Cursor vs offset pagination
Offset pagination asks for "20 items starting at item 40". It is easy to understand and lets you jump to any page, but it breaks down on lists that change while you read them.
| Offset (page=3, offset=40) | Cursor (cursor=abc123) | |
|---|---|---|
| New items arrive mid-read | Items shift down, so you see duplicates or skip some. | Stable: the cursor points after the last item you saw. |
| Deep pages | The database still counts past every earlier row, so later pages slow down. | Each page costs about the same. |
| Jump to page 50 | Yes. | No. You walk forward page by page. |
| Total count | Often available. | Often unknown until the end. |
That is why social feeds, comment threads and search results almost always use cursors. Stripe and Slack are well-known APIs that page this way.
Pitfalls to avoid
- Assuming an empty cursor means "everything": many sources stop serving history after a point. An empty cursor means there is no next page, not that you have every item ever posted.
- Reusing old cursors: some expire after minutes or hours. Restart from the first page if one is refused.
- Paging too fast: every page is a request and counts against rate limits. Pause between pages on long reads.
- Changing filters mid-loop: a cursor belongs to one query. Changing sort order or search terms needs a fresh start.
In Monocrawl
How Monocrawl pages results
List endpoints return data.items with data.cursor and data.has_more. Send the cursor back unchanged as the cursor parameter for the next page, and stop when it comes back null.
let cursor;
do {
const url = new URL('https://www.monocrawl.com/v1/tiktok/post-comments');
url.searchParams.set('url', videoUrl);
if (cursor) url.searchParams.set('cursor', cursor);
const { data } = await fetch(url, { headers: { 'x-api-key': KEY } }).then(r => r.json());
save(data.items);
cursor = data.cursor;
} while (cursor);Each page is its own request and is billed under that endpoint’s price. Empty pages cost nothing, and a page served from cache costs 0 credits. has_more: null means the source did not say, so the docs call out where history is bounded.
Common questions
Is a cursor the same as a page token?
Yes. APIs call it a cursor, page token, continuation token or next link, but the idea is the same.
Can I run pages in parallel?
Not with cursors, because each page needs the cursor from the page before it. You can run separate lists in parallel instead.
Why can’t I jump straight to page 10?
A cursor only knows where the last page ended. To reach page 10 you read pages 1 to 9 first.
Sources