Why APIs limit requests
Every API runs on finite servers, databases and upstream quotas. Without a cap, one busy script could slow the service for everyone else, run up costs, or trip the limits of the sources the API depends on.
Rate limits keep usage fair between customers, protect the service from runaway loops and abuse, and make capacity predictable. They also give clients a clear signal: slow down now, try again later.
How limits are counted
Most APIs count requests per API key, per account, per IP address, or some mix of these. The counting method changes how a burst of traffic is treated.
| Method | How it counts | What it feels like |
|---|---|---|
| Fixed window | A counter per clock minute or hour that resets at the boundary. | Simple, but two bursts either side of a reset can briefly double the rate. |
| Sliding window | Counts requests over the last N seconds, measured from now. | Smooth. No reset-boundary spikes. |
| Token bucket | Tokens refill at a steady rate; each request spends one. | Allows short bursts up to the bucket size, then settles to the refill rate. |
| Concurrency cap | Limits requests in flight at the same moment, not per minute. | Protects slow endpoints. Long calls hold a slot until they finish. |
Many APIs apply two limits together: a rate per minute and a cap on simultaneous requests. Hitting either one is enough to be refused.
The headers that tell you where you stand
Well-behaved APIs report your budget on every response, so a client can slow down before it is refused. Names vary between providers, but the ideas are the same:
- Limit: the ceiling for the current window.
- Remaining: how many requests are left in it.
- Reset: when the window refills, often as a Unix timestamp.
- Retry-After: sent with a refusal, the number of seconds to wait before trying again. It is defined in HTTP itself, in RFC 9110.
The IETF is working on standard RateLimit headers so every API can report this the same way, but most APIs still use their own X-RateLimit-* names today.
How to stay under a limit
- Read the remaining-requests header and pause before it reaches zero.
- Cap your own concurrency with a small worker pool instead of firing every request at once.
- On a refusal, wait for
Retry-Afterif it is present. Otherwise back off exponentially with random jitter, so many clients do not retry in lockstep. - Cache responses you read often. A repeated read that is served from cache often does not count against quotas at all.
- Put a ceiling on retries so a long outage fails cleanly instead of looping.
In Monocrawl
How Monocrawl limits requests
Monocrawl counts requests over a sliding 60-second window and also caps how many run at once. Each API key has its own ceiling, and the account has a higher shared one, so several keys can run side by side. REST, the MCP server and the dashboard draw from the same buckets.
| Plan | Per API key | Per account |
|---|---|---|
| Free | 600 a minute, 50 at once | 600 a minute, 50 at once |
| Starter | 600 a minute, 50 at once | 1200 a minute, 100 at once |
| Pro | 1200 a minute, 75 at once | 1800 a minute, 125 at once |
| Growth | 1800 a minute, 100 at once | 2400 a minute, 150 at once |
| Business | 2400 a minute, 150 at once | 3600 a minute, 200 at once |
Every response carries x-ratelimit-limit, x-ratelimit-remaining and x-ratelimit-reset, plus the concurrency equivalents. A refused call returns 429 with error type RATE_LIMITED and a retry-after header, and nothing is charged for it.
Common questions
Is a rate limit the same as a quota?
Not quite. A rate limit controls speed, such as requests per minute. A quota caps total use over a longer period, such as calls per day or credits per month. An API can have both.
Do limits apply per key or per account?
It depends on the API. Many count per key and per account at the same time. In Monocrawl each key has its own ceiling and the account has a higher shared one.
What happens if I keep sending requests after a 429?
They keep being refused, and some APIs extend the block or flag the client. Wait for the reset time before sending more.
Sources