How polling works
A job runs every few minutes, calls the API, compares the result with what it saw last time, and acts on anything new. It works with any API and needs nothing public-facing on your side.
The cost is waste and delay. Most checks find nothing new, yet each one is a request that counts against rate limits and budgets. And a change that lands just after a check waits a full interval before you see it.
How webhooks work
You give the API a URL. When an event happens, such as a new mention or a price change, it sends a POST to that URL with the details. Your server replies 2xx to confirm receipt.
Nothing is spent on empty checks and changes arrive in seconds. In return you need a public HTTPS endpoint, you must verify that requests really come from the sender, and you must handle retries and duplicates.
Side by side
| Polling | Webhooks | |
|---|---|---|
| Freshness | Up to one interval late. | Near real time. |
| Wasted calls | Most checks find nothing. | None. You only hear about real events. |
| What you run | A scheduled job. | A public HTTPS endpoint. |
| Failure mode | A missed run is caught next time. | Needs retries and a way to catch up after downtime. |
| Security | Your outbound calls only. | Must verify each incoming request’s signature. |
Many systems use both: webhooks for speed, and an occasional poll to catch anything missed while the receiver was down.
Verifying a webhook
Because anyone can send a request to a public URL, senders sign each delivery. The common pattern is an HMAC-SHA256 of a timestamp and the raw request body, made with a shared secret. The receiver recomputes the signature, compares it in constant time, and rejects old timestamps to stop replayed requests. The Standard Webhooks spec writes this pattern down.
- Verify against the raw body, before any JSON parsing changes it.
- Reply
2xxquickly and do slow work in the background, or the sender may time out and retry. - Deliveries are usually at least once, so store an event ID and ignore duplicates.
In Monocrawl
Monitors push changes to you
Monocrawl Monitors do the polling for you. A monitor watches a name across platforms, a review or listing feed, or a page, on a schedule you choose, and records only what is new.
Findings can go to the monitor page, email, Slack, Discord or your own webhook, straight away or as a daily or weekly digest. You can also read them through the API.
Webhook deliveries are signed in a monocrawl-signature header: t=<unix time>,v1=<HMAC-SHA256 of "t.body">. Failed deliveries retry on a backoff, and delivery is at least once, so dedupe on the delivery ID.
Common questions
Are webhooks always better than polling?
No. If you need data once a day, or you cannot expose a public endpoint, polling is simpler and perfectly fine.
What happens if my server is down when a webhook fires?
Most senders retry for a while on a backoff. After that, poll the API to catch up on anything you missed.
How often should I poll?
As rarely as your use allows. Match the interval to how quickly the data actually changes, and cache anything you read repeatedly.
Sources