# Cohorts and keyword sweeps

Save a bounded set of communities or accounts, run a keyword sweep, and inspect outcomes for each member. A completed sweep is a sample, not exhaustive history.

## Choose supported sweep members

| Platform | Identifier | Search behavior |
| --- | --- | --- |
| Reddit | Subreddit, such as rust | Keyword search ordered by new. |
| Hacker News | Username | Search with an author restriction. |
| GitHub | Owner or owner/repository | Search within the user or repository. |
| Bluesky | Account handle | Search posts from the account. |

These four platforms have sweep adapters. Members on other accepted catalogue platforms can be stored but report ok:false during a sweep. Inspect sweepable_count and warnings first. Membership does not verify identity or merge people across platforms.

Matching is case-insensitive substring matching over returned item text. A multi-word keyword can also match when every word occurs somewhere in the item. This is not semantic relevance or exact-phrase-only matching. Source filters also constrain the retrieved window.

## 1. Create and inspect a cohort

The current canonical REST operations below use GET, including named creation and deletion operations. They have effects despite the method: invoke explicitly from your server, not from prefetching links. Use header authentication and check the current price before creation or submission.

**curl · create once**

```
curl -G 'https://www.monocrawl.com/v1/cohorts/create' \
  -H "x-api-key: $MONOCRAWL_API_KEY" \
  -H 'Idempotency-Key: REPLACE_WITH_SAVED_CREATE_KEY' \
  --data-urlencode 'name=Rust discussions' \
  --data-urlencode 'members=[{"platform":"reddit","identifier":"rust"},{"platform":"github","identifier":"rust-lang/rust"}]'
```

Check success and the credit receipt, then save data.id as cohort_id. The cohort includes name, members, member_count, sweepable_count and created_at. Inspect it with cohorts/get and cohort_id. Replace example IDs below with returned IDs.

- [Create parameters and price](https://www.monocrawl.com/docs/endpoints/cohorts/create)

- [Inspect membership](https://www.monocrawl.com/docs/endpoints/cohorts/get)

## 2. Submit once, then poll

**curl · submit a keyword sweep**

```
curl -G 'https://www.monocrawl.com/v1/cohorts/queries' \
  -H "x-api-key: $MONOCRAWL_API_KEY" \
  -H 'Idempotency-Key: REPLACE_WITH_SAVED_QUERY_KEY' \
  --data-urlencode 'cohort_id=YOUR_COHORT_ID' \
  --data-urlencode 'keyword=async runtime'
```

Save data.id as query_id and retain job_id. Poll cohort-queries/get with query_id at a bounded interval. A lost submission response does not prove no work was queued: retain the original key and identical parameters while reconciling. A new submission can duplicate work and spend.

| Status | Next step |
| --- | --- |
| queued / running | Wait and poll the saved query. A client deadline stops waiting, not server execution. |
| completed | Read results and inspect per-member failures and warnings. |
| failed | Inspect error and the associated job refund_status; due means pending, not already credited. |

**curl · status**

```
curl -G 'https://www.monocrawl.com/v1/cohort-queries/get' \
  -H "x-api-key: $MONOCRAWL_API_KEY" --data-urlencode 'query_id=YOUR_QUERY_ID'
```

- [Submission contract](https://www.monocrawl.com/docs/endpoints/cohorts/queries)

- [Status contract](https://www.monocrawl.com/docs/endpoints/cohort-queries/get)

- [Request recovery](https://www.monocrawl.com/docs/api-reference#idempotency)

## 3. Read retained result pages

**curl · results**

```
curl -G 'https://www.monocrawl.com/v1/cohort-queries/results' \
  -H "x-api-key: $MONOCRAWL_API_KEY" \
  --data-urlencode 'query_id=YOUR_QUERY_ID' --data-urlencode 'limit=25'
```

| Field under data | Meaning |
| --- | --- |
| ready / status | HTTP 200 can carry ready:false and empty items while unfinished or failed. That does not mean no matches. |
| items / count / total | This page, its length and the retained total, not every match on the source. |
| cursor | Pass unchanged with the same query_id. Null ends the retained pages. |
| counts | members, swept, ok, failed, returned and matched summarize bounded work. |
| per_member | Inspect ok, returned/matched details, errors and truncation. |
| _warnings | Time, source-window and result-size limitations. Preserve alongside exports. |

**Illustrative abbreviated result**

```
{
  "success": true,
  "credits_used": 0,
  "data": {
    "query_id": "cq_example",
    "status": "completed",
    "ready": true,
    "items": [],
    "count": 0,
    "total": 0,
    "cursor": null,
    "counts": {
      "members": 2,
      "swept": 2,
      "ok": 1,
      "failed": 1,
      "returned": 10,
      "matched": 0
    },
    "_warnings": [
      "One member could not be swept; inspect per_member."
    ]
  }
}
```

The example omits per_member rows for brevity. Completion can include failed or unattempted members. Retain original IDs and URLs; an empty bounded sample does not prove a topic was never discussed.

## Limits, membership and cleanup

| Control | Current behavior |
| --- | --- |
| Membership | 200 cohorts/account; up to 100 supplied members/cohort, normalized and deduplicated. |
| Pending work | 25 queued/running queries/account. |
| Sweep | 60-second worker budget; at most one source call/member requesting up to 25 items. |
| Results | Up to 500 matches and a 1,000,000-byte result bound; warnings disclose truncation. |
| Result pages | Default 25, maximum 100 rows. |

cohorts/members replaces membership rather than appending. cohort-queries/delete with query_id deletes a query; cohorts/delete with cohort_id removes the cohort and its queries and attempts cancellation of queued/running work. Inspect cancellation/refund fields. Export needed results before deleting.

Resources are account-owned; missing and unowned resources return not found. No date-complete archive or fixed retention period is promised. Generic MCP call_endpoint does not provide cohort management or sweeps; use authenticated REST. MCP documentation and endpoint discovery can still explain them.

- [Cohort operations](https://www.monocrawl.com/docs/platforms/cohorts)

- [Query operations](https://www.monocrawl.com/docs/platforms/cohort-queries)

- [Jobs and refunds](https://www.monocrawl.com/docs/jobs-webhooks)