# API: Scraping

| Method | Path | What it does | Scope |
|---|---|---|---|
| GET | `/v1/scrape/domains` | The owner's allow and block rules per domain (sites whose terms forbid scraping are refused whatever the rule) | `prospecting:read` |
| PUT | `/v1/scrape/domains` | Allow or block a domain for scraping (owner; audited). Allowed domains may be scraped by agents on any page without a query string | `settings:write` |
| DELETE | `/v1/scrape/domains/:domain` | Remove a domain's rule (owner; audited) | `settings:write` |
| POST | `/v1/scrape/estimate` | What a scrape would cost (model and Apify), without creating it | `prospecting:read` |
| POST | `/v1/scrape/local/csv` | Import a CSV of local businesses (name, address, phone, website, category, rating…): deduplicated and matched against the CRM's accounts, then previewed (nothing reaches the CRM until imported) | `prospecting:write` |
| GET | `/v1/scrape/runs` | Scrape runs, newest first, with row counts | `prospecting:read` |
| POST | `/v1/scrape/runs` | Set up a scrape: pages and fields, an Apify actor, a recipe, or a local-business search (Google Maps via Apify). Returns the cost preview; nothing runs until it's started | `prospecting:write` |
| GET | `/v1/scrape/runs/:id` | A run: its input, status, notes (partial results, refusals, caps), cost and row counts | `prospecting:read` |
| PUT | `/v1/scrape/runs/:id/criteria` | Set a run's fit criteria (field, op, value): every row gets a pass / fail / unsure verdict with its reason, judged again now | `prospecting:write` |
| POST | `/v1/scrape/runs/:id/import` | Import a local-business run's rows (all, or the chosen ones) into a static list of accounts: new businesses become accounts, ones already in the CRM are linked, never duplicated | `prospecting:write` |
| GET | `/v1/scrape/runs/:id/rows` | The rows: values with the page's exact words, source URL and fetch time; for local businesses, the CRM account each matches | `prospecting:read` |
| POST | `/v1/scrape/runs/:id/start` | Start a previewed run in the background | `prospecting:write` |
| GET | `/v1/scrape/settings` | Scraping status: whether Apify is on (it waits for approval and a key), this month's Apify cap and spend, the allowlisted actors, the recipes and the sites never scraped | `prospecting:read` |
| PUT | `/v1/scrape/settings` | Set the organization's monthly Apify cap in USD (owner; audited) | `settings:write` |

## Examples

### GET /v1/scrape/domains

The owner's allow and block rules per domain (sites whose terms forbid scraping are refused whatever the rule)

```bash
curl -s "https://api.closefinch.com/v1/scrape/domains" \
  -H "Authorization: Bearer $RA_API_KEY"
```

```js
const res = await fetch(`https://api.closefinch.com/v1/scrape/domains`, {
  method: "GET",
  headers: { authorization: `Bearer ${process.env.RA_API_KEY}` },
});
const data = await res.json();
```

### PUT /v1/scrape/domains

Allow or block a domain for scraping (owner; audited). Allowed domains may be scraped by agents on any page without a query string

```bash
curl -s -X PUT "https://api.closefinch.com/v1/scrape/domains" \
  -H "Authorization: Bearer $RA_API_KEY" \
  -H "content-type: application/json" \
  -d '{}'
```

```js
const res = await fetch(`https://api.closefinch.com/v1/scrape/domains`, {
  method: "PUT",
  headers: { authorization: `Bearer ${process.env.RA_API_KEY}`, "content-type": "application/json" },
  body: JSON.stringify({}), // fields: see the OpenAPI schema
});
const data = await res.json();
```

### DELETE /v1/scrape/domains/$DOMAIN

Remove a domain's rule (owner; audited)

```bash
curl -s -X DELETE "https://api.closefinch.com/v1/scrape/domains/$DOMAIN" \
  -H "Authorization: Bearer $RA_API_KEY"
```

```js
const res = await fetch(`https://api.closefinch.com/v1/scrape/domains/${domain}`, {
  method: "DELETE",
  headers: { authorization: `Bearer ${process.env.RA_API_KEY}` },
});
const data = await res.json();
```

### POST /v1/scrape/estimate

What a scrape would cost (model and Apify), without creating it

```bash
curl -s -X POST "https://api.closefinch.com/v1/scrape/estimate" \
  -H "Authorization: Bearer $RA_API_KEY" \
  -H "content-type: application/json" \
  -d '{}'
```

```js
const res = await fetch(`https://api.closefinch.com/v1/scrape/estimate`, {
  method: "POST",
  headers: { authorization: `Bearer ${process.env.RA_API_KEY}`, "content-type": "application/json" },
  body: JSON.stringify({}), // fields: see the OpenAPI schema
});
const data = await res.json();
```

### POST /v1/scrape/local/csv

Import a CSV of local businesses (name, address, phone, website, category, rating…): deduplicated and matched against the CRM's accounts, then previewed (nothing reaches the CRM until imported)

```bash
curl -s -X POST "https://api.closefinch.com/v1/scrape/local/csv" \
  -H "Authorization: Bearer $RA_API_KEY" \
  -H "content-type: application/json" \
  -d '{}'
```

```js
const res = await fetch(`https://api.closefinch.com/v1/scrape/local/csv`, {
  method: "POST",
  headers: { authorization: `Bearer ${process.env.RA_API_KEY}`, "content-type": "application/json" },
  body: JSON.stringify({}), // fields: see the OpenAPI schema
});
const data = await res.json();
```

### GET /v1/scrape/runs

Scrape runs, newest first, with row counts

```bash
curl -s "https://api.closefinch.com/v1/scrape/runs" \
  -H "Authorization: Bearer $RA_API_KEY"
```

```js
const res = await fetch(`https://api.closefinch.com/v1/scrape/runs`, {
  method: "GET",
  headers: { authorization: `Bearer ${process.env.RA_API_KEY}` },
});
const data = await res.json();
```

### POST /v1/scrape/runs

Set up a scrape: pages and fields, an Apify actor, a recipe, or a local-business search (Google Maps via Apify). Returns the cost preview; nothing runs until it's started

```bash
curl -s -X POST "https://api.closefinch.com/v1/scrape/runs" \
  -H "Authorization: Bearer $RA_API_KEY" \
  -H "content-type: application/json" \
  -d '{}'
```

```js
const res = await fetch(`https://api.closefinch.com/v1/scrape/runs`, {
  method: "POST",
  headers: { authorization: `Bearer ${process.env.RA_API_KEY}`, "content-type": "application/json" },
  body: JSON.stringify({}), // fields: see the OpenAPI schema
});
const data = await res.json();
```

### GET /v1/scrape/runs/$ID

A run: its input, status, notes (partial results, refusals, caps), cost and row counts

```bash
curl -s "https://api.closefinch.com/v1/scrape/runs/$ID" \
  -H "Authorization: Bearer $RA_API_KEY"
```

```js
const res = await fetch(`https://api.closefinch.com/v1/scrape/runs/${id}`, {
  method: "GET",
  headers: { authorization: `Bearer ${process.env.RA_API_KEY}` },
});
const data = await res.json();
```

### PUT /v1/scrape/runs/$ID/criteria

Set a run's fit criteria (field, op, value): every row gets a pass / fail / unsure verdict with its reason, judged again now

```bash
curl -s -X PUT "https://api.closefinch.com/v1/scrape/runs/$ID/criteria" \
  -H "Authorization: Bearer $RA_API_KEY" \
  -H "content-type: application/json" \
  -d '{}'
```

```js
const res = await fetch(`https://api.closefinch.com/v1/scrape/runs/${id}/criteria`, {
  method: "PUT",
  headers: { authorization: `Bearer ${process.env.RA_API_KEY}`, "content-type": "application/json" },
  body: JSON.stringify({}), // fields: see the OpenAPI schema
});
const data = await res.json();
```

### POST /v1/scrape/runs/$ID/import

Import a local-business run's rows (all, or the chosen ones) into a static list of accounts: new businesses become accounts, ones already in the CRM are linked, never duplicated

```bash
curl -s -X POST "https://api.closefinch.com/v1/scrape/runs/$ID/import" \
  -H "Authorization: Bearer $RA_API_KEY" \
  -H "content-type: application/json" \
  -d '{}'
```

```js
const res = await fetch(`https://api.closefinch.com/v1/scrape/runs/${id}/import`, {
  method: "POST",
  headers: { authorization: `Bearer ${process.env.RA_API_KEY}`, "content-type": "application/json" },
  body: JSON.stringify({}), // fields: see the OpenAPI schema
});
const data = await res.json();
```

### GET /v1/scrape/runs/$ID/rows

The rows: values with the page's exact words, source URL and fetch time; for local businesses, the CRM account each matches

```bash
curl -s "https://api.closefinch.com/v1/scrape/runs/$ID/rows" \
  -H "Authorization: Bearer $RA_API_KEY"
```

```js
const res = await fetch(`https://api.closefinch.com/v1/scrape/runs/${id}/rows`, {
  method: "GET",
  headers: { authorization: `Bearer ${process.env.RA_API_KEY}` },
});
const data = await res.json();
```

### POST /v1/scrape/runs/$ID/start

Start a previewed run in the background

```bash
curl -s -X POST "https://api.closefinch.com/v1/scrape/runs/$ID/start" \
  -H "Authorization: Bearer $RA_API_KEY" \
  -H "content-type: application/json" \
  -d '{}'
```

```js
const res = await fetch(`https://api.closefinch.com/v1/scrape/runs/${id}/start`, {
  method: "POST",
  headers: { authorization: `Bearer ${process.env.RA_API_KEY}`, "content-type": "application/json" },
  body: JSON.stringify({}), // fields: see the OpenAPI schema
});
const data = await res.json();
```

### GET /v1/scrape/settings

Scraping status: whether Apify is on (it waits for approval and a key), this month's Apify cap and spend, the allowlisted actors, the recipes and the sites never scraped

```bash
curl -s "https://api.closefinch.com/v1/scrape/settings" \
  -H "Authorization: Bearer $RA_API_KEY"
```

```js
const res = await fetch(`https://api.closefinch.com/v1/scrape/settings`, {
  method: "GET",
  headers: { authorization: `Bearer ${process.env.RA_API_KEY}` },
});
const data = await res.json();
```

### PUT /v1/scrape/settings

Set the organization's monthly Apify cap in USD (owner; audited)

```bash
curl -s -X PUT "https://api.closefinch.com/v1/scrape/settings" \
  -H "Authorization: Bearer $RA_API_KEY" \
  -H "content-type: application/json" \
  -d '{}'
```

```js
const res = await fetch(`https://api.closefinch.com/v1/scrape/settings`, {
  method: "PUT",
  headers: { authorization: `Bearer ${process.env.RA_API_KEY}`, "content-type": "application/json" },
  body: JSON.stringify({}), // fields: see the OpenAPI schema
});
const data = await res.json();
```

Request and response schemas: [https://www.closefinch.com/openapi.json](https://www.closefinch.com/openapi.json). All resources: [API reference](https://www.closefinch.com/docs/api).