> ## Documentation Index
> Fetch the complete documentation index at: https://docs.usequotable.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Custom HTTP / NDJSON

> Send request logs from any server, proxy or pipeline to Agent Analytics with an authenticated JSON or NDJSON POST, and read the per-record receipt it returns.

Use the Custom HTTP connector when your platform isn't listed, or when you already ship logs through your own pipeline (a log shipper, a queue consumer, a reverse proxy hook). You POST batches of request records; Quotable keeps the AI agent requests and tells you what happened to every record.

> [!TIP]
>
> You can send **all** your request logs. Human traffic is dropped on arrival and doesn't count toward your quota. Filtering to AI agents on your side only saves bandwidth.

## Set up

**1. Create the source**

In **Agent Analytics**, select **Connect source**, choose **Custom HTTP / NDJSON** and select **Generate credentials**.

**2. Store the credentials**

Copy the token (`aa_live_…`) and the endpoint URL into your secret store. The token is shown once.

**3. Send a test delivery**

Run the `curl` command the wizard shows. It sets `x-agent-analytics-test: 1`, so the record is classified and answered but not stored.

**4. Start sending**

Point your sender at the endpoint and select **Check now** in the wizard.

## Request

```http theme={"system"}
POST <your endpoint URL>
Authorization: Bearer aa_live_...
Content-Type: application/x-ndjson
Content-Encoding: gzip
```

### Authentication

Send the source token in the `Authorization` header as `Bearer <token>`. The token is accepted **only** in this header, never in the query string, because query strings end up in access logs and browser history.

The endpoint is for server-to-server calls. It sends no CORS headers, so browsers refuse to call it. Never put the token in page-side JavaScript.

### Body formats

The body's shape decides how it's parsed; `Content-Type` is only used to force NDJSON.

| Format | Example | Notes |
| - | - | - |
| NDJSON | One JSON object per line | Recommended. Forced when `Content-Type` contains `ndjson`. |
| JSON array | `[{…}, {…}]` | |
| Wrapped array | `{"events": [{…}, {…}]}` | |
| Single object | `{…}` | One record. |

A malformed line or array element is counted as `invalid_record`; the rest of the batch is still processed.

### Compression

Send `Content-Encoding: gzip` or no compression. Any other encoding returns `415`.

### Limits

| Limit | Value |
| - | - |
| Body size | 5 MB, before and after decompression |
| Records per delivery | 1,000 |
| Deliveries per source | 600 per 5-minute window |
| Accepted events per source per UTC day | Set by your plan. See [Plans and limits](https://docs.usequotable.com/workspace/plans-and-limits.md#brand-plans) |

Batch your records. One request per log line will hit the delivery limit on a busy site.

## Record fields

Each record describes one HTTP request. `host` and `path` are required; everything else improves what you can measure.

**host** `string` *required*

Hostname the visitor requested, for example `www.example.com`. Must be the source's domain or a subdomain of it.

**path** `string` *required*

Request path. A full URL is accepted too (`url` is an alias). The query string and fragment are always stripped.

**user\_agent** `string`

The visitor's `User-Agent`. Without it, no record can be recognized as an AI agent. Alias: `userAgent`.

**client\_ip** `string`

The visitor's IP address (IPv4 or IPv6). Used only to verify the agent, then discarded. Without it, every event is unverified. Alias: `clientIp`.

**occurred\_at** `string | number`

When the request happened: an ISO 8601 timestamp, or a Unix epoch in seconds or milliseconds. Alias: `timestamp`. If you leave it out, the arrival time is used.

**method** `string` (default: GET)

HTTP method.

**status\_code** `integer`

HTTP status of the response. Alias: `statusCode`.

**response\_bytes** `integer`

Response size in bytes. Alias: `responseBytes`.

**duration\_ms** `number`

Response time in milliseconds. Alias: `durationMs`.

**country\_code** `string`

Two-letter country code of the visitor. Alias: `countryCode`.

**referrer** `string`

Referrer URL. Only the host is kept.

**cache\_status** `string`

Your cache's verdict, for example `HIT` or `MISS`. Alias: `cacheStatus`.

**id** `string`

Your unique request ID. Strongly recommended: with it, the same request keeps the same `event_id` in any later delivery and is stored only once. Without it, a record is identified by its position in the delivery, so resending an identical delivery is still stored once, but the same record sent inside a different batch is stored again.

Unknown fields are ignored.

### Example

```json theme={"system"}
{"id":"req-8f2c","occurred_at":"2026-10-01T09:12:44Z","host":"example.com","path":"/pricing","method":"GET","status_code":200,"response_bytes":18432,"duration_ms":120,"country_code":"US","referrer":"https://chatgpt.com/","cache_status":"HIT","user_agent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot","client_ip":"203.0.113.10"}
```

## Response

A processed delivery answers `202 Accepted`. Every record is accounted for exactly once: `received = accepted + duplicates + dropped`.

```json theme={"system"}
{
  "received": 3,
  "accepted": 1,
  "duplicates": 0,
  "dropped": 2,
  "invalid_record": 0,
  "not_ai_agent": 1,
  "excluded_path": 1,
  "timestamp_out_of_range": 0,
  "not_http_traffic": 0,
  "environment_filtered": 0,
  "project_mismatch": 0,
  "host_mismatch": 0,
  "quota_exceeded": 0,
  "lag_seconds": 4,
  "events": [
    { "position": 0, "outcome": "accepted", "event_id": "3f1c…" },
    { "position": 1, "outcome": "dropped", "reason": "not_ai_agent" },
    { "position": 2, "outcome": "dropped", "reason": "excluded_path" }
  ],
  "status": "accepted",
  "batch_id": "9a1e…"
}
```

**received** `integer`

Records in the delivery.

**accepted** `integer`

AI agent requests stored.

**duplicates** `integer`

Records that repeat another record in the same delivery. A record resent in a later delivery is counted there as `accepted` again, but it keeps its `event_id` and is stored only once.

**dropped** `integer`

Records not stored. One counter per reason follows.

**lag\_seconds** `integer | null`

Seconds between the newest accepted record and its arrival. `null` when nothing was accepted.

**events** `array`

One entry per record, by `position` (line number for NDJSON, array index for arrays, `0` for a single object), with `outcome` `accepted`, `duplicate` or `dropped`, and the `event_id` or `reason`.

**status** `string`

`accepted`, `test` for a test delivery, or `deferred` when the events were received and will appear shortly. In every case you don't need to resend.

**batch\_id** `string`

Your `X-Batch-Id` or `X-Request-Id` header when you send one, otherwise one Quotable assigns.

Event IDs are deterministic. Retrying the same delivery, or resending a record with the same `id`, produces the same `event_id`, so a retry doesn't create a second event.

### Drop reasons

| Reason | Why the record wasn't stored |
| - | - |
| `invalid_record` | Not a JSON object, missing `host` or `path`, or an unreadable timestamp. |
| `not_ai_agent` | The user agent isn't a recognized AI agent. This is normal for human traffic. |
| `excluded_path` | Admin, auth, private, framework or static-asset path. See [Privacy by design](https://docs.usequotable.com/agent-analytics/overview.md#privacy-by-design). |
| `timestamp_out_of_range` | Older than seven days, or more than five minutes in the future. |
| `not_http_traffic` | A platform log line that isn't a request, such as build output. |
| `environment_filtered` | A preview or non-production deployment. |
| `project_mismatch` | Comes from a platform project other than the one mapped to the source. |
| `host_mismatch` | `host` isn't the source's domain or one of its subdomains. |
| `quota_exceeded` | Over the source's daily quota. |

## Errors

| Status | Body | What to do |
| - | - | - |
| `400` | `{"error":"Invalid JSON or NDJSON"}` | Nothing in the body could be parsed. Check the encoding and format. |
| `401` | `{"error":"Unauthorized"}` | Missing or wrong token, or the token was rotated. |
| `404` | `{"error":"Not found"}` | Wrong endpoint URL, or the source was disconnected. |
| `413` | `{"error":"Payload too large"}`, `{"error":"Payload too large or invalid gzip"}` or `{"error":"Maximum 1000 records per delivery"}` | Send smaller batches, and check the gzip stream is valid. |
| `415` | `{"error":"Unsupported content encoding"}` | Use gzip or no compression. |
| `429` | `{"code":"rate_limited","retry_after":…}` | Over 600 deliveries in 5 minutes. Wait for `Retry-After` and batch more records per request. |
| `429` | Full receipt plus `"code":"quota_exceeded"`, `"quota"`, `"retry_after"` | Today's quota is used up. `Retry-After` points at the next UTC midnight. |
| `503` | `{"code":"queue_unavailable"}` | Nothing was stored. Retry with backoff. |

Retry on `429` and `503` after the `Retry-After` header when present. Don't retry `400`, `401`, `404`, `413` or `415` without fixing the request.

## Test deliveries

Add `x-agent-analytics-test: 1` (or `true`) to any delivery. It's authenticated, rate-limited and host-checked like a real one, and the `202` response shows the outcome for every record, but nothing is stored or counted toward your quota. A test delivery also moves a new source to **Connected, waiting for the first agent visit**.

## Example senders

```bash cURL theme={"system"}
gzip -c requests.ndjson | curl -sS -X POST "$QUOTABLE_ENDPOINT" \
  -H "Authorization: Bearer $QUOTABLE_TOKEN" \
  -H "Content-Type: application/x-ndjson" \
  -H "Content-Encoding: gzip" \
  --data-binary @-
```

```javascript JavaScript theme={"system"}
import { gzipSync } from "node:zlib";

export async function sendBatch(records) {
  const body = gzipSync(records.map((r) => JSON.stringify(r)).join("\n"));
  const response = await fetch(process.env.QUOTABLE_ENDPOINT, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.QUOTABLE_TOKEN}`,
      "Content-Type": "application/x-ndjson",
      "Content-Encoding": "gzip",
    },
    body,
  });
  if (response.status === 429 || response.status === 503) {
    const wait = Number(response.headers.get("retry-after") ?? 30);
    throw new Error(`Retry in ${wait}s`);
  }
  return response.json();
}
```

```python Python theme={"system"}
import gzip, json, os, requests

def send_batch(records):
    body = gzip.compress("\n".join(json.dumps(r) for r in records).encode())
    response = requests.post(
        os.environ["QUOTABLE_ENDPOINT"],
        data=body,
        headers={
            "Authorization": f"Bearer {os.environ['QUOTABLE_TOKEN']}",
            "Content-Type": "application/x-ndjson",
            "Content-Encoding": "gzip",
        },
        timeout=30,
    )
    if response.status_code in (429, 503):
        raise RuntimeError(f"Retry in {response.headers.get('Retry-After', '30')}s")
    response.raise_for_status()
    return response.json()
```

Keep batches at or under 1,000 records and 5 MB.

## Next steps

> [card] [Log file import](https://docs.usequotable.com/agent-analytics/connectors/log-file-import.md)
>
> Upload a log export instead of streaming.

> [card] [Troubleshooting](https://docs.usequotable.com/agent-analytics/troubleshooting.md)
>
> Read a receipt and fix the most common drops.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.