Skip to content
tracing.toolstracing.tools

Crawlers

GET /api/v1/sites/:id/crawlers

The same numbers the Crawlers screen shows: how the range’s bot traffic splits across the four categories, the crawlers in one of them ranked by hits, and the series behind the chart.

Parameters

Section titled “Parameters”
Parameter Values Default
range today, yesterday, 24h, 7d, 30d, 90d, 12m, week, month 30d
category answers, indexing, training, other answers

totals always covers all four categories; list and series cover the one in category. Filter keys are ignored — a crawler hit carries no visitor, path filter or campaign to slice by.

The range is resolved in the site’s timezone, not yours.

Request

Section titled “Request”
Terminal window
curl -s -H "Authorization: Bearer tk_live_…" \
  "https://tracing.tools/api/v1/sites/pk_live_a91c3f04/crawlers?range=7d&category=answers"

Response

Section titled “Response”
{
  "site": {
    "id": "6f2b1e7c-9a54-4f21-8c3a-1d0b7e5f4a92",
    "domain": "acme.com"
  },
  "range": {
    "key": "7d",
    "start": "2026-09-10T22:00:00.000Z",
    "end": "2026-09-16T17:41:12.913Z",
    "timezone": "Europe/Berlin",
    "unit": "day"
  },
  "data": {
    "totals": {
      "answers": 519,
      "indexing": 933,
      "training": 970,
      "other": 412,
      "total": 2834
    },
    "list": [
      { "crawler": "chatgpt",         "hits": 241, "pages": 63, "lastSeenAt": "2026-09-16T17:02:44Z" },
      { "crawler": "claude-user",     "hits": 168, "pages": 51, "lastSeenAt": "2026-09-16T16:41:09Z" },
      { "crawler": "perplexity-user", "hits": 110, "pages": 38, "lastSeenAt": "2026-09-16T15:58:20Z" }
    ],
    "series": [
      { "date": "2026-09-11T00:00:00", "chatgpt": 31, "claude-user": 22, "perplexity-user": 14 },
      { "date": "2026-09-12T00:00:00", "chatgpt": 0,  "claude-user": 0,  "perplexity-user": 0 },
      { "date": "2026-09-13T00:00:00", "chatgpt": 44, "claude-user": 29, "perplexity-user": 18 }
    ]
  }
}

The series is abbreviated; a real response has one element per bucket.

Fields

Section titled “Fields”
Field Meaning
data.totals.<category> Hits in the range for that category, across all four
data.totals.total Hits in the range, whatever the category
data.list[].crawler Crawler slug, e.g. gptbot. unknown is a bot with a token we do not recognise, suspected a client that behaved like a script
data.list[].hits Pages the crawler fetched in the range; rows are ranked by it
data.list[].pages Distinct paths among those hits
data.list[].lastSeenAt Most recent hit from the crawler, UTC
data.series[].date Bucket start in the site’s timezone, no offset suffix
data.series[]."<slug>" Hits from that crawler in the bucket; 0, never null

range.unit is the bucket size the series was cut in: hour, day or month.

Errors

Section titled “Errors”
Status Body
401 {"error":"Missing or invalid API key"}
404 {"error":"site not found"}

An unrecognised range falls back to 30d and an unrecognised category falls back to answers.

Section titled “Related”