Reading API /api/v1/logs with a stable cursor, and API tokens
GET /api/v1/logs?query=<LogsQL>&since=<cursor|time|duration>&limit=<n> returns the logs after a cursor, oldest first, with next_cursor and has_more. The cursor is a position on _time (the reception time) plus the number of rows already returned among those sharing that exact time, ordered by a hash of their content, so following the cursors gives every row once even within groups cut by a page. Rows count once older than the lag (30 s), and nothing is returned while the disk spool still holds logs. API tokens (Settings > API) give read-only access to /api/v1/ as "Authorization: Bearer lst_...", in every auth mode; only their SHA-256 is kept in /data/api_tokens.json. README (EN/FR) documents the route and an n8n workflow keeping the cursor in $getWorkflowStaticData. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
d61384ed2c
commit
c8f45044a3
12 files changed
+1327
-7
No files matched your search
@@ -354,13 +354,82 @@ chosen in Settings), `severity`, `facility`, `host`, `host_ip`, `app`, `pid`, `s
|
||||
|
||||
From the command line: `curl -o logs.csv 'localhost:8080/api/export.csv?q=error&range=24h&tz=Europe/Paris'`.
|
||||
|
||||
## Reading API (n8n, scripts)
|
||||
|
||||
`GET /api/v1/logs` returns the stored logs that follow a cursor, oldest first, so that a tool
|
||||
called on a schedule (n8n, a script, a SIEM connector) gets every new log once, without gaps
|
||||
or duplicates, even when many logs share the same timestamp.
|
||||
|
||||
Access goes through an **API token**, created in *Settings > API* by an administrator and sent
|
||||
as `Authorization: Bearer lst_…`. A token is read-only and only opens `/api/v1/`; it works
|
||||
whatever the [authentication](#authentication) mode (`local`, `oidc`, or none). It is shown once,
|
||||
when it is created: LogStream only keeps its SHA-256 hash, in `/data/api_tokens.json`.
|
||||
Revoking it cuts off access at once; the list shows when and from where each token was last
|
||||
used. A logged-in browser session can call the route too.
|
||||
|
||||
| Parameter | Meaning |
|
||||
|---|---|
|
||||
| `query` | LogsQL filter (default `*`), e.g. `severity:in(err,crit) app:sshd`. Pipes (`\| ...`) are refused. |
|
||||
| `since` | where to start: the `next_cursor` of the previous answer, an RFC 3339 time (`2026-11-01T08:00:00Z`) or a duration back from now (`15m`, `6h`, `7d`). Default: `1h`. |
|
||||
| `limit` | rows per answer, 1 to 10,000 (default 1,000). |
|
||||
| `lag` | only logs received at least this long ago are returned (default `30s`, at most `1h`), so that the logs still in the queue or waiting for their DNS name are not skipped. |
|
||||
|
||||
```json
|
||||
{
|
||||
"rows": [{"_time": "2026-11-01T14:02:11.52Z", "_msg": "…", "host": "fw01", "app": "sshd", "severity": "err", …}],
|
||||
"next_cursor": "eyJ0IjoiMjAyNi0xMS0wMVQxNDowMjoxMS41MjAwMDAwMDFaIiwicSI6IjJlYjE4YzNhIn0",
|
||||
"has_more": false,
|
||||
"until": "2026-11-01T14:05:30.1Z"
|
||||
}
|
||||
```
|
||||
|
||||
Rows have the stored fields (as in the UI's JSON view), sorted by `_time`, the reception time.
|
||||
Always send back `next_cursor`, even when `rows` is empty. `has_more: true` means more logs are
|
||||
ready: call again at once. `waiting: true` means logs from the [disk buffer](#configuration) are
|
||||
still being sent to VictoriaLogs: nothing is returned until they are stored, so none is missed.
|
||||
The cursor belongs to its `query`: changing the query keeps the position in time. Errors come
|
||||
as `{"error", "code"}` with status 400 (bad parameter or LogsQL), 401 (invalid or revoked token)
|
||||
or 502 (VictoriaLogs unreachable).
|
||||
|
||||
```bash
|
||||
curl -H "Authorization: Bearer lst_…" \
|
||||
"https://logs.example.org/api/v1/logs?query=severity:err&since=15m&limit=1000"
|
||||
```
|
||||
|
||||
### Example: n8n workflow
|
||||
|
||||
Four nodes, the cursor kept in the workflow's static data between runs:
|
||||
|
||||
1. **Schedule Trigger**: every minute.
|
||||
2. **Code** (*Run Once for All Items*), reads the cursor:
|
||||
```js
|
||||
const data = $getWorkflowStaticData('global');
|
||||
return [{ json: { since: data.cursor || '15m' } }];
|
||||
```
|
||||
3. **HTTP Request**: method `GET`, URL `https://logs.example.org/api/v1/logs`.
|
||||
*Authentication*: Generic Credential Type > Header Auth, with name `Authorization` and value
|
||||
`Bearer lst_…`. *Send Query Parameters*: `query` = `severity:in(emerg,alert,crit,err)`,
|
||||
`since` = `{{ $json.since }}`, `limit` = `1000`.
|
||||
4. **Code** (*Run Once for All Items*), saves the cursor and turns each log into an item:
|
||||
```js
|
||||
const data = $getWorkflowStaticData('global');
|
||||
const page = $input.first().json;
|
||||
data.cursor = page.next_cursor;
|
||||
return page.rows.map((row) => ({ json: row }));
|
||||
```
|
||||
|
||||
Then connect what the logs should feed (Slack, mail, ticket, database…). n8n only keeps static
|
||||
data for runs of the active workflow, not for manual test runs: while testing, each run starts
|
||||
again from `15m`. When more than 1,000 logs arrive per minute, `has_more` is true and the
|
||||
following runs catch up; raise `limit` or run the trigger more often if needed.
|
||||
|
||||
## Settings
|
||||
|
||||
The gear icon opens the settings, organized in tabs. Changes show at once but are only kept
|
||||
with the **Save** button, under the tab labels: it takes the accent color while changes are
|
||||
waiting. **Cancel** goes back to the saved settings, and closing the window with unsaved changes
|
||||
asks before dropping them. Localization and Appearance are remembered per browser; color tags
|
||||
and log sources are saved on the server. Purging and resetting column widths act at once.
|
||||
and log sources are saved on the server. Purging, API tokens and resetting column widths act at once.
|
||||
|
||||
- **Localization**
|
||||
- *Language*: English or French.
|
||||
@@ -408,6 +477,9 @@ and log sources are saved on the server. Purging and resetting column widths act
|
||||
(already set in `docker-compose.yml`). The feature is off by default: set `ALLOW_PURGE=true`
|
||||
to allow it. Any admin can then purge: turn on [authentication](#authentication) if the UI
|
||||
is reachable by others.
|
||||
- **API**: API tokens for the [reading API](#reading-api-n8n-scripts): create one with a name
|
||||
(shown once, copy it then), see when and from where each was last used, revoke it. Reserved
|
||||
to administrators.
|
||||
|
||||
## Authentication
|
||||
|
||||
@@ -577,6 +649,8 @@ To update one of them:
|
||||
| `hub.go` | pushes new messages to browsers (SSE) |
|
||||
| `rdns.go` | cached reverse DNS lookups |
|
||||
| `export.go` | streamed CSV export |
|
||||
| `logsapi.go` | reading API `/api/v1/logs`: cursor, groups of logs with the same time |
|
||||
| `apitokens.go` | API tokens: storage (hashed), Bearer check, `/api/tokens` |
|
||||
| `docker.go` | Docker container logs (API, followers, positions, level detection) |
|
||||
| `syslogserver.go` | syslog listeners opened and closed from Settings > Log sources |
|
||||
| `hostlogs.go`, `journal.go` | host system logs: systemd journal reader (no `journalctl`) and `/var/log` follower |
|
||||
|
||||
Reference in new issue
Block a user