opensource/loki
1
0
Fork 0
mirror of https://github.com/grafana/loki.git synced 2025-08-02 11:37:22 +08:00
Code Issues Packages Projects Releases Wiki Activity

Docs: Add doc-validator tool to lint technical documentation (#6569)

Browse Source 
Source code:

https://github.com/grafana/technical-documentation/tree/main/tools/doc-validator

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

**What this PR does / why we need it**:

Adds a CI step for pull requests that validates links in technical
documentation and performs linting of technical documentation source
files.

**Special notes for your reviewer**:

- Lots of issues will need to be fixed before we can merge this.
- The tool is still in early development so each error should be
scrutinized for validity.

**Checklist**
- [ ] Documentation added
- [na] Tests updated
- [na] Is this an important fix or new feature? Add an entry in the
`CHANGELOG.md`.
- [na] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/upgrading/_index.md`

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
This commit is contained in:
Jack Baldry
2023-01-19 09:55:12 +00:00
committed by GitHub
parent 86461b3faa
commit 3cb15d62d5
4 changed files with 85 additions and 65 deletions
Download Patch File Download Diff File Expand all files Collapse all files

115
docs/sources/api/_index.md
View File

@ -1,5 +1,5 @@
---
title: HTTP API
title: Grafana Loki HTTP API
menuTitle: "HTTP API"
description: "Loki exposes REST endpoints for operating on a Loki cluster. This section details the endpoints."
weight: 900
@ -75,12 +75,13 @@ These endpoints are exposed by the ruler:
- [`GET /prometheus/api/v1/alerts`](#list-alerts)
These endpoints are exposed by the compactor:
- [`GET /compactor/ring`](#compactor-ring-status)
- [`POST /loki/api/v1/delete`](#request-log-deletion)
- [`GET /loki/api/v1/delete`](#list-log-deletion-requests)
- [`DELETE /loki/api/v1/delete`](#request-cancellation-of-a-delete-request)
A [list of clients](../clients) can be found in the clients documentation.
A [list of clients]({{< relref "../clients" >}}) can be found in the clients documentation.
## Matrix, vector, and streams
@ -114,7 +115,7 @@ GET /loki/api/v1/query
`/loki/api/v1/query` allows for doing queries against a single point in time. The URL
query parameters support the following values:
- `query`: The [LogQL](../logql/) query to perform
- `query`: The [LogQL]({{< relref "../logql/" >}}) query to perform
- `limit`: The max number of entries to return. It defaults to `100`. Only applies to query types which produce a stream(log lines) response.
- `time`: The evaluation time for the query as a nanosecond Unix epoch or another [supported format](#timestamp-formats). Defaults to now.
- `direction`: Determines the sort order of logs. Supported values are `forward` or `backward`. Defaults to `backward`.
@ -174,6 +175,7 @@ See [statistics](#statistics) for information about the statistics returned by L
### Examples
This example query
```bash
curl -G -s "http://localhost:3100/loki/api/v1/query" \
--data-urlencode \
@ -222,7 +224,7 @@ gave this response:
```
If your cluster has
[Grafana Loki Multi-Tenancy](../operations/multi-tenancy/) enabled,
[Grafana Loki Multi-Tenancy]({{< relref "../operations/multi-tenancy/" >}}) enabled,
set the `X-Scope-OrgID` header to identify the tenant you want to query.
Here is the same example query for the single tenant called `Tenant1`:
@ -265,12 +267,12 @@ GET /loki/api/v1/query_range
`/loki/api/v1/query_range` is used to do a query over a range of time and
accepts the following query parameters in the URL:
- `query`: The [LogQL](../logql/) query to perform
- `query`: The [LogQL]({{< relref "../logql/" >}}) query to perform
- `limit`: The max number of entries to return. It defaults to `100`. Only applies to query types which produce a stream(log lines) response.
- `start`: The start time for the query as a nanosecond Unix epoch or another [supported format](#timestamp-formats). Defaults to one hour ago.
- `end`: The end time for the query as a nanosecond Unix epoch or another [supported format](#timestamp-formats). Defaults to now.
- `since`: A `duration` used to calculate `start` relative to `end`. If `end` is in the future, `start` is calculated as this duration before now. Any value specified for `start` supersedes this parameter.
- `step`: Query resolution step width in `duration` format or float number of seconds. `duration` refers to Prometheus duration strings of the form `[0-9]+[smhdwy]`. For example, 5m refers to a duration of 5 minutes. Defaults to a dynamic value based on `start` and `end`. Only applies to query types which produce a matrix response.
- `since`: A `duration` used to calculate `start` relative to `end`. If `end` is in the future, `start` is calculated as this duration before now. Any value specified for `start` supersedes this parameter.
- `step`: Query resolution step width in `duration` format or float number of seconds. `duration` refers to Prometheus duration strings of the form `[0-9]+[smhdwy]`. For example, 5m refers to a duration of 5 minutes. Defaults to a dynamic value based on `start` and `end`. Only applies to query types which produce a matrix response.
- `interval`: <span style="background-color:#f3f973;">This parameter is experimental; see the explanation under Step versus interval.</span> Only return entries at (or greater than) the specified interval, can be a `duration` format or float number of seconds. Only applies to queries which produce a stream response.
- `direction`: Determines the sort order of logs. Supported values are `forward` or `backward`. Defaults to `backward.`
@ -278,13 +280,11 @@ In microservices mode, `/loki/api/v1/query_range` is exposed by the querier and
### Step versus interval
Use the `step` parameter when making metric queries to Loki, or queries which return a matrix response. It is evaluated in exactly the same way Prometheus evaluates `step`. First the query will be evaluated at `start` and then evaluated again at `start + step` and again at `start + step + step` until `end` is reached. The result will be a matrix of the query result evaluated at each step.
Use the `interval` parameter when making log queries to Loki, or queries which return a stream response. It is evaluated by returning a log entry at `start`, then the next entry will be returned an entry with timestampe >= `start + interval`, and again at `start + interval + interval` and so on until `end` is reached. It does not fill missing entries.
<span style="background-color:#f3f973;">Note about the experimental nature of the interval parameter:</span> This flag may be removed in the future, if so it will likely be in favor of a LogQL expression to perform similar behavior, however that is uncertain at this time. [Issue 1779](https://github.com/grafana/loki/issues/1779) was created to track the discussion, if you are using `interval` please go add your use case and thoughts to that issue.
Use the `step` parameter when making metric queries to Loki, or queries which return a matrix response. It is evaluated in exactly the same way Prometheus evaluates `step`. First the query will be evaluated at `start` and then evaluated again at `start + step` and again at `start + step + step` until `end` is reached. The result will be a matrix of the query result evaluated at each step.
Use the `interval` parameter when making log queries to Loki, or queries which return a stream response. It is evaluated by returning a log entry at `start`, then the next entry will be returned an entry with timestampe >= `start + interval`, and again at `start + interval + interval` and so on until `end` is reached. It does not fill missing entries.
<span style="background-color:#f3f973;">Note about the experimental nature of the interval parameter:</span> This flag may be removed in the future, if so it will likely be in favor of a LogQL expression to perform similar behavior, however that is uncertain at this time. [Issue 1779](https://github.com/grafana/loki/issues/1779) was created to track the discussion, if you are using `interval` please go add your use case and thoughts to that issue.
Response:
@ -440,7 +440,7 @@ It accepts the following query parameters in the URL:
- `start`: The start time for the query as a nanosecond Unix epoch. Defaults to 6 hours ago.
- `end`: The end time for the query as a nanosecond Unix epoch. Defaults to now.
- `since`: A `duration` used to calculate `start` relative to `end`. If `end` is in the future, `start` is calculated as this duration before now. Any value specified for `start` supersedes this parameter.
- `since`: A `duration` used to calculate `start` relative to `end`. If `end` is in the future, `start` is calculated as this duration before now. Any value specified for `start` supersedes this parameter.
In microservices mode, `/loki/api/v1/labels` is exposed by the querier.
@ -482,7 +482,7 @@ It accepts the following query parameters in the URL:
- `start`: The start time for the query as a nanosecond Unix epoch. Defaults to 6 hours ago.
- `end`: The end time for the query as a nanosecond Unix epoch. Defaults to now.
- `since`: A `duration` used to calculate `start` relative to `end`. If `end` is in the future, `start` is calculated as this duration before now. Any value specified for `start` supersedes this parameter.
- `since`: A `duration` used to calculate `start` relative to `end`. If `end` is in the future, `start` is calculated as this duration before now. Any value specified for `start` supersedes this parameter.
In microservices mode, `/loki/api/v1/label/<name>/values` is exposed by the querier.
@ -521,9 +521,9 @@ GET /loki/api/v1/tail
`/loki/api/v1/tail` is a WebSocket endpoint that will stream log messages based on
a query. It accepts the following query parameters in the URL:
- `query`: The [LogQL](../logql/) query to perform
- `query`: The [LogQL]({{< relref "../logql/" >}}) query to perform
- `delay_for`: The number of seconds to delay retrieving logs to let slow
loggers catch up. Defaults to 0 and cannot be larger than 5.
loggers catch up. Defaults to 0 and cannot be larger than 5.
- `limit`: The max number of entries to return. It defaults to `100`.
- `start`: The start time for the query as a nanosecond Unix epoch. Defaults to one hour ago.
@ -590,18 +590,15 @@ JSON post body can be sent in the following format:
You can set `Content-Encoding: gzip` request header and post gzipped JSON.
Loki can be configured to [accept out-of-order writes](../configuration/#accept-out-of-order-writes).
In microservices mode, `/loki/api/v1/push` is exposed by the distributor.
### Examples
```bash
```console
$ curl -v -H "Content-Type: application/json" -XPOST -s "http://localhost:3100/loki/api/v1/push" --data-raw \
'{"streams": [{ "stream": { "foo": "bar2" }, "values": [ [ "1570818238000000000", "fizzbuzz" ] ] }]}'
```
## Identify ready Loki instance
```
@ -638,11 +635,11 @@ It accepts three URL query parameters `flush`, `delete_ring_tokens`, and `termin
**URL query parameters:**
* `flush=<bool>`:
- `flush=<bool>`:
Flag to control whether to flush any in-memory chunks the ingester holds. Defaults to `true`.
* `delete_ring_tokens=<bool>`:
- `delete_ring_tokens=<bool>`:
Flag to control whether to delete the file that contains the ingester ring tokens of the instance if the `-ingester.token-file-path` is specified. Defaults to `false.
* `terminate=<bool>`:
- `terminate=<bool>`:
Flag to control whether to terminate the Loki process after service shutdown. Defaults to `true`.
This handler, in contrast to the deprecated `/ingester/flush_shutdown` handler, terminates the Loki process by default.
@ -665,7 +662,7 @@ GET /metrics
```
`/metrics` returns exposed Prometheus metrics. See
[Observing Loki](../operations/observability/)
[Observing Loki]({{< relref "../operations/observability/" >}})
for a list of exported metrics.
In microservices mode, the `/metrics` endpoint is exposed by all components.
@ -715,6 +712,7 @@ POST /loki/api/v1/format_query
```
Params:
- `query`: A LogQL query string. Can be passed as URL param (`?query=<query>`) in case of both `GET` and `POST`. Or as form value in case of `POST`.
The `/loki/api/v1/format_query` endpoint allows to format LogQL queries. It returns an error if the passed LogQL is invalid. It is exposed by all Loki components and helps to improve readability and the debugging experience of LogQL queries.
@ -722,6 +720,7 @@ The `/loki/api/v1/format_query` endpoint allows to format LogQL queries. It retu
## List series
The Series API is available under the following:
- `GET /loki/api/v1/series`
- `POST /loki/api/v1/series`
- `GET /api/prom/series`
@ -734,7 +733,7 @@ URL query parameters:
- `match[]=<series_selector>`: Repeated log stream selector argument that selects the streams to return. At least one `match[]` argument must be provided.
- `start=<nanosecond Unix epoch>`: Start timestamp.
- `end=<nanosecond Unix epoch>`: End timestamp.
- `since`: A `duration` used to calculate `start` relative to `end`. If `end` is in the future, `start` is calculated as this duration before now. Any value specified for `start` supersedes this parameter.
- `since`: A `duration` used to calculate `start` relative to `end`. If `end` is in the future, `start` is calculated as this duration before now. Any value specified for `start` supersedes this parameter.
You can URL-encode these parameters directly in the request body by using the POST method and `Content-Type: application/x-www-form-urlencoded` header. This is useful when specifying a large or dynamic number of stream selectors that may breach server-side URL character limits.
@ -742,7 +741,7 @@ In microservices mode, these endpoints are exposed by the querier.
### Examples
``` bash
```console
$ curl -s "http://localhost:3100/loki/api/v1/series" --data-urlencode 'match[]={container_name=~"prometheus.*", component="server"}' --data-urlencode 'match[]={app="loki"}' | jq '.'
{
"status": "success",
@ -792,38 +791,38 @@ $ curl -s "http://localhost:3100/loki/api/v1/series" --data-urlencode 'match[]={
}
```
## Index Stats
The `/loki/api/v1/index/stats` endpoint can be used to query the index for the number of `streams`, `chunks`, `entries`, and `bytes` that a query resolves to.
URL query parameters:
- `query`: The [LogQL](../logql/) matchers to check (i.e. `{job="foo", env!="dev"}`)
- `query`: The [LogQL]({{< relref "../logql/" >}}) matchers to check (i.e. `{job="foo", env!="dev"}`)
- `start=<nanosecond Unix epoch>`: Start timestamp.
- `end=<nanosecond Unix epoch>`: End timestamp.
You can URL-encode these parameters directly in the request body by using the POST method and `Content-Type: application/x-www-form-urlencoded` header. This is useful when specifying a large or dynamic number of stream selectors that may breach server-side URL character limits.
Response:
```json
{
"streams": 100,
"chunks": 1000,
"entries": 5000,
"bytes": 100000,
"bytes": 100000
}
```
It is an approximation with the following caveats:
* It does not include data from the ingesters
* It is a probabilistic technique
* streams/chunks which span multiple period configurations may be counted twice.
- It does not include data from the ingesters
- It is a probabilistic technique
- streams/chunks which span multiple period configurations may be counted twice.
These make it generally more helpful for larger queries.
It can be used for better understanding the throughput requirements and data topology for a list of matchers over a period of time.
## Statistics
Query endpoints such as `/api/prom/query`, `/loki/api/v1/query` and `/loki/api/v1/query_range` return a set of statistics about the query execution. Those statistics allow users to understand the amount of data processed and at which speed.
@ -837,7 +836,7 @@ The example belows show all possible statistics returned with their respective d
"resultType": "streams",
"result": [],
"stats": {
"ingester" : {
"ingester": {
"compressedBytes": 0, // Total bytes of compressed chunks (blocks) processed by ingesters
"decompressedBytes": 0, // Total bytes decompressed and processed by ingesters
"decompressedLines": 0, // Total lines decompressed and processed by ingesters
@ -851,7 +850,7 @@ The example belows show all possible statistics returned with their respective d
},
"store": {
"compressedBytes": 0, // Total bytes of compressed chunks (blocks) processed by the store
"decompressedBytes": 0, // Total bytes decompressed and processed by the store
"decompressedBytes": 0, // Total bytes decompressed and processed by the store
"decompressedLines": 0, // Total lines decompressed and processed by the store
"chunksDownloadTime": 0, // Total time spent downloading chunks in seconds (float)
"totalChunksRef": 0, // Total chunks found in the index for the current query
@ -863,8 +862,8 @@ The example belows show all possible statistics returned with their respective d
"execTime": 0, // Total execution time in seconds (float)
"linesProcessedPerSecond": 0, // Total lines processed per second
"queueTime": 0, // Total queue time in seconds (float)
"totalBytesProcessed":0, // Total amount of bytes processed overall for this request
"totalLinesProcessed":0 // Total amount of lines processed overall for this request
"totalBytesProcessed": 0, // Total amount of bytes processed overall for this request
"totalLinesProcessed": 0 // Total amount of lines processed overall for this request
}
}
}
@ -885,7 +884,6 @@ Displays a web page with the ruler hash ring status, including the state, health
### List rule groups
```
GET /loki/api/v1/rules
```
@ -972,6 +970,7 @@ Creates or updates a rule group. This endpoint expects a request with `Content-T
#### Example request
Request headers:
- `Content-Type: application/yaml`
Request body:
@ -1044,16 +1043,16 @@ PUT /loki/api/v1/delete
```
Create a new delete request for the authenticated tenant.
The [log entry deletion](../operations/storage/logs-deletion/) documentation has configuration details.
The [log entry deletion]({{< relref "../operations/storage/logs-deletion/" >}}) documentation has configuration details.
Log entry deletion is supported _only_ when the BoltDB Shipper is configured for the index store.
Query parameters:
* `query=<series_selector>`: query argument that identifies the streams from which to delete with optional line filters.
* `start=<rfc3339 | unix_seconds_timestamp>`: A timestamp that identifies the start of the time window within which entries will be deleted. This parameter is required.
* `end=<rfc3339 | unix_seconds_timestamp>`: A timestamp that identifies the end of the time window within which entries will be deleted. If not specified, defaults to the current time.
* `max_interval=<duration>`: The maximum time period the delete request can span. If the request is larger than this value, it is split into several requests of <= `max_interval`. Valid time units are `s`, `m`, and `h`.
- `query=<series_selector>`: query argument that identifies the streams from which to delete with optional line filters.
- `start=<rfc3339 | unix_seconds_timestamp>`: A timestamp that identifies the start of the time window within which entries will be deleted. This parameter is required.
- `end=<rfc3339 | unix_seconds_timestamp>`: A timestamp that identifies the end of the time window within which entries will be deleted. If not specified, defaults to the current time.
- `max_interval=<duration>`: The maximum time period the delete request can span. If the request is larger than this value, it is split into several requests of <= `max_interval`. Valid time units are `s`, `m`, and `h`.
A 204 response indicates success.
@ -1084,7 +1083,7 @@ GET /loki/api/v1/delete
```
List the existing delete requests for the authenticated tenant.
The [log entry deletion](../operations/storage/logs-deletion/) documentation has configuration details.
The [log entry deletion]({{< relref "../operations/storage/logs-deletion/" >}}) documentation has configuration details.
Log entry deletion is supported _only_ when the BoltDB Shipper is configured for the index store.
@ -1121,7 +1120,7 @@ DELETE /loki/api/v1/delete
```
Remove a delete request for the authenticated tenant.
The [log entry deletion](../operations/storage/logs-deletion/) documentation has configuration details.
The [log entry deletion]({{< relref "../operations/storage/logs-deletion/" >}}) documentation has configuration details.
Loki allows cancellation of delete requests until the requests are picked up for processing. It is controlled by the `delete_request_cancel_period` YAML configuration or the equivalent command line option when invoking Loki. To cancel a delete request that has been picked up for processing or is partially complete, pass the `force=true` query parameter to the API.
@ -1135,8 +1134,8 @@ DELETE /loki/api/v1/delete
Query parameters:
* `request_id=<request_id>`: Identifies the delete request to cancel; IDs are found using the `delete` endpoint.
* `force=<boolean>`: When the `force` query parameter is true, partially completed delete requests will be canceled. NOTE: some data from the request may still be deleted and the deleted request will be listed as 'processed'
- `request_id=<request_id>`: Identifies the delete request to cancel; IDs are found using the `delete` endpoint.
- `force=<boolean>`: When the `force` query parameter is true, partially completed delete requests will be canceled. NOTE: some data from the request may still be deleted and the deleted request will be listed as 'processed'
A 204 response indicates success.
@ -1168,9 +1167,9 @@ curl -u "Tenant1:$API_TOKEN" \
`/api/prom/tail` is a WebSocket endpoint that will stream log messages based on
a query. It accepts the following query parameters in the URL:
- `query`: The [LogQL](../logql/) query to perform
- `query`: The [LogQL]({{< relref "../logql/" >}}) query to perform
- `delay_for`: The number of seconds to delay retrieving logs to let slow
loggers catch up. Defaults to 0 and cannot be larger than 5.
loggers catch up. Defaults to 0 and cannot be larger than 5.
- `limit`: The max number of entries to return
- `start`: The start time for the query as a nanosecond Unix epoch. Defaults to one hour ago.
@ -1217,11 +1216,11 @@ will be sent over the WebSocket multiple times.
`/api/prom/query` supports doing general queries. The URL query parameters
support the following values:
- `query`: The [LogQL](../logql/) query to perform
- `query`: The [LogQL]({{< relref "../logql/" >}}) query to perform
- `limit`: The max number of entries to return
- `start`: The start time for the query as a nanosecond Unix epoch. Defaults to one hour ago.
- `end`: The end time for the query as a nanosecond Unix epoch. Defaults to now.
- `since`: A `duration` used to calculate `start` relative to `end`. If `end` is in the future, `start` is calculated as this duration before now. Any value specified for `start` supersedes this parameter.
- `since`: A `duration` used to calculate `start` relative to `end`. If `end` is in the future, `start` is calculated as this duration before now. Any value specified for `start` supersedes this parameter.
- `direction`: Determines the sort order of logs. Supported values are `forward` or `backward`. Defaults to `backward.`
- `regexp`: a regex to filter the returned results
@ -1255,7 +1254,7 @@ See [statistics](#statistics) for information about the statistics returned by L
#### Examples
```bash
```console
$ curl -G -s "http://localhost:3100/api/prom/query" --data-urlencode 'query={foo="bar"}' | jq
{
"streams": [
@ -1289,7 +1288,7 @@ the URL:
- `start`: The start time for the query as a nanosecond Unix epoch. Defaults to 6 hours ago.
- `end`: The end time for the query as a nanosecond Unix epoch. Defaults to now.
- `since`: A `duration` used to calculate `start` relative to `end`. If `end` is in the future, `start` is calculated as this duration before now. Any value specified for `start` supersedes this parameter.
- `since`: A `duration` used to calculate `start` relative to `end`. If `end` is in the future, `start` is calculated as this duration before now. Any value specified for `start` supersedes this parameter.
In microservices mode, `/api/prom/label/<name>/values` is exposed by the querier.
@ -1306,7 +1305,7 @@ Response:
#### Examples
```bash
```console
$ curl -G -s "http://localhost:3100/api/prom/label/foo/values" | jq
{
"values": [
@ -1326,7 +1325,7 @@ accepts the following query parameters in the URL:
- `start`: The start time for the query as a nanosecond Unix epoch. Defaults to 6 hours ago.
- `end`: The end time for the query as a nanosecond Unix epoch. Defaults to now.
- `since`: A `duration` used to calculate `start` relative to `end`. If `end` is in the future, `start` is calculated as this duration before now. Any value specified for `start` supersedes this parameter.
- `since`: A `duration` used to calculate `start` relative to `end`. If `end` is in the future, `start` is calculated as this duration before now. Any value specified for `start` supersedes this parameter.
In microservices mode, `/api/prom/label` is exposed by the querier.
@ -1343,7 +1342,7 @@ Response:
#### Examples
```bash
```console
$ curl -G -s "http://localhost:3100/api/prom/label" | jq
{
"values": [
@ -1384,13 +1383,11 @@ JSON post body can be sent in the following format:
}
```
Loki can be configured to [accept out-of-order writes](../configuration/#accept-out-of-order-writes).
In microservices mode, `/api/prom/push` is exposed by the distributor.
#### Examples
```bash
```console
$ curl -H "Content-Type: application/json" -XPOST -s "https://localhost:3100/api/prom/push" --data-raw \
'{"streams": [{ "labels": "{foo=\"bar\"}", "entries": [{ "ts": "2018-12-18T08:28:06.801064-04:00", "line": "fizzbuzz" }] }]}'
```