* Refactor the store API to be in terms of DeleteRequest
* backfill remove tests
* backfill GetAll tests
* Make the delete request store deal in groups of requests instead of single ones
* Make cancelation aware of request groups
* remove cancelation deadline
* shard deletes based on a query parameter
* merge request group on gets
* global configuration for max query range
* sort delete requests by start time so batches are grouped to similar chunks
* Fix status when some deletes haven't happened yet
* fix tests
* Review feedback
* review feedback
* make max_interval > deletion interval a validation error
* list partially completed deletes
* list partially completed deletes
* fix flaky test
* clarify naming
* disallow cancelation of partial deletes unless the 'force' flag is passed
* more comment specificity for the delete request hash
* break out the middleware
* Add validation to the API
- Time must be in RFC3339 or a 10-digit-unix-seconds timestamp
- Start time must always exist
* lint
* docs
* improve tests
* cleanup
* cleanup
* access runtime config via function on validation
* Add API delete curl examples
* Update docs/sources/api/_index.md
Co-authored-by: Jennifer Villa <jvilla2013@gmail.com>
* Update docs/sources/api/_index.md
Co-authored-by: Jennifer Villa <jvilla2013@gmail.com>
* Update docs/sources/api/_index.md
Co-authored-by: Jennifer Villa <jvilla2013@gmail.com>
* Update docs/sources/api/_index.md
Co-authored-by: Jennifer Villa <jvilla2013@gmail.com>
Co-authored-by: Jennifer Villa <jvilla2013@gmail.com>
* Update grafana/dskit to 07166f9
Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
* Add /ingester/shutdown handler
This handler replaces the deprecated /ingester/flush_shutdown handler
and can be used to gracefully shut down a Loki instance and delete the
file that persists the tokens of the ingester ring.
In production environments you usually want to persist ring tokens so
that during a restart of an ingester instance, or during rollout, the
tokens from that instance are not re-distributed to other instances, but
instead kept so that the same streams end up on the same instance once
it is up and running again. For that, the tokens are written to a file
that can be specified via the `-ingester.tokens-file-path` argument.
In certain cases, however, you want to forget the tokens and
re-distribute them when shutting down an ingester instance. This was
already possible by calling `/ingester/flush_shutdown`, deleting the
tokens file and terminating the process. The new handler
`/ingester/shutdown` combines these manual steps into a
single handler.
Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
* Add changelog entry
Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
* docs: describe ordering in http query results
* improved text
Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>
* better text
Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>
* better text
Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>
Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>
* Add docs for updated deletion api
Signed-off-by: Michel Hollands <michel.hollands@grafana.com>
* Fix parameter name
Signed-off-by: Michel Hollands <michel.hollands@grafana.com>
* Clarify that only the delete requests for the current tenant are shown
Signed-off-by: Michel Hollands <michel.hollands@grafana.com>
* Update docs/sources/api/_index.md
Co-authored-by: Christian Simon <simon@swine.de>
* Update docs/sources/operations/storage/logs-deletion.md
Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>
* Update docs/sources/operations/storage/logs-deletion.md
Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>
* Update docs/sources/operations/storage/logs-deletion.md
Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>
* Specify all delete endpoints use authentication
Signed-off-by: Michel Hollands <michel.hollands@grafana.com>
* Add description of delete modes
Signed-off-by: Michel Hollands <michel.hollands@grafana.com>
* Apply suggestions from code review
Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>
Co-authored-by: Christian Simon <simon@swine.de>
Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>
Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>
* Documented /services API endpoint
Signed-off-by: Danny Kopping <danny.kopping@grafana.com>
* Grammar
Signed-off-by: Danny Kopping <danny.kopping@grafana.com>
* docs: update the labels timespan info
* updated docs
Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>
* updated docs
Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>
Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>
* Log time in queue per request
* Add querierID to log message
* Add unescape request url and tenant_ids to log message
* Add query enqueue time to metrics.go
* Remove "querier request dequeued" log from scheduler
* Address comments
* Rename enqueue_time to queue_time
* Store queue time in nanoseconds (int64)
* Use CanonicalMIMEHeaderKey for setting the httpgrpc header
* Implement new IngestionRateStrat Enum.
- This new IngestionRateStrat is used internally to represent the chosen
ingestion rate strategy, to avoid using plain strings.
* Add a ring to the distributor.
As of now, the distributor only has a pointer to its lifecycler. Since
the ring HTTP handler isn't implemented for the lifecycler but it is
for the ruler, this extends the distributor by also holding a ring
instance.
* Implement /distributor/ring page.
* Reuse existing global strat const.
* Add distributor ring HTTP page tests.
* Add missing CHANGELOG entry.
* Document missing ring status pages.
* Fork Cortex util/http.go into source code.
* update changelog, release notes, and upgrade guide for 2.4
* tweak release notes a little
* make path relative for kafka config
* update upgrade guide
* more docs
fix the cache size not applied to results cache.
* fix prefix name in ring_config
The product name is Grafana Loki. The first use of the product name on
each web page should be the full product name. After that, within each
page, it is fine to use just Loki.
This PR also set up some frontmatter with weights where it was missing
on a few files, and correct where "Loki" was not capitalized.
I merged their change before the CLA was signed. We reverted that in #4299 and now I'm reopening
Signed-off-by: Danny Kopping <danny.kopping@grafana.com>
As far as I can tell from reading the "next" documentation, the
/loki/api/v1/status/buildinfo endpoint should return the build
information. The /version endpoint is also listed further down in the
documentation but from the code prior to this change, it looks like it
should perform the same function as buildinfo endpoint so I have
removed it.
I've updated the documentation to reflect my changes.
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Return build info under `/version`.
Closes#3221
* Update documentation.
* Correct docs.
* Format version handler code.
* Ignore write error.
* Sort imports.
* Use `PrometheusVersion` struct.
* Set Go version.
- Revise section titles
- Add weight metadata to specify ordering of subsections
- Remove TOC info from content prose, as it is already in the right side navigation panel
- Remove a redundant section about upgrading.
