Christian Haudum 27411cff08 Introduce worker queue in bloom gateway (#10976) ...

Instead of calling the bloom store directly on each and every request to filter chunk refs based on the given filters, we want to queue requests in per-tenant queues and process batches of requests that can be multiplexed to avoid excessive seeking in the bloom block queriers when checking chunk matches.

This PR re-uses the request queue implementation used in the query scheduler. To do so, it moves the queue related code from the scheduler into a separate package `pkg/queue` and renames occurrences of "querier" to "consumer" to be more generic.

The bloom gateway instantiates the request queue when starting the service. The gRPC method `FilterChunkRefs` then enqueues incoming requests to that queue.

**Special notes for your reviewer**:

For testing purposes, this PR also contains a dummy implementation of the workers. The worker implementation - which includes multiplexing of multiple tasks - is subject to a separate PR.

---------

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>

2023-10-20 15:27:55 +02:00

3.3 KiB

Raw Blame History

View Raw