Ashwanth
5203a0c9ad
chore(dataobj): add reader stats ( #18694 )
2025-08-14 18:29:02 +05:30
George Robinson
414538ae24
fix: mistake in error, should have said logs section ( #18844 )
2025-08-14 06:47:18 -04:00
George Robinson
36b2615156
feat: decode tenant when reading streams and logs sections ( #18846 )
2025-08-14 09:33:16 +00:00
George Robinson
0941b90614
fix: dataobj-consumers did not commit offsets on idle flush ( #18834 )
2025-08-14 10:30:33 +01:00
George Robinson
916246a2be
feat: add tenant to section metadata ( #18843 )
2025-08-14 10:25:31 +01:00
Ashwanth
88c8b44f2d
chore(dataobj): read non-projected predicate columns ( #18797 )
2025-08-14 11:25:47 +05:30
George Robinson
8895090643
fix: dataobj-consumers commit incorrect offset, which can cause data loss ( #18833 )
2025-08-13 15:33:34 +01:00
Robert Fratto
100e259a8d
chore(dataobj): introduce the concept of section info "extensions" ( #18832 )
2025-08-13 10:14:04 -04:00
benclive
e57877e208
chore: Refactor metastore Updater (breaking change) ( #18821 )
2025-08-13 15:03:00 +01:00
Robert Fratto
61c40fca0c
chore(dataobj): remove support for deprecated fields (breaking change) ( #18822 )
2025-08-13 08:16:18 -04:00
Robert Fratto
2ca8e58b0e
chore(dataobj): add ability to buffer pending sections to disk ( #18780 )
...
Signed-off-by: Robert Fratto <robertfratto@gmail.com >
2025-08-12 11:57:04 -04:00
George Robinson
53416a65ef
chore: use quartz.Clock to improve tests ( #18810 )
2025-08-12 16:35:31 +01:00
Robert Fratto
40057f5ce2
chore(dataobj): permit reading subset of section metadata ( #18815 )
...
Signed-off-by: Robert Fratto <robertfratto@gmail.com >
2025-08-12 10:12:56 -04:00
George Robinson
69a70dda88
chore: use zero value for time in partition_processor.go ( #18798 )
2025-08-12 08:42:14 +01:00
George Robinson
c47841024e
chore: move mocks into mock.go ( #18799 )
2025-08-12 08:32:50 +01:00
Periklis Tsirakidis
0ec355c349
chore(dataobj): Use storage prefix for metastore index lookups ( #18555 )
2025-08-11 11:07:37 +00:00
Robert Fratto
1c06a8fc11
chore(dataobj): fix compilation error following #18622 ( #18738 )
2025-08-06 18:57:29 +00:00
Robert Fratto
d4b027ae0c
chore(dataobj): return Object on Builder.Flush ( #18622 )
2025-08-06 12:52:28 -04:00
Robert Fratto
a5c722cbed
chore(dataobj): list pages across all columns in one request ( #18678 )
2025-07-31 10:10:11 -04:00
Ashwanth
3faec029bf
chore(metastore): include traceID in metastore logs ( #18660 )
2025-07-31 15:33:13 +05:30
benclive
613116e371
chore: Fix empty matchers returning all entries in Metastore Section lookup ( #18642 )
2025-07-30 11:37:02 +01:00
Robert Fratto
517b59f087
chore: update github.com/thanos-io/objstore fork ( #18620 )
2025-07-28 13:36:31 -04:00
Robert Fratto
9129d8cdd0
chore(engine): use columnar logs.Reader for DataObjScan ( #18577 )
2025-07-25 15:23:55 -04:00
Robert Fratto
061d1c1d22
chore(dataobj): make copy of pages from downloaded window ( #18566 )
2025-07-24 10:55:28 -04:00
Ashwanth
da4e59f8c6
chore(dataobj): store sort order information in section metadata ( #18499 )
2025-07-24 18:25:36 +05:30
benclive
41d93f8a86
fix: Fix empty streamIDs in multi-section index objects ( #18556 )
2025-07-24 13:30:54 +01:00
Christian Haudum
ce03b7943a
chore(dataobj): Improve default builder target sizes and settings help ( #18543 )
...
Explain which settings are based on compressed and which are based on uncompressed size and why.
Signed-off-by: Christian Haudum <christian.haudum@gmail.com >
2025-07-23 18:11:30 +02:00
benclive
f075da4f94
chore: Use correct TimeRange predicate for indexpointers ( #18529 )
2025-07-23 07:21:27 +02:00
Robert Fratto
009dd388f2
chore(dataobj/consumer): track latest delay to Kafka ( #18528 )
2025-07-22 13:54:32 -04:00
benclive
798f77892e
chore: Fix slicelabels merge conflict in calculator.go ( #18517 )
2025-07-21 12:46:23 +01:00
benclive
077e4196ef
chore: Add indexes to LogQL benchmarks ( #18500 )
2025-07-18 19:03:41 +00:00
Robert Fratto
86c57c8616
chore: consistently use new labels.Labels API ( #18490 )
2025-07-18 13:15:40 -04:00
benclive
517c317c97
chore: Integrate indexes with the new query engine ( #18427 )
2025-07-17 15:04:40 +00:00
benclive
d0a3cd86a7
chore: Remove lookback from stats in unsupported sections ( #18489 )
2025-07-17 16:00:37 +01:00
Periklis Tsirakidis
17d6853535
chore: Add support for indexpointers section ( #18485 )
2025-07-17 16:52:11 +02:00
Periklis Tsirakidis
bcf24e4cfb
chore(dataobj): Use indexpointer sections in metastore reads and writes ( #18448 )
2025-07-15 15:29:22 +02:00
Periklis Tsirakidis
6eedcaaded
chore(dataobj): Add new index pointers section type ( #18441 )
2025-07-14 15:27:41 +02:00
benclive
1f0edcd4ac
feat: Initial index-builder implementation ( #18297 )
2025-07-11 15:21:38 +01:00
Christian Haudum
d9f7d785ff
chore(engine): Sort entries in dataobj sections by timestamp descending ( #18367 )
...
Sorting sections by `timestamp DESC` first and `streamID ASC` second allows for more efficient reading (by section) for BACKWARD log queries.
- This change breaks FORWARD log queries. Which is ok for the time being. We will add support later again when columnar reading of the dataobj is introduced.
- This change renders existing dataobj in storage incompatible.
Signed-off-by: Christian Haudum <christian.haudum@gmail.com >
2025-07-11 07:18:20 +00:00
benclive
5429647b81
chore: Add pointers to dataobj-inspect UI ( #18372 )
2025-07-09 15:47:32 +01:00
benclive
f66781d426
fix: Row reader with no stream ID predicates ( #18368 )
2025-07-09 11:29:54 +00:00
Robert Fratto
26cb50e65a
chore(dataobj): Improve performance of dataset.Value API ( #18339 )
...
Co-authored-by: Ashwanth <iamashwanth@gmail.com >
2025-07-08 09:40:31 -04:00
benclive
a238816751
perf: Use map for InPredicate when reading dataobj ( #18325 )
2025-07-07 09:45:11 +01:00
Robert Fratto
807f06baf3
chore(dataobj/uploader): expose logs for uploaded objects ( #18338 )
2025-07-04 09:34:18 -04:00
Robert Fratto
9c3d86be52
chore(dataobj): cut logs section based on uncompressed size ( #18329 )
2025-07-03 12:04:15 -04:00
Ashwanth
5c02493fe2
chore(dataobj): avoid reading a predicate column more than once ( #18324 )
2025-07-03 18:52:59 +05:30
Christian Haudum
2cbbbd1f46
chore: Fixup metric names for dataobj-consumer and histogram bucket ( #18295 )
...
Signed-off-by: Christian Haudum <christian.haudum@gmail.com >
2025-07-01 15:03:08 +02:00
Christian Haudum
b1fc685397
chore: Extend instrumentation of dataobject uploader ( #18292 )
...
These changes provide better visibility into the dataobj uploader.
Signed-off-by: Christian Haudum <christian.haudum@gmail.com >
2025-07-01 13:40:21 +02:00
Robert Fratto
30ccf5172c
chore(util/arrowtest): export arrowtest package ( #18255 )
2025-06-27 09:55:01 -04:00
benclive
9642af3ac6
chore: Add new pointers section type for dataobj ( #18243 )
2025-06-27 14:00:51 +01:00
