Commit Graph

451 Commits

Author SHA1 Message Date
Patrick Mann
8ff1300a42 Enforce DB 7.0 and remove 5.0 references (#23578)
* enforce DB 7.0 and remove 5.0 references

* bump mongo version for datanode

* add links to Mongo upgrade docs

* more upgrade guidance

---------

Co-authored-by: Anton Ebel <anton.ebel@graylog.com>
2025-09-19 18:33:00 +02:00
Bernd Ahlers
28a7b6691d Remove WithBeanGetter annotation (#23551)
* Replace WithBeanGetterTest with AutoValueValidatorTest

We want to ensure that auto-value adds the validation annotations to the
generated class fields.

* Remove all WithBeanGetter annotation usages

- Fix classes where the constructor became private because of the
  builder usage
- Fix wrong "$AutoValue" references

* Remove auto-value-javabean dependency

* Restore InputCreateRequest#create method

* Revert some JsonSeserialize conversions

The changes introduced a breaking change in InputCreateRequest, so we
play safe for now.

* Remove JsonDeserialize from ShardRouting

* Convert CreateEntityRequest to a POJO to fix validation issues

* Check that CreateEntityRequest#title is not null

* Fix botched merge

* Explicitly set JsonProperty names to fix parsing issue

* Remove unused files

* Use explicit values for JsonProperty annotations

Removing the WithBeanGetter annotation resulted in some de/serialization
errors. We prevent those by using explicit annotation values.

* Fix wrong property name

* Add PMD rule to check for missing values in JsonProperty annotations

* Remove newly introduced WithBeanGetter usages

* Add missing license header

* Fix "$AutoValue_*" usage
2025-09-18 18:03:52 +02:00
Matthias Oesterheld
d7a0cf4fee Bump OpenSearch to 2.19.3 (#21537)
* use os 2.19.0-rc4 in data node

* use 2.18.0 version of repository-s3 plugin

* fix s3 plugin version

* use repository-s3 from build

* remove expired certificates from data node truststore

* use ga release

* disable system indices temporarily because replicas are not set correctly in single-node installations (see https://github.com/opensearch-project/security/issues/4739)

* remove query-insights plugin

* do not use shaded lucene classes to avoid running into UnsupportedOperationException - if the singleton getter is not called from known Lucene classes in the lucene `VectorizationProvider` called by `DirectoryReader#open`

* use OS 2.19.0 shaded clients

* use patched repository-s3 (see
https://github.com/opensearch-project/OpenSearch/issues/17363)

* add lucene-backward-codecs

* use OS 2.19.1

* update to 2.19.3

* correct testcontainers version

* change order of deleting os plugins

* temporarily use 2.19.2

* temporarily use 2.19.2

* switch back to 2.19.3, temporarily disable remote reindexing it

* Changelog

* update containermatrix os server to 2.19.3

* adjust hdfs plugin hash

* Better error message in test

* add more verbose failure message

* change default_number_of_indices for test OS

* add setting only for 2.19.3

---------

Co-authored-by: Tomas Dvorak <tomas.dvorak@graylog.com>
2025-09-18 15:22:28 +02:00
Tomas Dvorak
8ff655a801 Install repository-hdfs plugin in datanode (#23507)
* Install repository-hdfs plugin in datanode/opensearch

* fixed sha for hdfs download

* Added HdfsRepositoryConfiguration and support for snapshots in datanode config

* added changelog

* fixed changelog

* fixed typo
2025-09-04 16:55:37 +02:00
Jenkins
9cfd72871a [graylog2-server] prepare for next development iteration 2025-08-29 13:16:04 +02:00
Jenkins
d59c56b2c1 [graylog2-server] prepare release 7.0.0-alpha.2 2025-08-29 12:57:02 +02:00
Bernd Ahlers
5dc5cc8740 Update to Java 21 (#22908)
* Remove maven.compiler.* properties from plugin archetype

They are already defined in org.graylog:graylog-parent

* Maven: update java compiler version from 17 to 21

* Update java in Github workflows

* Fix deprecated usage of URL constructor

* Fix deprecated usage of Thread#id

* Catch IllegalArgumentException when URLs are not absolute

* Remove PMD plugin for lack of java 21 support

* Update to Java 21

* Inherig compiler source/target settings from parent

* Use Java 21 in test containers

* Fix URL deprecation error for Java 21

* Setup Mockito Java Agent for surefire

See "Explicitly setting up instrumentation for inline mocking (Java 21+)"
at https://javadoc.io/doc/org.mockito/mockito-core/latest/org.mockito/org/mockito/Mockito.html#0.3

* Switch dependency plugin to test phase and fix argLine

* Get dependency properties in test-compile phase

* Add changelog entry and adjust UPGRADING.md

---------

Co-authored-by: Othello Maurer <othello@graylog.com>
2025-08-18 12:42:27 +02:00
Tomas Dvorak
d4ad323a14 Opensearch runtime policy for log rotation (#23347)
* Opensearch runtime policy for log rotation

* Fixed usage of opensearch.policy

* Fixed usage of opensearch.policy

* added changelog

* Fixed url to opensearch.policy file

* Fixed test

---------

Co-authored-by: Matthias Oesterheld <33032967+moesterheld@users.noreply.github.com>
2025-08-14 13:20:45 +02:00
Matthias Oesterheld
b70522ae9c Data Node: fix initial manager node list (#23319)
* fix initial manager node list

* add test
2025-08-11 14:32:30 +02:00
Tomas Dvorak
67f69fe01e Workaround for broken clock skew tolerance in opensearch (#23355)
* Workaround for broken clock skew tolerance in opensearch

* fixed test

* extended unit test
2025-08-11 13:44:27 +02:00
Tomas Dvorak
661006a7b9 Remove duplicated Configuration bean from datanode (#23340)
Co-authored-by: Matthias Oesterheld <33032967+moesterheld@users.noreply.github.com>
2025-08-07 15:30:33 +02:00
Tomas Dvorak
c97dc7a0af Refactoring/datanode keystore cleanup (#23259)
* add new Cluster Configuration Reader role

* cl

* Datanode keystore code cleanup

* fixed import

---------

Co-authored-by: Matthias Oesterheld <matthias.oesterheld@graylog.com>
Co-authored-by: Matthias Oesterheld <33032967+moesterheld@users.noreply.github.com>
2025-08-07 15:28:17 +02:00
Matthias Oesterheld
dafeafd3bd always add self to initial manager nodes if role is set (#23284) 2025-07-29 07:53:16 +02:00
Jenkins
4fc979053f [graylog2-server] prepare for next development iteration 2025-07-28 13:48:14 +02:00
Jenkins
dac42698ec [graylog2-server] prepare release 7.0.0-alpha.1 2025-07-28 13:28:19 +02:00
Tomas Dvorak
84c483454f Datanode: opensearch heap validator (#23246)
* opensearch heap validator

* added changelog
2025-07-25 16:13:59 +02:00
Tomas Dvorak
20c13c8228 Fix list of opensearch initial manager nodes in datanode (#23276)
* Fix list of opensearch initial manager nodes in datanode

* Added changelog

* fixed test
2025-07-25 13:11:18 +02:00
Matthias Oesterheld
9d55e113d5 temporarily disable DatanodeClusterIT (#23271) 2025-07-24 09:14:29 +02:00
Tomas Dvorak
2d72d93250 Better conversion and validation of path_repo in datanode (#23116)
* Better conversion and validation of path_repo in datanode

* added changelog

* Fixed test
2025-07-10 15:58:32 +02:00
Tomas Dvorak
0d0e2c58ee Deduplicate opensearch roles in datanode (#23100)
* Deduplicate opensearch roles in datanode #22569

* added test, fixed impl

* added changelog
2025-07-08 13:41:30 +02:00
Matthias Oesterheld
d8f447e492 Download and use static jdk for OpenSearch with data node on Mac (#22993)
* download and use static jdk for os with data node on mac

* move opensearch java home resolution for mac to OpensearchDistribution

---------

Co-authored-by: Tomas Dvorak <tomas.dvorak@graylog.com>
2025-06-30 10:40:05 +02:00
Tomas Dvorak
8f0c4c69eb move datanode keystore init logic out of preflight checks (#22801)
Co-authored-by: Jan Heise <jan.heise@graylog.com>
2025-06-25 12:55:25 +02:00
Tomas Dvorak
cd3849c8bc Filter out duplicated configuration beans for documentation purposes (#22902)
* Filter out duplicated configuration beans for documentation purposes

* code simplifications
2025-06-17 16:34:59 +02:00
github-actions[bot]
17468be528 Bump to 7.0.0-SNAPSHOT 2025-06-16 12:12:52 +00:00
Jenkins
84e794377f [graylog2-server] prepare for next development iteration 2025-06-16 13:46:08 +02:00
Jenkins
a4d383c613 [graylog2-server] prepare release 6.3.0-rc.1 2025-06-16 13:28:04 +02:00
Jenkins
cca9396e0d [graylog2-server] prepare for next development iteration 2025-06-10 13:39:16 +02:00
Jenkins
000fdd91d0 [graylog2-server] prepare release 6.3.0-beta.5 2025-06-10 13:19:38 +02:00
Tomas Dvorak
ef10dc42cb add node name to datanode certificate SAN (#22656)
* add node name to datanode certificate SAN

* added changelog
2025-06-06 10:52:38 +02:00
dependabot[bot]
28693ea083 Bump org.codehaus.mojo:build-helper-maven-plugin from 3.6.0 to 3.6.1 (#22797)
Bumps [org.codehaus.mojo:build-helper-maven-plugin](https://github.com/mojohaus/build-helper-maven-plugin) from 3.6.0 to 3.6.1.
- [Release notes](https://github.com/mojohaus/build-helper-maven-plugin/releases)
- [Commits](https://github.com/mojohaus/build-helper-maven-plugin/compare/3.6.0...3.6.1)

---
updated-dependencies:
- dependency-name: org.codehaus.mojo:build-helper-maven-plugin
  dependency-version: 3.6.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-05 07:08:13 +02:00
Matthias Oesterheld
6c0c816fc2 Fix data node native lib dir location (#22791)
* set native lib dir inside of config dir to make sure it is on a writable, non no-exec mounted fs

* cl

* add pr to cl
2025-06-04 14:27:40 +02:00
Jenkins
9f846d1632 [graylog2-server] prepare for next development iteration 2025-06-02 13:39:57 +02:00
Jenkins
5cb4bf5c27 [graylog2-server] prepare release 6.3.0-beta.4 2025-06-02 13:21:39 +02:00
dependabot[bot]
ee6ec212bb Bump org.codehaus.mojo:exec-maven-plugin from 3.5.0 to 3.5.1 (#22710)
Bumps [org.codehaus.mojo:exec-maven-plugin](https://github.com/mojohaus/exec-maven-plugin) from 3.5.0 to 3.5.1.
- [Release notes](https://github.com/mojohaus/exec-maven-plugin/releases)
- [Commits](https://github.com/mojohaus/exec-maven-plugin/compare/3.5.0...3.5.1)

---
updated-dependencies:
- dependency-name: org.codehaus.mojo:exec-maven-plugin
  dependency-version: 3.5.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-02 12:12:22 +02:00
Bernd Ahlers
ce7863009d Add custom MongoDB interface (#22679)
* Add custom MongoDB interface

We need more control over MongoCollection methods that modify data.

* Fix imports for LegacyKDatanodeKeystoreProvider

* Add more methods to our custom MongoCollection interface

* Fix type arguments in MongoCollection#getOrCreate and add tests

* Remove MongoCollection#getCollectionName

We can get the collection name via #getNamespace and we only have a
single consumer.

* Add missing license header

* Use more descriptive parameter names in MongoCollection

Basically mirror what the MongoDB MongoCollection class uses.

* Remove unused method from MongoCollection

* Add Javadoc to MongoCollection and rename params for consistency

* Remove commented methods in MongoEntityCollection

---------

Co-authored-by: Othello Maurer <othello@graylog.com>
2025-05-28 14:47:18 +02:00
Jenkins
f0e82a976c [graylog2-server] prepare for next development iteration 2025-05-26 14:59:11 +02:00
Jenkins
0b368d7338 [graylog2-server] prepare release 6.3.0-beta.3 2025-05-26 14:40:53 +02:00
Jenkins
6974930cfe [graylog2-server] prepare for next development iteration 2025-05-26 13:37:27 +02:00
Jenkins
40f548e67a [graylog2-server] prepare release 6.3.0-beta.2 2025-05-26 13:19:02 +02:00
Matthias Oesterheld
f331bb9172 Data node: Use same environment for OS and OS cli (#22654)
* remove usage of separate environment in os cli than os processes

* do not pass in system environment for test/plugin management
2025-05-22 13:14:57 +02:00
Matthias Oesterheld
d509ff7649 set OPENSEARCH_JAVA_HOME to system jdk (#22623) 2025-05-20 14:50:30 +02:00
Jenkins
dc562aa772 [graylog2-server] prepare for next development iteration 2025-05-19 13:42:17 +02:00
Jenkins
9971fa995e [graylog2-server] prepare release 6.3.0-beta.1 2025-05-19 13:23:23 +02:00
Tomas Dvorak
18c4716358 Explicitly configure OPENSEARCH_JAVA_HOME in datanode (#22534)
* Explicitly configure OPENSEARCH_JAVA_HOME in datanode

* Code cleanup

* Added changelog

---------

Co-authored-by: Matthias Oesterheld <33032967+moesterheld@users.noreply.github.com>
2025-05-16 15:56:46 +02:00
Jenkins
496d040a36 [graylog2-server] prepare for next development iteration 2025-05-13 13:37:20 +02:00
Jenkins
f3f1429f00 [graylog2-server] prepare release 6.3.0-alpha.4 2025-05-13 13:18:25 +02:00
Jenkins
b3cc660efe [graylog2-server] prepare for next development iteration 2025-05-07 16:17:27 +02:00
Jenkins
d1561d6f30 [graylog2-server] prepare release 6.3.0-alpha.3 2025-05-07 15:58:33 +02:00
Jenkins
6cace88acc [graylog2-server] prepare for next development iteration 2025-04-30 17:14:03 +02:00
Jenkins
4f798b75a7 [graylog2-server] prepare release 6.3.0-alpha.2 2025-04-30 16:55:11 +02:00