Commit Graph

414 Commits

Author SHA1 Message Date
Jenkins
6974930cfe [graylog2-server] prepare for next development iteration 2025-05-26 13:37:27 +02:00
Jenkins
40f548e67a [graylog2-server] prepare release 6.3.0-beta.2 2025-05-26 13:19:02 +02:00
Matthias Oesterheld
f331bb9172 Data node: Use same environment for OS and OS cli (#22654)
* remove usage of separate environment in os cli than os processes

* do not pass in system environment for test/plugin management
2025-05-22 13:14:57 +02:00
Matthias Oesterheld
d509ff7649 set OPENSEARCH_JAVA_HOME to system jdk (#22623) 2025-05-20 14:50:30 +02:00
Jenkins
dc562aa772 [graylog2-server] prepare for next development iteration 2025-05-19 13:42:17 +02:00
Jenkins
9971fa995e [graylog2-server] prepare release 6.3.0-beta.1 2025-05-19 13:23:23 +02:00
Tomas Dvorak
18c4716358 Explicitly configure OPENSEARCH_JAVA_HOME in datanode (#22534)
* Explicitly configure OPENSEARCH_JAVA_HOME in datanode

* Code cleanup

* Added changelog

---------

Co-authored-by: Matthias Oesterheld <33032967+moesterheld@users.noreply.github.com>
2025-05-16 15:56:46 +02:00
Jenkins
496d040a36 [graylog2-server] prepare for next development iteration 2025-05-13 13:37:20 +02:00
Jenkins
f3f1429f00 [graylog2-server] prepare release 6.3.0-alpha.4 2025-05-13 13:18:25 +02:00
Jenkins
b3cc660efe [graylog2-server] prepare for next development iteration 2025-05-07 16:17:27 +02:00
Jenkins
d1561d6f30 [graylog2-server] prepare release 6.3.0-alpha.3 2025-05-07 15:58:33 +02:00
Jenkins
6cace88acc [graylog2-server] prepare for next development iteration 2025-04-30 17:14:03 +02:00
Jenkins
4f798b75a7 [graylog2-server] prepare release 6.3.0-alpha.2 2025-04-30 16:55:11 +02:00
Matthias Oesterheld
b478d9f7cf Add data node heap check (#20168)
* add data node hep check and notification

* fix template names

* suppress forbidden invocation as it is handled by exception handling

* add recommended heap setting to notification

* changelog

* switch to oshi

* use floating point arithmetic

* change system notification text

* fixed test

* add paragraphs

---------

Co-authored-by: Tomas Dvorak <tomas.dvorak@graylog.com>
2025-04-28 09:33:13 +02:00
Tomas Dvorak
3dd934ae31 Automatically generate datanode.conf.example (#19141)
* Automatically generate datanode.conf.example + csv documentation of configuration options

* code cleanup, tests

* add spaces around = in generated config example

* adjust spaces in generated config files, add tests for ordering

* Added support for sections - both on class level and field level

* Human readable required params in csv export of datanode docs

* distribute the manually created datanode.conf.example till we agree on generated format

* Use SPI to locate configuration beans for documentation

* Code cleanup around datanode documentation

* added comment

---------

Co-authored-by: Matthias Oesterheld <33032967+moesterheld@users.noreply.github.com>
2025-04-24 16:13:28 +02:00
Jenkins
0f5088213e [graylog2-server] prepare for next development iteration 2025-04-23 13:44:20 +02:00
Jenkins
7859409af6 [graylog2-server] prepare release 6.3.0-alpha.1 2025-04-23 13:25:52 +02:00
dependabot[bot]
01d799691d Bump org.apache.commons:commons-collections4 from 4.4 to 4.5.0 (#22390)
Bumps org.apache.commons:commons-collections4 from 4.4 to 4.5.0.

---
updated-dependencies:
- dependency-name: org.apache.commons:commons-collections4
  dependency-version: 4.5.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-04-23 12:12:14 +02:00
Matthias Oesterheld
ff099df468 Refactor data node process state machine (#21210)
* Initial reworked, simplified abstract node startup

* use Graylog CmdLineTool in data node

* remove unused pid file declaration

* remove freshInstallation check as it doesn't make sense since it checks for fresh installation of the Graylog cluster

* move mongo preflight check into regular preflight checks since it doesn't do the fresh installation detection any more

* remove migration feature for data node

* code cleanup

* remove unused local flag, code cleaup

* rename server to datanode for better clarification

* fix feature flag test to use config for env/sys props

* remove Graylog path configuration from data node configuration

* temporary fix for excluding zstd library fix from data node

* add requireexplicitbindings for new node types

* remove unneeded settings bean

* do not use `pluginLoaderConfig` for fixing zstd temp directory

* do tls protocols configuration and setting of netty defaults only in Graylog node

* add selective loading of plugins for a specific node type

* add trino node plugin skeleton

* remove ExampleCommand, replace with MinimalNode test

* change forbidden method invocation

* adjust log

Co-authored-by: Bernd Ahlers <bernd@users.noreply.github.com>

* Change cmdline option description

Co-authored-by: Bernd Ahlers <bernd@users.noreply.github.com>

* Change cmdline option description

Co-authored-by: Bernd Ahlers <bernd@users.noreply.github.com>

* Change cmdline option description

Co-authored-by: Bernd Ahlers <bernd@users.noreply.github.com>

* make usage of NodeIdFile configurable

* adjust ServerStatus and provide common node command test

* add missing method to test interface

* code cleanup

* make configuration a field, remove unused constructor

* code cleanup

* use parent field

* make Datanode and Server extend AbstractNodeCommand

* change field names to camel case

* remove redundant bindings, make journal commands extends AbstractNodeCommand

* remove static field

* remove redundant bindings

* move plugin path configuration out of PathConfiguration to not require all properties of PathConfiguration only for plugin loading

* always applySecuritySettings

* add javadocs to GraylogNodeModule

* add javadoc

* add test for plugin filtering

* code cleanup

* change path resolution

* add an abstraction layer for state machine handling to avoid duplicate code

* load plugin rest resources in data node jersey service

* Fix trino plugin configuration beans binding, add start endpoint

* change plugin loading from filtering to explicit override in data node

* prepare test backend for plugin loading

* adds a `datanodePluginJarsProvider` to ContainerMatrixTests to allow loading of datanode jars in full backend tests

---------

Co-authored-by: Tomas Dvorak <tomas.dvorak@graylog.com>
Co-authored-by: Bernd Ahlers <bernd@users.noreply.github.com>
2025-04-22 13:51:08 +02:00
Tomas Dvorak
af28c54dce Refactoring of IndexerJwtAuthTokenProvider (#22196)
* Refactoring of IndexerJwtAuthTokenProvider

* Fix DatanodeConnectivityCheck

* fixed jwt token usage in OpensearchRestClient

* add clock to the jwt token provider, so we can test expiration of tokens

* fixed rest client provider with jwt

* fixed error and sniffer logging

---------

Co-authored-by: Matthias Oesterheld <33032967+moesterheld@users.noreply.github.com>
2025-04-22 07:45:47 +02:00
Tomas Dvorak
b22016cd0a Simplify opensearch plugin installation from datanode pom.xml (#22309)
* Simplify opensearch plugin installation from datanode pom.xml

* Implement correct opensearch plugins removal

* fix plugin removal

* Better handling of env setting for opensearch cli

* code cleanup, typos

---------

Co-authored-by: Matthias Oesterheld <33032967+moesterheld@users.noreply.github.com>
2025-04-16 11:36:03 +02:00
Tomas Dvorak
efb8c3fdd7 Add support for google cloud storage in datanode snapshots (#22282)
* Add support for google cloud storage in datanode snapshots

* added license

* resolve relative gcs credential file paths

* datanode directories provider

* added license

* better docs

* more logging

* added changelog

---------

Co-authored-by: Matthias Oesterheld <33032967+moesterheld@users.noreply.github.com>
2025-04-15 15:08:39 +02:00
github-actions[bot]
252fe8b1ff Bump to 6.3.0-SNAPSHOT 2025-04-14 11:56:37 +00:00
Jenkins
01f139bf87 [graylog2-server] prepare for next development iteration 2025-04-14 13:44:02 +02:00
Jenkins
a672433a1f [graylog2-server] prepare release 6.2.0-rc.1 2025-04-14 13:26:14 +02:00
Jenkins
84fca955b5 [graylog2-server] prepare for next development iteration 2025-04-07 13:45:59 +02:00
Jenkins
651fffc35b [graylog2-server] prepare release 6.2.0-beta.4 2025-04-07 13:28:09 +02:00
Matthias Oesterheld
ccd2148521 Use nativeLibPath in data node (#22224)
* use nativeLibPath in data node

* cl

* revert unnecessary changes

* license header
2025-04-03 16:11:24 +02:00
Jenkins
42bf7433a1 [graylog2-server] prepare for next development iteration 2025-03-31 13:40:18 +02:00
Jenkins
ae28350ba7 [graylog2-server] prepare release 6.2.0-beta.3 2025-03-31 13:22:25 +02:00
Tomas Dvorak
af16eaa0e8 Fix/datanode cert renewal changed hostname (#22057)
* Rate-limit access to the datanode keystore in cert renewal task

* added changelog

* trigger cert renewal when datanode hostname changed

* changelog

* allow restarting opensearch process during startup

* Updated changelog
2025-03-28 16:58:41 +01:00
Tomas Dvorak
54266d0513 Rate-limit access to the datanode keystore in cert renewal task (#22054)
* Rate-limit access to the datanode keystore in cert renewal task

* added changelog
2025-03-26 11:13:03 +01:00
Jenkins
37d267d9a1 [graylog2-server] prepare for next development iteration 2025-03-24 13:38:38 +01:00
Jenkins
f5648775f7 [graylog2-server] prepare release 6.2.0-beta.2 2025-03-24 13:20:48 +01:00
Jenkins
0de5558de2 [graylog2-server] prepare for next development iteration 2025-03-17 14:18:22 +01:00
Jenkins
12da37d648 [graylog2-server] prepare release 6.2.0-beta.1 2025-03-17 14:00:36 +01:00
Tomas Dvorak
a6084c3851 Merge pull request #21491 from Graylog2/feature/password_secret_preflight_check
preflight check for password_secret
2025-03-14 08:34:36 +01:00
Matthias Oesterheld
7236956438 Merge pull request #21707 from Graylog2/feature/passthrough-opensearch-properties
support for opensearch config override properties
2025-03-13 14:35:26 +01:00
Matthias Oesterheld
8fb154facc compile patterns statically 2025-03-13 11:39:33 +01:00
Matthias Oesterheld
a173d53f81 Merge branch 'master' into feature/passthrough-opensearch-properties 2025-03-13 11:38:58 +01:00
Tomas Dvorak
4fe6a3798b Merge branch 'master' into fix/datanode-enterprise-cluster-events 2025-03-13 10:01:47 +01:00
Bernd Ahlers
d66a409da1 Don't include test dependencies in Data Node artifact (#21949)
Only include "runtime" scope in copy-dependencies configuration.
This saves about 40 MB from the artifact.
2025-03-13 09:41:00 +01:00
Matthias Oesterheld
64b9eb4d6e add index.codec to allowlist 2025-03-11 10:08:01 +01:00
Tomas Dvorak
4111fb98bb Merge pull request #21734 from Graylog2/refactor/restrict-shaded-lucene-usage
Restrict usage of shaded lucene classes
2025-03-11 09:13:43 +01:00
Matthias Oesterheld
eb8165c40a make opensearch_configuration_overrides_file optional and log error if it is set but doesn't exist 2025-03-11 09:06:05 +01:00
Matthias Oesterheld
ce9c0bb654 add centralized warning for unsupported properties.
If additional types of warnings are needed, we may need to add typed warnings
2025-03-11 07:46:47 +01:00
Matthias Oesterheld
b9ffcb00f8 remove config value from warning to avoid leaking sensitive information 2025-03-11 07:45:36 +01:00
Matthias Oesterheld
458df4c6c6 add dto to status to include in support bundle 2025-03-11 07:45:05 +01:00
Matthias Oesterheld
7ea78c4e30 adjust allowlist test 2025-03-10 17:12:34 +01:00
Matthias Oesterheld
ba1e3d3ae6 add allowlist for os override properties 2025-03-10 17:10:18 +01:00