Commit Graph

324 Commits

Author SHA1 Message Date
Axel François
c24f3f4eef Update password hashing command in configuration files (#21263) 2025-01-07 13:15:32 +01:00
Tomas Dvorak
45e5c07e4c Better handling of intermediate CAs in datanode truststore (#21062)
* TruststoreCreator adds the whole certchain, code cleanup

* further truststore simplifications, unique aliases to prevent overriding already added certs

* added changelog
2024-12-18 14:29:10 +01:00
Tomas Dvorak
f197bbb9dc Code cleanup (#21177)
Co-authored-by: Matthias Oesterheld <33032967+moesterheld@users.noreply.github.com>
2024-12-17 13:08:01 +01:00
Matthias Oesterheld
b4e1e075f9 Refactor data node startup, add startup mechanism for new node types (#20181)
* Initial reworked, simplified abstract node startup

* use Graylog CmdLineTool in data node

* remove unused pid file declaration

* remove freshInstallation check as it doesn't make sense since it checks for fresh installation of the Graylog cluster

* move mongo preflight check into regular preflight checks since it doesn't do the fresh installation detection any more

* remove migration feature for data node

* code cleanup

* remove unused local flag, code cleaup

* rename server to datanode for better clarification

* fix feature flag test to use config for env/sys props

* remove Graylog path configuration from data node configuration

* temporary fix for excluding zstd library fix from data node

* add requireexplicitbindings for new node types

* remove unneeded settings bean

* do not use `pluginLoaderConfig` for fixing zstd temp directory

* do tls protocols configuration and setting of netty defaults only in Graylog node

* remove ExampleCommand, replace with MinimalNode test

* change forbidden method invocation

* adjust log

Co-authored-by: Bernd Ahlers <bernd@users.noreply.github.com>

* Change cmdline option description

Co-authored-by: Bernd Ahlers <bernd@users.noreply.github.com>

* Change cmdline option description

Co-authored-by: Bernd Ahlers <bernd@users.noreply.github.com>

* Change cmdline option description

Co-authored-by: Bernd Ahlers <bernd@users.noreply.github.com>

* make usage of NodeIdFile configurable

* adjust ServerStatus and provide common node command test

* code cleanup

* make configuration a field, remove unused constructor

* code cleanup

* use parent field

* make Datanode and Server extend AbstractNodeCommand

* change field names to camel case

* remove redundant bindings, make journal commands extends AbstractNodeCommand

* remove static field

* remove redundant bindings

* always applySecuritySettings

* add javadocs to GraylogNodeModule

---------

Co-authored-by: Tomas Dvorak <tomas.dvorak@graylog.com>
Co-authored-by: Bernd Ahlers <bernd@users.noreply.github.com>
2024-12-12 16:02:32 +01:00
Tomas Dvorak
a4c16342e0 Datanode/opensearch configuration beans (#21082)
* Preflight check for usable space and cache size

* add warning when search cache size occupies more than 80% of free disk space

* Added changelog

* unified searchable snapshots configuration for datanode

* removed node cache from OpensearchConfiguration (is managed by searchable snaphots config)

* code cleanup

* better node search cache documentation

* updated changelog

* Opensearch configuration beans refactoring

* code cleanup, renaming

* further configuration simplifications

* fixed configuration, seed hosts file moved under configuration

* fixed ssl configuration checks

* Added support for opensearch configuration files

* opensearch config dir handling overhaul, files cleanup, temp dirs

* Safer input stream creation for configuration files

* Fix opensearch configuration files copy from jar

* renamed and standardized configuration beans

* license

* Code cleanup

* move configuration beans out of opensearch package

---------

Co-authored-by: Matthias Oesterheld <33032967+moesterheld@users.noreply.github.com>
2024-12-12 10:32:11 +01:00
Tomas Dvorak
fe5102b3fb Unified snapshots configuration for datanode (#21031)
* Preflight check for usable space and cache size

* add warning when search cache size occupies more than 80% of free disk space

* Added changelog

* unified searchable snapshots configuration for datanode

* removed node cache from OpensearchConfiguration (is managed by searchable snaphots config)

* code cleanup

* better node search cache documentation

* updated changelog

---------

Co-authored-by: Matthias Oesterheld <33032967+moesterheld@users.noreply.github.com>
2024-12-03 08:03:04 +01:00
Tomas Dvorak
ab51ce8a33 Remove opensearch observability plugin from datanode (#21045) 2024-11-26 17:08:34 +01:00
Tomas Dvorak
ec1fd113c1 opensearch keystore used for key/truststore passwords (#20923)
* opensearch keystore used for key/truststore passwords

* code cleanup

* Add changelog

* Added integration test for opensearch keystore cli

* add `-x` to add OS keystore secrets, bypassing the prompt

---------

Co-authored-by: Matthias Oesterheld <matthias.oesterheld@graylog.com>
2024-11-15 17:02:02 +01:00
Matthias Oesterheld
d5bfa630c2 set opensearch.transport.cname_in_publish_address=true on startup (#20960) 2024-11-15 11:53:07 +01:00
Tomas Dvorak
671eacc4d3 Automatic cert renewal policy and self-signed CA creation (#20842)
* Automatic cert renewal policy and self-signed CA creation

* Datanode CSR rate limited, added IT

* selfsigned_startup property name for automatic cert renewal and selfsigned CA configuration

* Add warning to the insecure configuration

* default for relaxedHTTPSValidation in RestOperationParameters

* add changelog

* code cleanup

* Simplify rate limiting in DataNodeCertRenewalPeriodical

---------

Co-authored-by: Matthias Oesterheld <33032967+moesterheld@users.noreply.github.com>
2024-11-14 15:29:14 +01:00
Tomas Dvorak
60e1ca3dfb Empty dir error in datanode inplace migration (#20912)
* Empty dir error in datanode inplace migration

* better place for empty data dir compatibility check

* added changelog

* compatibility warning for inplace migration in case of empty data dir

* display warning

* rephrased empty datadir warning

* only show green alert when there are no warnings

---------

Co-authored-by: Mohamed Ould Hocine <mohamed.ould.hocine@graylog.com>
Co-authored-by: Matthias Oesterheld <33032967+moesterheld@users.noreply.github.com>
2024-11-14 14:14:51 +01:00
Tomas Dvorak
e44a4381aa removed dead code (#20920) 2024-11-11 10:02:15 +01:00
Tomas Dvorak
13162e47be Remove unused GRAYLOG_DATANODE_ROOT_PASSWORD_SHA2 from datanode containers (#20724)
* Remove unused GRAYLOG_DATANODE_ROOT_PASSWORD_SHA2 from datanode containers

* Remove rootPasswordSha2 from datanode container builders
2024-11-07 17:29:52 +01:00
Jenkins
77cc1541bb [graylog2-server] prepare for next development iteration 2024-11-04 15:30:14 +01:00
Jenkins
c4f044b153 [graylog2-server] prepare release 6.2.0-alpha.1 2024-11-04 15:13:29 +01:00
Jenkins
42b2a6c7f5 [graylog2-server] prepare for next development iteration 2024-11-04 14:00:59 +01:00
Jenkins
b7e0c12c30 [graylog2-server] prepare release 6.2.0-alpha.1 2024-11-04 13:44:12 +01:00
Tomas Dvorak
0d94bf11f4 datanode preflight dns check (#20492)
* datanode preflight dns check

* code cleanup

* better error message

* fix dns preflight check

* changelog

* Update data-node/src/main/java/org/graylog/datanode/bootstrap/preflight/DatanodeDnsPreflightCheck.java

Co-authored-by: Matthias Oesterheld <33032967+moesterheld@users.noreply.github.com>

---------

Co-authored-by: Matthias Oesterheld <33032967+moesterheld@users.noreply.github.com>
2024-10-28 11:36:37 +01:00
dependabot[bot]
2eca314a82 Bump org.codehaus.mojo:exec-maven-plugin from 3.4.1 to 3.5.0 (#20765)
Bumps [org.codehaus.mojo:exec-maven-plugin](https://github.com/mojohaus/exec-maven-plugin) from 3.4.1 to 3.5.0.
- [Release notes](https://github.com/mojohaus/exec-maven-plugin/releases)
- [Commits](https://github.com/mojohaus/exec-maven-plugin/compare/3.4.1...3.5.0)

---
updated-dependencies:
- dependency-name: org.codehaus.mojo:exec-maven-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Bernd Ahlers <bernd@users.noreply.github.com>
2024-10-28 08:21:14 +01:00
Othello Maurer
9e097a37e1 Bump severity of ArrayRecordComponent EP check to ERROR (#20619)
* Bump severity of ArrayRecordComponent EP check to ERROR

* Make `FileystemKeystoreInformation` a POJO

Also, discourages String conversions of passwords by removing the convenience method.

String conversion still happens all over the place and resolving that would be beyond the scope of this PR, but at least we shouldn't offer a convenience method right at the interface level.

* Convert records with mutable components to POJOs

* Add #equals and #hashCode
2024-10-16 13:29:23 +02:00
Tomas Dvorak
e562d1ad00 Explicit type for JwtSecret data, password handling (#20656)
* Explicit type for JwtSecret data, automatically extend password if too short.

* repeat given password instead of appending known character

* fixed DatanodeContainerizedBackend

* Configure JwtSecretProvider as singleton

* Fix bindings

* Fix integration test

* code cleanup

* fix jwt secret in JerseyService
2024-10-11 11:04:58 +02:00
github-actions[bot]
b4b04f2121 Bump to 6.2.0-SNAPSHOT 2024-10-07 12:47:54 +00:00
Jenkins
54e43af9cf [graylog2-server] prepare for next development iteration 2024-10-07 13:49:58 +02:00
Jenkins
b7ffdc6fac [graylog2-server] prepare release 6.1.0-rc.1 2024-10-07 13:33:24 +02:00
Tomas Dvorak
8c827ddca6 Safer in-place migration handling (#20366)
* Remove side effects, set preflight finished onexit in migration

* Preflight chck for node.lock

* added changelog

* use opensearch lock check in migration actions

* safer allDatanodesAvailable impl

* license

* use opensearch locks check during inplace migration

* adapted test

---------

Co-authored-by: Matthias Oesterheld <33032967+moesterheld@users.noreply.github.com>
2024-10-02 10:27:17 +02:00
Jenkins
f0bf7e413d [graylog2-server] prepare for next development iteration 2024-09-30 14:11:05 +02:00
Jenkins
a15060e0bb [graylog2-server] prepare release 6.1.0-beta.4 2024-09-30 13:54:15 +02:00
Matthias Oesterheld
c83e5a0d39 resolve node_name from configuration getter instead of named parameter (#20524) 2024-09-24 17:07:47 +02:00
Jenkins
d7af5a4619 [graylog2-server] prepare for next development iteration 2024-09-23 13:53:01 +02:00
Jenkins
7bd2445e6d [graylog2-server] prepare release 6.1.0-beta.3 2024-09-23 13:36:19 +02:00
Matthias Oesterheld
9c778cc1df pass in node name to correct shard relocation (#20463)
handle stop in state machine
2024-09-19 11:08:02 +02:00
Jenkins
6fbca6f4c3 [graylog2-server] prepare for next development iteration 2024-09-16 10:29:03 +02:00
Jenkins
3f1795e622 [graylog2-server] prepare release 6.1.0-beta.2 2024-09-16 10:12:21 +02:00
Kay Roepke
b501b88251 Revert version to 6.1.0-SNAPSHOT 2024-09-16 09:37:54 +02:00
Jenkins
b772a694c8 [graylog2-server] prepare release 6.1.0-beta.2 2024-09-16 08:43:44 +02:00
Jenkins
5e0d3aa98b [graylog2-server] prepare for next development iteration 2024-09-09 09:02:49 +02:00
Jenkins
2d25b0d3cc [graylog2-server] prepare release 6.1.0-beta.1 2024-09-09 08:46:08 +02:00
dependabot[bot]
b1c1c9d8eb Bump org.codehaus.mojo:buildnumber-maven-plugin from 3.2.0 to 3.2.1 (#20332)
Bumps [org.codehaus.mojo:buildnumber-maven-plugin](https://github.com/mojohaus/buildnumber-maven-plugin) from 3.2.0 to 3.2.1.
- [Release notes](https://github.com/mojohaus/buildnumber-maven-plugin/releases)
- [Commits](https://github.com/mojohaus/buildnumber-maven-plugin/compare/3.2.0...3.2.1)

---
updated-dependencies:
- dependency-name: org.codehaus.mojo:buildnumber-maven-plugin
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-05 10:29:03 +02:00
Jenkins
bdba42ac84 [graylog2-server] prepare for next development iteration 2024-08-30 12:02:49 +02:00
Jenkins
9793e053d7 [graylog2-server] prepare release 6.1.0-alpha.8 2024-08-30 11:45:56 +02:00
Jenkins
a74ff7c718 [graylog2-server] prepare for next development iteration 2024-08-16 14:34:31 +02:00
Jenkins
a259c1dfeb [graylog2-server] prepare release 6.1.0-alpha.7 2024-08-16 14:18:11 +02:00
Tomas Dvorak
ee9a81d48e Handle closed indices in remote reindex migration (#20081)
* Handle closed indices in remote reindex migration

* selectAll shouldn't select the closed indices

* added changelog

* Remove support for empty indices list in remote reindex migration (not used anywhere)

* Reopen closed indices for remote reindex migration

* updated changelog

* fix setting authorization header only for non-null parameters

* remove check for message from closed index

---------

Co-authored-by: Mohamed OULD HOCINE <106236152+gally47@users.noreply.github.com>
Co-authored-by: Mohamed Ould Hocine <mohamed.ould.hocine@graylog.com>
Co-authored-by: Matthias Oesterheld <33032967+moesterheld@users.noreply.github.com>
Co-authored-by: Matthias Oesterheld <matthias.oesterheld@graylog.com>
2024-08-16 13:27:48 +02:00
dependabot[bot]
55867c5fb7 Bump org.codehaus.mojo:exec-maven-plugin from 3.4.0 to 3.4.1 (#20165)
Bumps [org.codehaus.mojo:exec-maven-plugin](https://github.com/mojohaus/exec-maven-plugin) from 3.4.0 to 3.4.1.
- [Release notes](https://github.com/mojohaus/exec-maven-plugin/releases)
- [Commits](https://github.com/mojohaus/exec-maven-plugin/compare/3.4.0...3.4.1)

---
updated-dependencies:
- dependency-name: org.codehaus.mojo:exec-maven-plugin
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-08-13 11:13:00 +02:00
Matthias Oesterheld
27c236323a Add certificate information to support bundle (#20120)
* read and add endpoint for datanode certificates

* read and add endpoint for Graylog certificates

* doc and permission

* add error logging

* add certificates to support bundle

* cl

* change certrenewal swagger annotation

* move security config resolution to provider

* add singleton annotation

* fix wrong imports
2024-08-12 14:38:32 +02:00
Matthias Oesterheld
fbefdcc97f fix certificate renewal periodicals to parse both Duration and Period (#20056)
* fix certificate renewal periodicals to use PeriodDuration

* move threshold calculation to `RenewalPolicy`
create & use typed method for parsed lifetime

* remove threeten-extra dependency

* fix test for dates using 10% threshold
2024-08-09 15:42:08 +02:00
Matthias Oesterheld
c49f79f8d4 wrap all exceptions in response (#20139) 2024-08-09 14:14:23 +02:00
Matthias Oesterheld
c86b0d5c90 Add datanode information to support bundle (#20090)
* add additional information to DataNodeStatus

* add datanode information to support bundle

* cl
2024-08-08 11:55:00 +02:00
Jan Heise
356d2521e1 Bumping the OpenSearch Client in general and the Server in the DataNode to 2.15 (#20097)
* bumping the opensearch client and the server in the datanode to 2.15

* fixing digest

* fixed digest

* fixed last digest

* added changelog
2024-08-07 14:48:51 +02:00
dependabot[bot]
be74221482 Bump org.codehaus.mojo:exec-maven-plugin from 3.3.0 to 3.4.0 (#20101)
Bumps [org.codehaus.mojo:exec-maven-plugin](https://github.com/mojohaus/exec-maven-plugin) from 3.3.0 to 3.4.0.
- [Release notes](https://github.com/mojohaus/exec-maven-plugin/releases)
- [Commits](https://github.com/mojohaus/exec-maven-plugin/compare/3.3.0...3.4.0)

---
updated-dependencies:
- dependency-name: org.codehaus.mojo:exec-maven-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-08-07 09:42:19 +02:00