Commit Graph

13 Commits

Author SHA1 Message Date
Jan Heise
cc52dcb5b0 DataNode: using JWT for authentication/authorisation (#16334)
* jwt implementation, adding preflight + versionprobe

* adding JWT auth to anomaly detection client, some minor refactorings

* added missing license

* added provider, refactorings

* fixing small bugs

* set smaller default expiration

* using the correct OpenSearch distro for the erchitecture

* added config options

* small refactorings, fixing tests, adding dependencies for running tests from inside IntelliJ

* fixing import to correct Duration class

* removing unnecessary debug output

* fixing tests

* fixing tests

* adding JWT to Cluster test

* fixing path

* fixing path problems

* fix architecture logic for DatanodeContainerizedBackend

* fixing path problems

* fixing JWT settings in tests

* fixing JWT settings in tests

* fixing cluster test

* fixing cluster test

* fixing cluster test

* adding missing jwt token

* small refactoring to force thread-safety on creation

* the datanode uses mongodb - so the closing sequence needs to be adjusted

* moving config changes into config file, keep only change that has to be done by code

* adding breaking change info for JWT and clock synchronization.

* renaming config settings

* renaming config settings

* removed debug settings that are no longer in use

* renamed provider/annotation to be more precise

* lowering log level

* smaller refactorings/renamings

* keep config change in code until PR is merged, remove code in a follow-up PR

* moved changelog file to correct folder

* clock synchronsation

Co-authored-by: Dennis Oelkers <dennis@graylog.com>

* added jwt auth into changes after merge

* upped the timeout

* log curl connection information in DatanodeOpensearchWait

* replacing with jwt auth

* changing wait strategy

* fixing imports

* remove uname/pwd from opensearch rest base url

* propagate jwt token provider to http rest client callback, obtain fresh token every time

* inject IndexerJwtAuthTokenProvider in DatanodeConfigurationProvider

* remove rest uname/pwd from datanode configuration

---------

Co-authored-by: Tomas Dvorak <tomas.dvorak@graylog.com>
Co-authored-by: Dennis Oelkers <dennis@graylog.com>
2023-09-12 11:48:44 +02:00
Dennis Oelkers
b4d1b06dbe Disabling auto-creation for data node. (#16413) 2023-09-11 12:37:47 +02:00
luk-kaminski
e7025a9da8 Insecure startup of OS in datanode only if explicitly configured (#16108)
* Insecure startup of OS in datanode only if explicitly configured

* Data node can be started without starting OS, just to be discovered by pre-flight mode
2023-08-07 17:21:08 +02:00
Tomas Dvorak
e39a53ed2d Refactoring/opensearch security setup (#16087)
* Start datanode jersey only after ssl configuration resolved, listen to change events

* Adapted DatanodeSecuritySetupIT to work with https version of the datanode interface

* simplify hostnames in DatanodeSecuritySetupIT

* code cleanup

* added changelog

* code cleanup

* disable security plugin and perf analyzer in matrix tests opensearch

* Fixed datanode jersey service

* Fixed opensearch instance env

* Opensearch security setup refactored

* code cleanup

* removed commented out code from datanodes JerseyService

* add fqdn to the http cert in the DatanodeSecuritySetupIT

* configure truststore only if the security is enabled

* code cleanup

* reintroduced root certificate finder

* adjusted class and methods visibility of RootCertificateFinder
2023-08-02 14:33:50 +02:00
Jan Heise
34d5b47d2a Include Graylog CA into places where we connect to the DataNode (#15925)
* first version adding gl CA

* adding custom trutstore

* fixed SSLContext

* fixed SSLContext for connectivity check

* fix comments

* fixed imports

* reverting altnames tmp change

* reverting altnames tmp change

* fixed preflightconfig, fixed hostname-verifier

* added credentials to target string, set hostname

* added credentials from target string

* only add TrustManager if https is requested

* fixing tests

* add another alt name

* change creation of restBaseUrl

* DatanodeSecuritySetupIT adaptation for stable hostname of the datanode

* trying to fix naming/binding for docker

* adding another alt name

* added s to http

* changing exception

* more debug output

* change test

* change test

* adding debug

* switch back to http

* more debugging

* more debugging

* added exception to retryer

* fixed boolean eval

* going to the next step on FAILED, too

* fixing if clause

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* removed wrong credentials provider

* added name verifying

* fix alt name handling

* removed unneccessary hostname verifier

* reverted DefaultX509TrustManager

* removed link to DefaultX509TrustManager because it's used in different contexts, singleton for SSL related stuff so we don't instantiate it all the time

* full rollback DefaultX509TrustManager

* reverted change on logging

* revert VersionProbe, unnecessary changes

* DatanodeSecuritySetupIT fixed, separated bind address and hostname

* changed to lambda

* changed to lambda

* small refactorings in csrsigner

* added test for CustomCAX509TrustManager

* added binding for CaService into Preflight

* fixing bindings

* fixing bindings

---------

Co-authored-by: Tomas Dvorak <tomas.dvorak@graylog.com>
2023-07-21 12:59:55 +02:00
Jan Heise
cfb3386908 DataNode: CA upload/generation or load existing configured CA during startup (#15546)
Co-authored-by: Lukasz Kaminski <lukasz.kaminski@graylog.com>
2023-06-20 15:33:07 +02:00
Tomas Dvorak
9105655645 Code cleanup around opensearch process info (#15709)
* Code cleanup around opensearch process info

* Code cleanup

* fixed naming and test

* fixed test

* fixed test

* code cleanup

* code cleanup
2023-06-19 06:36:32 +02:00
Tomas Dvorak
bc56d397c3 Datanode config separation (#15701)
* Different variants of security configuration (from file, from mongo, insecure).
Encryption/decryption pattern used in practice.

* Reverting temporary changes

* More precise certificate presence check

* Added unit tests for testHasCert() method

* Datanode and opensearch configuration refactored

* code cleanup

* code cleanup

* code cleanup

* Node preflight config service refactored

* restbaseurl moved to OpensearchConfiguration

---------

Co-authored-by: Lukasz Kaminski <lukasz.kaminski@graylog.com>
2023-06-06 14:44:06 +02:00
Tomas Dvorak
5651f90aa9 Process watchdog reimplemented to follow transitions of the process state machine (#15676)
* Process watchdog reimplemented to follow transitions of the process state machine

* code cleanup

* code cleanup
2023-06-02 10:52:24 +02:00
Tomas Dvorak
a487ca2f86 Delay opensearch start till configuration is available (#15615)
* Delay opensearch start till configuration is available

* code cleanup
2023-05-30 10:06:47 +02:00
Tomas Dvorak
2fa7b7b9aa Synchronize opensearch config to a user-defined directory (#15307)
* Synchronize opensearch config to a user-defined directory (fixes #15223)

* fixed check naming

* fixed opensearch configuration location

* simplified Environment constructor

* point gc logs of datanodes opensearch to /tmp for now

* full dir sync for opensearch config

* Fixed full dir sync when removing non-empty dir

* simpified recursive directory deletion
2023-05-02 15:01:12 +02:00
Tomas Dvorak
eba2d54498 Extended preflight check, added logging and adapted state machine transitions (#15149)
* Extended preflight check, added logging and adapted state machine transitions

* fixed opensearch preflight check

* fixed opensearch preflight check, add tests
2023-04-05 18:14:15 +02:00
Tomas Dvorak
21e55cbcb6 Initial Data Node version PR (#14417)
Co-authored-by: Jan Heise <jan.heise@graylog.com>
2023-02-27 13:11:24 +01:00