* use dependency injection for process and state machine tracer
* fix forbidden annotations
* move removal logic from tracer to process
* add state machine exception handling
* call reset via statemachine, not process
* do not start os process during configure
* temporary fix of tests
* move start/stop process triggers to state machine
* use state machine instead of process
* fix watchdog test
* move removal tracer test to process and remove tracer
* set statemachine for tracer directly
* trigger `PREPARED` instead of `STARTING` after provisioning, startup is handled by Graylog
* add restart transition for reconfiguring test data node
* add missing transition
* make `fire` with errorEvent private
* remove static process variable
* set tracer in `createNew`
* provide default implementation
* revert to default attempts_count
* rename state machine to DataNodeStateMachine to make clear it is not only handling the process anymore
* initial refactoring of packages and classes:
clearly separate processing and opensearch-related classes
* Make ManageableProcess independent from OpensearchStateMachine
* do not write/check for lock file in data dir during preflight test
* remove unnecessary directory read check, handled by validator
* add lockfile check to startup
* remove unused exception
* fix compatibility check url
* add more information and show data node list on restart step
* ignore PROCESS_STOPPED event when in WAITING_FOR_CONFIGURATION state to avoid ugly exception
* fix test and show MigrationDatanodeList
* added new states
* fix error message
* adding new state to test
* removed doubled line
* start if insecure has been chosen
* adding jump
* always start if everything's confgured
* fixing test
* publish data node status
* add paginated datanode service
* changelog
* adjust changelog
* adjust NodeImplTest
* refactor initial separation of server node and data node
* fix backward compatibility
* refactor separation of server node and data node
* add timezone
* add new service tests
* add entity tests
* inject DataNodeDto NodeService
* service, rest & permissions for data node management
* add tests
* add reset data node event
* add states and transitions for node removal
add node removal functionality
* changelog
* add datanode permissions to permission list
* add audit event types to list
* add custom thread factory
* add node removal integration test
refactor rest calls in ITs
* wait for primary shard node availability
* refactor duplicate code
* simplify node retrieval
* add `@OnlyInSecuredNode` annotation and request filter
move delete endpoint to management controller and secure it
adjust IT
change javadoc
* stabilize tests
* using the hostname as the default node-name, if none was specified explicitly
* using the hostname as the default node-name, if none was specified explicitly
* using the hostname as the default node-name, if none was specified explicitly
* nodeName is either set explicitly in the conf file or will be the result of the hostname setting otherwise
* small refactoring
* fixing discovery process for new nodes
* only write discovery seeds on start of the opensearch process depending on the list of active nodes.
* fix dependent init
* fix method call to not depend on pre-initialized members
* fixing parameter map creation
* fixing test and using correct parameter
* fixing test, fixing default roles
* fixing tests
* jwt implementation, adding preflight + versionprobe
* adding JWT auth to anomaly detection client, some minor refactorings
* added missing license
* added provider, refactorings
* fixing small bugs
* set smaller default expiration
* using the correct OpenSearch distro for the erchitecture
* added config options
* small refactorings, fixing tests, adding dependencies for running tests from inside IntelliJ
* fixing import to correct Duration class
* removing unnecessary debug output
* fixing tests
* fixing tests
* adding JWT to Cluster test
* fixing path
* fixing path problems
* fix architecture logic for DatanodeContainerizedBackend
* fixing path problems
* fixing JWT settings in tests
* fixing JWT settings in tests
* fixing cluster test
* fixing cluster test
* fixing cluster test
* adding missing jwt token
* small refactoring to force thread-safety on creation
* the datanode uses mongodb - so the closing sequence needs to be adjusted
* moving config changes into config file, keep only change that has to be done by code
* adding breaking change info for JWT and clock synchronization.
* renaming config settings
* renaming config settings
* removed debug settings that are no longer in use
* renamed provider/annotation to be more precise
* lowering log level
* smaller refactorings/renamings
* keep config change in code until PR is merged, remove code in a follow-up PR
* moved changelog file to correct folder
* clock synchronsation
Co-authored-by: Dennis Oelkers <dennis@graylog.com>
* added jwt auth into changes after merge
* upped the timeout
* log curl connection information in DatanodeOpensearchWait
* replacing with jwt auth
* changing wait strategy
* fixing imports
* remove uname/pwd from opensearch rest base url
* propagate jwt token provider to http rest client callback, obtain fresh token every time
* inject IndexerJwtAuthTokenProvider in DatanodeConfigurationProvider
* remove rest uname/pwd from datanode configuration
---------
Co-authored-by: Tomas Dvorak <tomas.dvorak@graylog.com>
Co-authored-by: Dennis Oelkers <dennis@graylog.com>
* Insecure startup of OS in datanode only if explicitly configured
* Data node can be started without starting OS, just to be discovered by pre-flight mode
* Start datanode jersey only after ssl configuration resolved, listen to change events
* Adapted DatanodeSecuritySetupIT to work with https version of the datanode interface
* simplify hostnames in DatanodeSecuritySetupIT
* code cleanup
* added changelog
* code cleanup
* disable security plugin and perf analyzer in matrix tests opensearch
* Fixed datanode jersey service
* Fixed opensearch instance env
* Opensearch security setup refactored
* code cleanup
* removed commented out code from datanodes JerseyService
* add fqdn to the http cert in the DatanodeSecuritySetupIT
* configure truststore only if the security is enabled
* code cleanup
* reintroduced root certificate finder
* adjusted class and methods visibility of RootCertificateFinder
* first version adding gl CA
* adding custom trutstore
* fixed SSLContext
* fixed SSLContext for connectivity check
* fix comments
* fixed imports
* reverting altnames tmp change
* reverting altnames tmp change
* fixed preflightconfig, fixed hostname-verifier
* added credentials to target string, set hostname
* added credentials from target string
* only add TrustManager if https is requested
* fixing tests
* add another alt name
* change creation of restBaseUrl
* DatanodeSecuritySetupIT adaptation for stable hostname of the datanode
* trying to fix naming/binding for docker
* adding another alt name
* added s to http
* changing exception
* more debug output
* change test
* change test
* adding debug
* switch back to http
* more debugging
* more debugging
* added exception to retryer
* fixed boolean eval
* going to the next step on FAILED, too
* fixing if clause
* cleanup
* cleanup
* cleanup
* cleanup
* cleanup
* cleanup
* cleanup
* cleanup
* removed wrong credentials provider
* added name verifying
* fix alt name handling
* removed unneccessary hostname verifier
* reverted DefaultX509TrustManager
* removed link to DefaultX509TrustManager because it's used in different contexts, singleton for SSL related stuff so we don't instantiate it all the time
* full rollback DefaultX509TrustManager
* reverted change on logging
* revert VersionProbe, unnecessary changes
* DatanodeSecuritySetupIT fixed, separated bind address and hostname
* changed to lambda
* changed to lambda
* small refactorings in csrsigner
* added test for CustomCAX509TrustManager
* added binding for CaService into Preflight
* fixing bindings
* fixing bindings
---------
Co-authored-by: Tomas Dvorak <tomas.dvorak@graylog.com>
* Synchronize opensearch config to a user-defined directory (fixes#15223)
* fixed check naming
* fixed opensearch configuration location
* simplified Environment constructor
* point gc logs of datanodes opensearch to /tmp for now
* full dir sync for opensearch config
* Fixed full dir sync when removing non-empty dir
* simpified recursive directory deletion
