* Adding new additional dependencies.
* Setting up injection infrastructure for new client in parallel.
* Adding license headers.
* Including new deps in test jar.
* Removing jakarta deps from assembly.
* Building separate jars for integration tests & distribution.
* Fix class loader issue
Exclude "module-info.class" files from the graylog-storage-opensearch
JAR file to avoid class loader issues.
Revert previous assembly and dependency changes that worked around the
issue in tests.
* Adjusting to Jakarta and recent data node changes.
* fixing wrong builder usage
---------
Co-authored-by: Bernd Ahlers <bernd@graylog.com>
Co-authored-by: Jan Heise <jan.heise@graylog.com>
* Create Data Stream template, prepare ism policy
* add ism
* create ism and collect jvm heap as first metric
* change lookup and term query to use data streams
* start moving data stream functionality to graylog
* move data stream management into graylog
* code cleanup, add tests
* better validation, clean up after tests
* additional clean up after IT
* don't remove `index-management` plugin from datanode
* inject ObjectMapper
* change number of replicas from default 1 to 0 for test
* set `number_of_replicas` to 0 for .opendistro-ism-config in test
* replace javax with jakarta classes
* fix integration test
* remove data streams on test cleanup
* set high priority for data stream template
* Update to OpenSearch 2.11.0
* Update to OpenSearch 2.11.0
* reverting test back
* reverting test back
* better handling of waiting for datanode startups in integration tests
---------
Co-authored-by: Tomas Dvorak <tomas.dvorak@graylog.com>
* ensure password_secret meets length requirement
* update CL
* update test
* adjusted wording so it references the data node
---------
Co-authored-by: Jan Heise <jan.heise@graylog.com>
* Parser for opensearch/elasticsearch data directories
* add opensearch 1.3 and elasticsearch 6.8.0 to indexer parser
* fixed paths
* code cleanup
* remove explicit dependency version of jackson smile
* Preflight check for index dir compatibility
* indexer dir parser fixed for empty dirs
* added changelog
* rest controller for indices directory parser
* require auth for indices compatibility rest api
* catch incompatibility exceptions of indices dir parsing
* code cleanup
* better dir detection handling
---------
Co-authored-by: Matthias Oesterheld <33032967+moesterheld@users.noreply.github.com>
- Read the "max_map_count" from the /proc system instead of calling the
/sbin/sysctl command
- Only run check on Linux operating systems (avoids warning on FreeBSD)
- Throw "PreflightCheckException" instead of "RuntimeException"
- Don't use "System.out.println()" for logging
- Enhance changelog entry to help users to understand the fix
Refs https://github.com/Graylog2/graylog2-server/pull/17436
* publish data node status
* add paginated datanode service
* changelog
* adjust changelog
* adjust NodeImplTest
* refactor initial separation of server node and data node
* fix backward compatibility
* refactor separation of server node and data node
* add timezone
* add new service tests
* add entity tests
* inject DataNodeDto NodeService
* service, rest & permissions for data node management
* add tests
* add reset data node event
* add states and transitions for node removal
add node removal functionality
* changelog
* add datanode permissions to permission list
* add audit event types to list
* add custom thread factory
* add node removal integration test
refactor rest calls in ITs
* wait for primary shard node availability
* refactor duplicate code
* simplify node retrieval
* add `@OnlyInSecuredNode` annotation and request filter
move delete endpoint to management controller and secure it
adjust IT
change javadoc
* stabilize tests
* using the hostname as the default node-name, if none was specified explicitly
* using the hostname as the default node-name, if none was specified explicitly
* using the hostname as the default node-name, if none was specified explicitly
* nodeName is either set explicitly in the conf file or will be the result of the hostname setting otherwise
* small refactoring
* Fix file permissions for key material #17009
* added changelog
* correct file permissions for private key used during preflight CSR
* better http and transport keystore files generation
* removed changelog
---------
Co-authored-by: Jan Heise <jan.heise@graylog.com>
* Include local reverse lookups for loopback addresses in CSR.
* Add reverse lookups for local loopback-addresses to CSR.
* Suppressing forbidden method invocation warning.
* writing opensearch.yml instead of env vars
* appending to an existing original config file to be able to check in default config options in git
* remove original config file, always create/write config
* removed unnecessary imports
* refactor datanode directories checks, use configuration validators
* comments, error messages
* fix datanode config dir in docker
* force image pull strategy for datanode dev container
* remove image pull strategy
* More detailed DirectoryWritableValidator for datanode
* remove commons dependency from UploadedCertFilesSecureConfiguration
* Fixing compilation of single module.
* Adding test case to ensure that unwanted plugins are not included/loaded.
* Adding license header.
* Doing simple web frontend change to trigger reviewbot.
* Fixing linter hint.
* Use random http auth password during preflight web setup
* set initial preflight password once and persist it in mongodb
* refactor structure of the preflight config service, add migration
* add license
* code cleanup
* manually trigger password creation migration in preflight
* Fixing up import.
* fixed preflight auth filter init
* migrations for prefligh boot phase
* code cleanup
* add migration to the PreflightWebModule
* fix injector reference in preflight migrations
* Include logo, make sure that all relevant information is shown at last.
* Consolidating banner logging into single call.
---------
Co-authored-by: Dennis Oelkers <dennis@graylog.com>
* fixing discovery process for new nodes
* only write discovery seeds on start of the opensearch process depending on the list of active nodes.
* fix dependent init
* fix method call to not depend on pre-initialized members
* fixing parameter map creation
* fixing test and using correct parameter
* fixing test, fixing default roles
* fixing tests
