mirror of
https://github.com/Graylog2/graylog2-server.git
synced 2026-03-13 09:32:21 +08:00
e0cf91db16
* first batch of changes to remove the matrix test engine we are migrating matrix tests into github actions and controlling versions externally this change starts moving the relevant container management code into the extension this is work in progress, most tests will fail and fixtures aren't imported yet. more sweeping changes need to be done to avoid constructor injection due to lifecycle calling order issues * implement class-lifecycle container handling, and adjust inputcreationIT * have junit close the backends when their store goes out of context, convert more tests * we no longer need a vm shutdown hook, junit's extension takes care of the closing * allow setting the mongodb and search server versions using system properties * add support for requiring data node in tests, if we test functionality only present for that backend * fix wrong check for data node and skipping tests, i've borked it * convert constructor injection to lifecycle method * convert constructor injection to lifecycle method * convert constructor injection to lifecycle method * convert constructor injection to lifecycle method * convert constructor injection to lifecycle method * convert constructor injection to lifecycle method * convert constructor injection to lifecycle method * convert constructor injection to lifecycle method * convert constructor injection to lifecycle method * convert constructor injection to lifecycle method * convert constructor injection to lifecycle method and fix warnigs * convert constructor injection to lifecycle method * convert constructor injection to lifecycle method * convert constructor injection to lifecycle method * convert constructor injection to lifecycle method * convert constructor injection to lifecycle method * convert constructor injection to lifecycle method and fix warnings * load mongodb fixtures and pass through feature flags to allow running plugin tests * Speed up MavenPackager by by 7 minutes when building JARs We don't need to create the assemblies. * Use a single container network for all tests We cache MongoDB and OpenSearch services so we can't use a new network on every test. Otherwise, the server would be in a different network when using cached services. * More verbose logging for class-lifecycle server backends * Exit server backend startup early when seeing an unknown host error * Log MongoDB version in ContainerizedGraylogBackend * Return the version string instead of the enum * Fix warning in ContainerizedGraylogBackend * Don't create a new MongoDB instance for every test * don't use constructor argument for test base class, it must be static through the lifecycle method * convert remaining tests to use lifecycle injection * Introduce #cachedInstance to SearchServerBuilder Allows the creation of uncached search instances. * Add MongoDBInstance#instanceId * Add SearchServerInstance#instancdId * Stop server early if MongoDB is not reachable * Fix caching issues in services provider Disable multi-layered caches and only cache at the service layer. * We need the server assembly * Remove connection failure detector again Exiting early is problematic because a retry might fix the problem. Add check for NoClassDefFoundError * make error message nicer * tests require data node, so skip them on other search versions * test only runs on data node * remove custom junit engine rename annotation to FullBackendTest remove deprecated annotation options always start mail and webhook services, they are tiny and fast, simplifies this further (and cuts down on container starts) some warning cleanup * re-enable maven packager, committed this by mistake * Only allow FullBackendTest annotation in full-backend-tests * Let WebInterfaceAssetsResourceWithPrefixIT run in CLASS lifecycle The test needs custom configuration. * Don't allow VM lifecycle with customized config or feature flags * test with custom config must run in class lifecycle * Remove commented cache key parts * Shutdown CLASS services with non-empty config or feature flags We don't need services for custom configurations to stay around. * Improve Guice error detection on server startup * Replace MongodbServer with MongoDBVersion * Remove unused MongoDBVersion#PATTERN * Remove unused ContainerVersions * Extract search server and MongoDB version provider * Remove unused imports * Remove SearchServerInstance#searchServer method Also convert SearchServer from an enum to a POJO with some constants. * Move SearchServer to org.graylog.testing.storage * Change log level in GraylogBackendExtension * Add EnabledIfSearchServer annotation and condition * Switch DatanodeOpensearchProxyIT to EnabledIfSearchServer * Switch more tests to EnabledIfSearchServer annotation * Remove GraylogBackendConfiguration#onlyOnDataNode * Fix EnabledIfSearchServerCondition class name * Remove unused DatanodeDevInstance and its builder and provider * Ensure Maven packaging before starting backend services Also move check for GRAYLOG_IT_SKIP_PACKAGING environment variable into the MavenPackager to avoid duplication. * Set Data Node version to Version.CURRENT_CLASSPATH * Improve log message in ContainerizedGraylogBackend * Fix EnabledIfSearchServerCondition logic Don't check the presence of a version when distribution is DATANODE. * Add Javadoc to EnabledIfSearchServer * Fix DatanodeInstance to build the local image * Don't set GRAYLOG_ELASTICSEARCH_VERSION env when running Data Node Fixes a startup error where the search version validator failed. * Pass missing env to DatanodeInstance * Remove backend services from cache when they get closed * Adjust name for local Data Node image to be consistent with the server * Adjust log message in MavenPackager when skipping * Handle Exception and AssertError in GraylogRestApi#waitFor Otherwise, we would exit the wait loop when the predicate throws an exception. (e.g., when we get a 404 instead of a 200) * Add log().ifError() in WebInterfaceAssetsResourceBase * take the cluster id from the imported licenses instead of manually setting it through a fixture * Remove unused GraylogBackendExtension#resolveFixtures method * Fix Data Node SearchVersionRange checks This was hardcoded to 5.2.0 because that's the version we used in our tests. (regardless of which actual version was running) We are now building the version from the Version.CURRENT_CLASSPATH. * Relax Data Node search server check * Pass MavenProjectDirProvider into services provider * Add the include-frontend flag to the services cache key * Revert "Add the include-frontend flag to the services cache key" This reverts commitc25b1a5b3a. * Revert "Pass MavenProjectDirProvider into services provider" This reverts commit37001edcc9. * Remove MavenProjectDirProviderWithFrontend * Rename ContainerMatrixElasticsearchBaseTest to SearchServerBaseTest * Remove unused NodeContainerFactory#containerFileExists * Don't memoize customization config in customization config provider It makes testing really hard and requires a server restart to change the values. * Fix signal handling in webhook.js test server Stopping webhook server container was blocking for a long time. * Add EnabledIfMongoDB annotation * Move MavenPackager call up into ContainerizedGraylogBackend That way we will build the JAR files at the very beginning and with the correct MavenProjectDirProvider. (at least for the first test that runs) * Adjust MavenPackager logs * Make completebackend.api.Views#getAll more robust * Silence MavenPackager * Use defaultIfBlank instead of requireNonNullElse * Also use defaultIfBlank for MongoDB version * Remove unused service image properties We can add them back if we should implement image selection. * Cleanup GraylogBackendExtension * Rename ConfigurationParameter to Env in GraylogBackendConfiguration * Prevent importLicenses=false in VM lifecycle * Add Javadoc to GraylogBackendConfiguration * Move GraylogBackendConfiguration and FullBackendTest annotations Those were the last files in the containermatrix package. * More Javadoc * Remove services from cache before stopping them * Log if validation fails * Introduce Server and Data Node product configuration This replaces the low-level provider configurations for Maven and plugin JAR files. * Don't implement AutoClosable for ContainerizedGraylogBackendServicesProvider We never close it anywhere. Convert the instance method to a static cleanup method. * Remove mavenProjectDirProvider, pluginJarsProvider, datanodePluginJarsProvider * Fix duplicate BeforeAll methods * Fix logging in completebackend.apis.Views * Re-enable DbEntitiesCatalogProviderTest * Use SearchServer.DEFAULT_VERSION for the default distribution * Remove unused RunningGraylogBackend * Check composable index templates feature flag in AdaptersES7 Fixes IndicesIT with Elasticsearch 7 and OpenSearch 1.x because they've always been using composable index templates. * Remove unused RunningElasticsearchInstanceES7 class * Add sleep to ensure updatedAt differs when reacquiring lock * Make EnabledIfSearchServer annotation repeatable * Improve wording in EnabledIfSearchServerCondition * Bump default heap in SearchServerBuilder to 1 GB * Don't set "elasticsearch_version" and let the server detect the version * Adjust IndexRangesCleanupIT to handle AssertionError in retry loop * Remove unneccessary warning supression in DatanodeProvisioningIT * Remove unused testClass param in GraylogBackendExtension#createBackend --------- Co-authored-by: Bernd Ahlers <bernd@graylog.com> Co-authored-by: Bernd Ahlers <bernd@users.noreply.github.com> Co-authored-by: Othello Maurer <othello@graylog.com>
First version, additional info to follow.