* Initial working state of OpenAPI spec generation
Generates an OpenAPI spec but still has lots of issues
* Fix wrong import
* Add NoAuditEvent annotation to test resource methods
* Add OpenAPI browser frontend (POC)
Add basic OpenAPI browser UI using Redoc loaded from CDN. This is a proof
of concept implementation not yet ready for production use.
- Loads Redoc from CDN to avoid webpack bundling complexity
- Displays OpenAPI spec from /api/openapi.yaml endpoint
- Accessible via Help menu → "OpenAPI browser"
- Route: /openapi-browser
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Make both json & yaml formats work
* Implement plugin path prefixing
+ configure the openAPI context directly to be able to
use dependency injection
* Remove logging
* Migrate REST resources from Swagger to OpenAPI 3.x annotations
Migrated ~150 REST resource files from old Swagger annotations
(io.swagger.annotations.*) to OpenAPI 3.x annotations
(io.swagger.v3.oas.annotations.*).
Migration performed using OpenRewrite and manual fixes:
- @Api → @Tag
- @ApiOperation → @Operation
- @ApiParam → @Parameter
- @ApiResponse code → responseCode
- @ApiResponse message → description
- Removed invalid attributes (allowableValues, defaultValue, tags)
- Fixed nickname → operationId
- Removed response= attribute (needs manual restoration)
Note: Some information was intentionally lost and needs restoration:
- Response type schemas (33 instances)
- Parameter allowableValues constraints (58 instances)
- CLOUD_VISIBLE tags (82 instances)
🤖 Generated with Claude Code (https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Restore response type information to OpenAPI annotations
Adds @ApiResponses with @Content and @Schema to restore the 33 instances
of response type information that were lost during the OpenAPI 3.x migration.
- Automated script restored 20 instances
- Manual additions for 8 instances with multi-line annotations
- 5 instances already fixed in previous commit
- All response codes verified against actual implementation
- Total: 33 response types restored with accurate HTTP status codes
Co-Authored-By: Claude <noreply@anthropic.com>
* Restore parameter allowableValues constraints for OpenAPI 3.x migration
Restore allowableValues constraints that were lost during OpenAPI 3.x
migration by converting from @ApiParam(allowableValues) to
@Parameter with @Schema(allowableValues).
This restores API documentation for parameter value constraints across
22 REST resource files, covering ~58 parameter instances.
Changes:
- Migrated allowableValues from Swagger 2.x @ApiParam to OpenAPI 3.x
@Parameter with @Schema(allowableValues = {...})
- Added missing Schema imports where needed
- Fixed several bugs and inconsistencies from master branch
Summary of Semantic Differences (11 files):
Bug Fixes:
- AuthzRolesResource.getUsersForRole(): Fixed incorrect field list that
had user fields (username,full_name,email) instead of role fields
(id,name,description)
- DatanodeResource: Fixed completely wrong field list (title,description,
type) to actual DataNodeDto fields (hostname,data_node_status,
transport_address,cert_valid_until,datanode_version)
- IndexSetTemplateResource: Fixed "name" to "title" (correct field name)
Added Missing Valid Fields:
- DashboardsResource, SavedSearchesResource, ViewsResource: Added
"last_updated_at" which exists in SEARCH_FIELD_MAPPING
- ViewsResource: Added "owner", "description", "summary" (was very
incomplete in master)
- AuthzRolesResource (getList, getListForUser): Added "id" field
- GrokResource, IndexFieldTypeProfileResource: Added "id" and other
valid fields from SEARCH_FIELD_MAPPING
Removed Invalid Fields:
- StreamResource: Removed "updated_at", "status" (not in
SEARCH_FIELD_MAPPING, didn't work)
- GrokResource: Removed "pattern" (not in search mapping)
- IndexSetsMappingResource: Reduced to "field_name" only (other fields
not implemented)
Fixed Constraint Type:
- ClusterLoggersResource: Removed inappropriate allowableValues range
constraint for numeric limit parameter (already has @Min validation)
Files with Identical Field Lists (13 files):
EventDefinitionsResource, EventNotificationsResource, PipelineResource,
RuleResource, CollectorResource, ConfigurationResource, SidecarResource,
ScriptingApiResource, AuthServiceBackendsResource, UsersResource,
TokenUsageResource, LookupTableResource - all have identical semantics,
just migrated annotation syntax.
Rationale:
All semantic differences were introduced to correct API documentation to
match actual backend implementation. Changes ensure that:
1. All advertised sort fields actually work
2. All functional sort fields are advertised
3. API spec accurately reflects backend capabilities
4. Bugs in master branch are fixed
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Add replacement for CLOUD_VISIBLE tag
* Add @PublicCloudAPI annotation to cloud-visible REST resources
Add @PublicCloudAPI annotation to 82 REST resources that were previously
marked with tags = {CLOUD_VISIBLE} in Swagger 2.x @Api annotations.
This annotation is used by CustomOpenAPIScanner to filter resources when
IS_CLOUD=true, ensuring only cloud-appropriate APIs are exposed in the
OpenAPI specification for cloud deployments.
Resources annotated:
- Events (4 resources): EventDefinitionsResource, EventNotificationsResource, etc.
- Pipeline Processor (5 resources): PipelineResource, RuleResource, SimulatorResource, etc.
- Sidecar (6 resources): SidecarResource, CollectorResource, ConfigurationResource, etc.
- Views/Search (15 resources): DashboardsResource, SavedSearchesResource, ViewsResource, etc.
- Security (6 resources): AuthzRolesResource, CAResource, ClientCertResource, etc.
- System (30+ resources): StreamResource, InputsResource, NotificationsResource, etc.
- Other resources (15+): MessageResource, EntitySuggestionResource, ContentPackResource, etc.
The @PublicCloudAPI annotation:
- Marks resources as available in public cloud deployments
- Does not appear in the generated OpenAPI specification
- Filters resources during OpenAPI generation when isCloud=true
- Replaces the old tags = {CLOUD_VISIBLE} approach from Swagger 2.x
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Migrate McpRestResource from Swagger 2.x to OpenAPI 3.x
Updates the MCP REST resource to use OpenAPI 3.x annotations, completing the migration of all REST resources in the project.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* WIP: add TODOS and subtype resolver prototype
* Add OpenAPI browser to help menu after merge conflict resolution
The OpenAPI browser menu item was lost during the merge when the help menu
was refactored from hardcoded items to a plugin-based registration system.
This restores the menu item in the new bindings.ts format.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Make legacy generator work with new annotations
* Remove unused CLOUD_VISIBLE flag
* Temporarily make test pass
* Use @RequestBody annotation instead of @Parameter
* Synchronize our not thread-safe custom reader
* Add discriminator mapping based on jackson annotations
* WIP command to generate a spec without a running server
* Add output file argument
* Don't hardcode any config parameters
* Add spec generation to maven build
* Move spec generation into profile
* Add generated spec to resources
* Attempt to stabilize spec format
* Adjust test to openapi 3.1 schema
* Resolve registered subtypes
* bind object mapper to provider
* don't skip empty subtype names
* update spec
* Revert extraction of ObjectMapperConfiguration
With the latest changes, it didn't have an effect
anymore.
* Expose OpenAPIContext to not hide the caching behavior
* Apply sorting for both web and command context
* use fully qualified type names in schema
* Fix broken discriminator mapping
* Add custom handling for ImmutableMap
* Split event expression types
Swagger can't deal with the generic type hierarchy.
It would generate faulty allOf's with refs to both
Expression<Boolean> and Expression<Number> in one
schema, e.g. for
`org.graylog.events.conditions.Expr.And`.
* Prevent duplicate operation IDs
* Skip NonApiResource annotated resources
* Add validation to spec command
* Replace terms "spec"/"specification" with "description"
* Update openapi description
* Switch to swagger-ui-react
* Serve description at openapi.yaml / openapi.json again
* Add missing license headers
* Add security schemes but hide them in swagger ui
* Update openapi description
* configure swagger ui
* add changelog
* qualify url to openapi.yaml
* Remove old api browser
* Rudimentary styling for swagger ui
* Fix broken handling of duplicate paths
* update openapi description
* fix changelog
* Remove swagger CSP group & deprecate DocumentationResource
* Update graylog2-server/src/main/java/org/graylog/api/GenerateOpenApiDescriptionCommand.java
Co-authored-by: Kay Roepke <kroepke@googlemail.com>
* create parent path if non-existent
* Polish print statements
* Remove term "Graylog"
* adjust RequestBody annotations to include schema ref in each case
* Update graylog2-server/src/main/java/org/graylog/api/GenerateOpenApiDescriptionCommand.java
Co-authored-by: Kay Roepke <kroepke@googlemail.com>
* Add annotations for success responses with declared response types
* treat Immutable(List|Set) as the java.util equivalents
* semver Version is serialized as a String in our api
* extract helper method and add overrides for joda-time and threeten classes
* HostAndPort is a string in openapi
* for ImmutableMap treat keys with GRN or joda-time DateTime types as strings for schema generation
* Add more missing response schemas
* Remove swagger annotations from datanode resource
* Add forbidden-apis rule for io.swagger.annotations.*
* Get rid of remaining ImmutableMap instances
* Remove openapi.yaml from VCS
* Execute OpenAPI desc generation in separate JVM
to avoid classloader issues due to bouncycastle
registration during command startup
* Add handling for java.time.Duration and simplify scalar type mapping
* More custom handling for various types
* `yarn format`
* Add special handling for FormDataContentDisposition in old swagger specs
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Kay Roepke <kroepke@googlemail.com>
We used several hand-rolled percentage computations in our code base.
This change refactors all of them to use the tool method.
---------
Co-authored-by: Othello Maurer <othello@graylog.com>
* removal of classes that went to Jadconfig
adding new steps to the server package
* adding missing license
* adding the config descriptions from server.conf to the annotations
* adding the config descriptions from server.conf to the annotations
* Update feature-flag.config
* jadconfig 1.0.0
* setting a valid default for the node id file in tests
* fixing dependency
* fix validation + fix test
* fixing test
* fixing test
* fix directories
* fix test
* fix test
* fix test
* fix test
* adding missing doc annotations
* merge master & fix missing doc annotation
* Adapt section headings for documentation
* Removed documentation test (will be replaced by annotation processor)
---------
Co-authored-by: Jan Heise <jan.heise@graylog.com>
* removal of classes that went to Jadconfig
adding new steps to the server package
* adding missing license
* adding the config descriptions from server.conf to the annotations
* adding the config descriptions from server.conf to the annotations
* Update feature-flag.config
* jadconfig 1.0.0
* setting a valid default for the node id file in tests
* fixing dependency
* fix validation + fix test
* fixing test
* fixing test
* fix directories
* fix test
* fix test
* fix test
* fix test
* adding missing doc annotations
* merge master & fix missing doc annotation
* Adapt section headings for documentation
---------
Co-authored-by: Tomas Dvorak <tomas.dvorak@graylog.com>
* add openrewrite infrastructure and necessary junit jupiter api deps
* update openrewrite docs and remove configLocation from plugin config, so we can override it
* initial openrewrite migration to junit5, work in progress
* replace rule with extension, use mongocollections instead of the instance
* fix parameterized test
* replace jukito with guice-extension for junit5 compatibility
* yet more mongocollections injection
* manually create MongoCollections where we need a custom objectmapper for subtypes
* fix license header
* replace junit4 conditional runner with enabledif static methods, only used in a single test anyway
* fix resource name collision in class loader test
* how did this ever compile?
* use explicit temp dir in tests, otherwise we'd need to create the directories
* add extension that tracks search instances and properly calls their clean up methods according to test lifecycle
* add comment explaining how the closing of testable search instances works with the extension
* use convert rule to extension
* convert rule to extension
* remove old import
* convert SkipDefaultIndexTemplateWatcher to junit5 extension api
* migrate from @Rule to @SearchInstance
* convert remaining @Rule to @SearchInstance
* clean up imports
* replace legacy RestoreSystemProperties rules with custom extension
* migrate all remaining junit4 references
* replace legacy system-rules dep with more modern one for system stream capturing
* remove unused junit4 import
* remove junit5-system-extensions, causes junit engine crashes
* remove legacy junit4 dependency for good :party:
* remove unnecessary openrewrite doc
* move forbiddenapi back out of profile, to avoid it being disabled when other profiles are explicitly given
* suppress forbidden api check for search instance detection
* remove jukito + vintage engine and adapt usage of legacy junit4 apis
* add core junit4 classes Test and Assert to forbidden apis
* replace forbidden junit4 api with junit6 equivalent
* remove guava testlib as it pulls in junit4, replaced FakeTicker with a simpler custom implementation for tests
* update testcontainers to 2.0.2 to get rid of the junit4 transitive dependency
needed to remove all ExternalResources
* migrated MongoDBInstance to MongoDBTestService (basically identically anyway)
* inlined some unnecessary interfaces
removed unused imports of junit4 classes
* migrate junit4 api usage
mockito with junit5 is much stricter about argument matcher types, so we need to adapt to Set -> Collection signature change
* actually start the container now that MongoDBInstance.java is gone
* improve test code
* simplify test case
* simplify test case
* simplify test case
* simplify test case
* simplify test case
* simplify test case
* forgot to remove the commented junit4 superclass
* rename MongoDBTestService#create methods to #createStarted and make start() private
* don't close the per-class mongodb instance for @Nested junit test classes
otherwise the first afterAll callback will destroy the instance and subsequent tests will fail
* Hot reloading of datanode certificates
* Differentiate between certificate change and certificate renewal in datanode events
* fixed log message
* added changelog
* code cleanup
* code cleanup and typos
* remove event handling as this also triggers the `OpensearchConfigurationChangeEvent`
* Fix DatanodeRestApiProxy
* make keystore a singleton to respect synchronized calls
---------
Co-authored-by: Matthias Oesterheld <33032967+moesterheld@users.noreply.github.com>
Co-authored-by: Matthias Oesterheld <matthias.oesterheld@graylog.com>
* first batch of changes to remove the matrix test engine
we are migrating matrix tests into github actions and controlling versions externally
this change starts moving the relevant container management code into the extension
this is work in progress, most tests will fail and fixtures aren't imported yet.
more sweeping changes need to be done to avoid constructor injection due to lifecycle calling order issues
* implement class-lifecycle container handling, and adjust inputcreationIT
* have junit close the backends when their store goes out of context, convert more tests
* we no longer need a vm shutdown hook, junit's extension takes care of the closing
* allow setting the mongodb and search server versions using system properties
* add support for requiring data node in tests, if we test functionality only present for that backend
* fix wrong check for data node and skipping tests, i've borked it
* convert constructor injection to lifecycle method
* convert constructor injection to lifecycle method
* convert constructor injection to lifecycle method
* convert constructor injection to lifecycle method
* convert constructor injection to lifecycle method
* convert constructor injection to lifecycle method
* convert constructor injection to lifecycle method
* convert constructor injection to lifecycle method
* convert constructor injection to lifecycle method
* convert constructor injection to lifecycle method
* convert constructor injection to lifecycle method and fix warnigs
* convert constructor injection to lifecycle method
* convert constructor injection to lifecycle method
* convert constructor injection to lifecycle method
* convert constructor injection to lifecycle method
* convert constructor injection to lifecycle method
* convert constructor injection to lifecycle method and fix warnings
* load mongodb fixtures and pass through feature flags to allow running plugin tests
* Speed up MavenPackager by by 7 minutes when building JARs
We don't need to create the assemblies.
* Use a single container network for all tests
We cache MongoDB and OpenSearch services so we can't use a new network
on every test. Otherwise, the server would be in a different network
when using cached services.
* More verbose logging for class-lifecycle server backends
* Exit server backend startup early when seeing an unknown host error
* Log MongoDB version in ContainerizedGraylogBackend
* Return the version string instead of the enum
* Fix warning in ContainerizedGraylogBackend
* Don't create a new MongoDB instance for every test
* don't use constructor argument for test base class, it must be static through the lifecycle method
* convert remaining tests to use lifecycle injection
* Introduce #cachedInstance to SearchServerBuilder
Allows the creation of uncached search instances.
* Add MongoDBInstance#instanceId
* Add SearchServerInstance#instancdId
* Stop server early if MongoDB is not reachable
* Fix caching issues in services provider
Disable multi-layered caches and only cache at the service layer.
* We need the server assembly
* Remove connection failure detector again
Exiting early is problematic because a retry might fix the problem.
Add check for NoClassDefFoundError
* make error message nicer
* tests require data node, so skip them on other search versions
* test only runs on data node
* remove custom junit engine
rename annotation to FullBackendTest
remove deprecated annotation options
always start mail and webhook services, they are tiny and fast, simplifies this further (and cuts down on container starts)
some warning cleanup
* re-enable maven packager, committed this by mistake
* Only allow FullBackendTest annotation in full-backend-tests
* Let WebInterfaceAssetsResourceWithPrefixIT run in CLASS lifecycle
The test needs custom configuration.
* Don't allow VM lifecycle with customized config or feature flags
* test with custom config must run in class lifecycle
* Remove commented cache key parts
* Shutdown CLASS services with non-empty config or feature flags
We don't need services for custom configurations to stay around.
* Improve Guice error detection on server startup
* Replace MongodbServer with MongoDBVersion
* Remove unused MongoDBVersion#PATTERN
* Remove unused ContainerVersions
* Extract search server and MongoDB version provider
* Remove unused imports
* Remove SearchServerInstance#searchServer method
Also convert SearchServer from an enum to a POJO with some constants.
* Move SearchServer to org.graylog.testing.storage
* Change log level in GraylogBackendExtension
* Add EnabledIfSearchServer annotation and condition
* Switch DatanodeOpensearchProxyIT to EnabledIfSearchServer
* Switch more tests to EnabledIfSearchServer annotation
* Remove GraylogBackendConfiguration#onlyOnDataNode
* Fix EnabledIfSearchServerCondition class name
* Remove unused DatanodeDevInstance and its builder and provider
* Ensure Maven packaging before starting backend services
Also move check for GRAYLOG_IT_SKIP_PACKAGING environment variable into
the MavenPackager to avoid duplication.
* Set Data Node version to Version.CURRENT_CLASSPATH
* Improve log message in ContainerizedGraylogBackend
* Fix EnabledIfSearchServerCondition logic
Don't check the presence of a version when distribution is DATANODE.
* Add Javadoc to EnabledIfSearchServer
* Fix DatanodeInstance to build the local image
* Don't set GRAYLOG_ELASTICSEARCH_VERSION env when running Data Node
Fixes a startup error where the search version validator failed.
* Pass missing env to DatanodeInstance
* Remove backend services from cache when they get closed
* Adjust name for local Data Node image to be consistent with the server
* Adjust log message in MavenPackager when skipping
* Handle Exception and AssertError in GraylogRestApi#waitFor
Otherwise, we would exit the wait loop when the predicate throws an
exception. (e.g., when we get a 404 instead of a 200)
* Add log().ifError() in WebInterfaceAssetsResourceBase
* take the cluster id from the imported licenses instead of manually setting it through a fixture
* Remove unused GraylogBackendExtension#resolveFixtures method
* Fix Data Node SearchVersionRange checks
This was hardcoded to 5.2.0 because that's the version we used in our
tests. (regardless of which actual version was running)
We are now building the version from the Version.CURRENT_CLASSPATH.
* Relax Data Node search server check
* Pass MavenProjectDirProvider into services provider
* Add the include-frontend flag to the services cache key
* Revert "Add the include-frontend flag to the services cache key"
This reverts commit c25b1a5b3a.
* Revert "Pass MavenProjectDirProvider into services provider"
This reverts commit 37001edcc9.
* Remove MavenProjectDirProviderWithFrontend
* Rename ContainerMatrixElasticsearchBaseTest to SearchServerBaseTest
* Remove unused NodeContainerFactory#containerFileExists
* Don't memoize customization config in customization config provider
It makes testing really hard and requires a server restart to change the
values.
* Fix signal handling in webhook.js test server
Stopping webhook server container was blocking for a long time.
* Add EnabledIfMongoDB annotation
* Move MavenPackager call up into ContainerizedGraylogBackend
That way we will build the JAR files at the very beginning and with the
correct MavenProjectDirProvider. (at least for the first test that runs)
* Adjust MavenPackager logs
* Make completebackend.api.Views#getAll more robust
* Silence MavenPackager
* Use defaultIfBlank instead of requireNonNullElse
* Also use defaultIfBlank for MongoDB version
* Remove unused service image properties
We can add them back if we should implement image selection.
* Cleanup GraylogBackendExtension
* Rename ConfigurationParameter to Env in GraylogBackendConfiguration
* Prevent importLicenses=false in VM lifecycle
* Add Javadoc to GraylogBackendConfiguration
* Move GraylogBackendConfiguration and FullBackendTest annotations
Those were the last files in the containermatrix package.
* More Javadoc
* Remove services from cache before stopping them
* Log if validation fails
* Introduce Server and Data Node product configuration
This replaces the low-level provider configurations for Maven and plugin
JAR files.
* Don't implement AutoClosable for ContainerizedGraylogBackendServicesProvider
We never close it anywhere. Convert the instance method to a static
cleanup method.
* Remove mavenProjectDirProvider, pluginJarsProvider, datanodePluginJarsProvider
* Fix duplicate BeforeAll methods
* Fix logging in completebackend.apis.Views
* Re-enable DbEntitiesCatalogProviderTest
* Use SearchServer.DEFAULT_VERSION for the default distribution
* Remove unused RunningGraylogBackend
* Check composable index templates feature flag in AdaptersES7
Fixes IndicesIT with Elasticsearch 7 and OpenSearch 1.x because they've
always been using composable index templates.
* Remove unused RunningElasticsearchInstanceES7 class
* Add sleep to ensure updatedAt differs when reacquiring lock
* Make EnabledIfSearchServer annotation repeatable
* Improve wording in EnabledIfSearchServerCondition
* Bump default heap in SearchServerBuilder to 1 GB
* Don't set "elasticsearch_version" and let the server detect the version
* Adjust IndexRangesCleanupIT to handle AssertionError in retry loop
* Remove unneccessary warning supression in DatanodeProvisioningIT
* Remove unused testClass param in GraylogBackendExtension#createBackend
---------
Co-authored-by: Bernd Ahlers <bernd@graylog.com>
Co-authored-by: Bernd Ahlers <bernd@users.noreply.github.com>
Co-authored-by: Othello Maurer <othello@graylog.com>
