opensource/graylog2-server
1
0
Fork 0
mirror of https://github.com/Graylog2/graylog2-server.git synced 2026-03-13 09:32:21 +08:00
Code Issues Packages Projects Releases Wiki Activity
22,194 Commits 400 Branches 498 Tags
bc56d397c366e3394cd1f2d193db7e7c0fc0fd18
Commit Graph

18 Commits

Author SHA1 Message Date
Tomas Dvorak
bc56d397c3 Datanode config separation (#15701) 
* Different variants of security configuration (from file, from mongo, insecure).
Encryption/decryption pattern used in practice.

* Reverting temporary changes

* More precise certificate presence check

* Added unit tests for testHasCert() method

* Datanode and opensearch configuration refactored

* code cleanup

* code cleanup

* code cleanup

* Node preflight config service refactored

* restbaseurl moved to OpensearchConfiguration

---------

Co-authored-by: Lukasz Kaminski <lukasz.kaminski@graylog.com>
 2023-06-06 14:44:06 +02:00
luk-kaminski
ea4804b874 OTR Reencryption and Key Storage refactoring for password change (#15668) 
* OTR Reencryption and Key Storage refactoring for password change

* Wrong password was used in reencryption. Fixed.
 2023-06-02 14:07:54 +02:00
Tomas Dvorak
5651f90aa9 Process watchdog reimplemented to follow transitions of the process state machine (#15676) 
* Process watchdog reimplemented to follow transitions of the process state machine

* code cleanup

* code cleanup
 2023-06-02 10:52:24 +02:00
luk-kaminski
6f83f2be63 Refactoring and additional tests (#15640) 
* Refactoring and additional tests

* One more try-with-resources
 2023-05-30 14:40:27 +02:00
Tomas Dvorak
a487ca2f86 Delay opensearch start till configuration is available (#15615) 
* Delay opensearch start till configuration is available

* code cleanup
 2023-05-30 10:06:47 +02:00
luk-kaminski
f9dc221627 Change ConfigProvider, so that it stops using PEM files, and starts u… (#15460) 
* Change ConfigProvider, so that it stops using PEM files, and starts using PKCS12 files.
Refactoring, splitting the functionality to different classes.
Some config changes that make things work in IDEA.
Reusable truststore.
Truststore stores only root certificates.

* Simpler root certificate finding

* Dir Sync does not remove cert files

* Removed unnecessary default config changes. Cleared confusing certificate copying and unnecessary certificate duplication in 2 folders

* Truststore path created outside of TruststoreCreator. Root certificate found in a different way, without considering its place in the certificate chain

* Fixed log message

* Integration test fixed

* Certs are not an exception in FullDirSync. Introduced second config location as a step towards final configuration. Certificates are moved between locations, as it was done previously

* Fixed a test. Minor refactoring.

* Config location has no default value. It is checked if it is provided, as it is required
 2023-05-16 08:38:03 +02:00
Tomas Dvorak
2d59457315 Fixed env propagation for datanodes opensearch (#15458) 2023-05-09 14:54:56 +02:00
Tomas Dvorak
2fa7b7b9aa Synchronize opensearch config to a user-defined directory (#15307) 
* Synchronize opensearch config to a user-defined directory (fixes #15223)

* fixed check naming

* fixed opensearch configuration location

* simplified Environment constructor

* point gc logs of datanodes opensearch to /tmp for now

* full dir sync for opensearch config

* Fixed full dir sync when removing non-empty dir

* simpified recursive directory deletion
 2023-05-02 15:01:12 +02:00
Tomas Dvorak
c36a63495d Opensearch dist dir autodetection (#15216) 
* Opensearch dist dir autodetection

* Add arm64 as supported architecture for datanode

* Added changelog
 2023-04-14 09:53:33 +02:00
luk-kaminski
b48335a7dd Move certutil code to graylog-server (#15193) 2023-04-13 08:03:01 +02:00
Tomas Dvorak
7a66235734 datanode registers itself with opensearch REST endpoint uri (#15142) 
* datanode registers itself with opensearch REST endpoint uri

* code cleanup

* Updating yarn lockfile (#15132)

Co-authored-by: Gary Bot <garybot2@graylog.com>

* Fix isFreshInstallation preflight check (#15141)

* Fix isFreshInstallation preflight check

* added changelog

* removed outdated comment

* code cleanup

* Fix package name and use correct Java property value for log4j2 (#15144)

* Add undo reset button to encrypted form fields (#15147)

* Add undo reset button to encrypted form fields

* Also reset the file name

* Reference PRs in changelog

* Input form: Only reset title when initial title changes or modal is closed (#15154)

* Extended preflight check, added logging and adapted state machine transitions (#15149)

* Extended preflight check, added logging and adapted state machine transitions

* fixed opensearch preflight check

* fixed opensearch preflight check, add tests

* Fix OpenSearch permissions in data node tarball (#15158)

We tried to use fileSet/fileMode to make the OpenSearch config
permissions work when running OpenSearch with a different user.

That didn't work because the fileMode changed all files and removed the
executable bit from all executable scripts.

Now we use the Maven antrun plugin to set the permissions on the
OpenSearch files right after extraction.

* Bump webpack in /graylog2-web-interface/packages/graylog-web-plugin (#15159)

Bumps [webpack](https://github.com/webpack/webpack) from 5.77.0 to 5.78.0.
- [Release notes](https://github.com/webpack/webpack/releases)
- [Commits](https://github.com/webpack/webpack/compare/v5.77.0...v5.78.0)

---
updated-dependencies:
- dependency-name: webpack
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add CIDR Range lookup support to CSV file data adapters (#15016)

* Add CIDR Range lookup support to CSV file data adapters

* Convert JSX -> functional TSX

* Adds unit tests

* Support single IP keys

* Adapted default configuration, fixed logging, extended datanode.conf template (#15056)

Co-authored-by: Bernd Ahlers <bernd@users.noreply.github.com>

* Feat/4848 modal add to investigation (#15148)

* feat(4848): Adds buttons to open select investigation modal

* feat(4848): Implements AddEvidenceModal & fix for OverlayDropdown

* feat(4848): Adds tests

* feat(4848): Adds changelog

* feat(4848): Adds missing license header

* Implement node heartbeats without server-side scripting. (#15160)

* Implement node heartbeats without server-side scripting.

* Adding backwards compatibility.

* Removing unused method.

* Don't pass JAVA_HOME environment variable to OpenSearch process (#15166)

The OpenSearch process should use its bundled JVM version.

* Bump eslint-plugin-compat (#15169)

Bumps [eslint-plugin-compat](https://github.com/amilajack/eslint-plugin-compat) from 4.1.2 to 4.1.4.
- [Release notes](https://github.com/amilajack/eslint-plugin-compat/releases)
- [Changelog](https://github.com/amilajack/eslint-plugin-compat/blob/main/CHANGELOG.md)
- [Commits](https://github.com/amilajack/eslint-plugin-compat/compare/v4.1.2...v4.1.4)

---
updated-dependencies:
- dependency-name: eslint-plugin-compat
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump @tanstack/react-query (#15179)

Bumps [@tanstack/react-query](https://github.com/tanstack/query) from 4.28.0 to 4.29.1.
- [Release notes](https://github.com/tanstack/query/releases)
- [Commits](https://github.com/tanstack/query/compare/v4.28.0...v4.29.1)

---
updated-dependencies:
- dependency-name: "@tanstack/react-query"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump typescript in /graylog2-web-interface/packages/graylog-web-plugin (#15173)

Bumps [typescript](https://github.com/Microsoft/TypeScript) from 5.0.3 to 5.0.4.
- [Release notes](https://github.com/Microsoft/TypeScript/releases)
- [Commits](https://github.com/Microsoft/TypeScript/compare/v5.0.3...v5.0.4)

---
updated-dependencies:
- dependency-name: typescript
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump eslint in /graylog2-web-interface/packages/eslint-config-graylog (#15174)

Bumps [eslint](https://github.com/eslint/eslint) from 8.37.0 to 8.38.0.
- [Release notes](https://github.com/eslint/eslint/releases)
- [Changelog](https://github.com/eslint/eslint/blob/main/CHANGELOG.md)
- [Commits](https://github.com/eslint/eslint/compare/v8.37.0...v8.38.0)

---
updated-dependencies:
- dependency-name: eslint
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump puppeteer from 19.8.0 to 19.8.5 in /graylog2-web-interface (#15170)

Bumps [puppeteer](https://github.com/puppeteer/puppeteer) from 19.8.0 to 19.8.5.
- [Release notes](https://github.com/puppeteer/puppeteer/releases)
- [Changelog](https://github.com/puppeteer/puppeteer/blob/main/release-please-config.json)
- [Commits](https://github.com/puppeteer/puppeteer/compare/puppeteer-v19.8.0...puppeteer-v19.8.5)

---
updated-dependencies:
- dependency-name: puppeteer
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump @typescript-eslint/eslint-plugin (#15181)

Bumps [@typescript-eslint/eslint-plugin](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/eslint-plugin) from 5.57.1 to 5.58.0.
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/eslint-plugin/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v5.58.0/packages/eslint-plugin)

---
updated-dependencies:
- dependency-name: "@typescript-eslint/eslint-plugin"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump @types/react in /graylog2-web-interface/packages/graylog-web-plugin (#15178)

Bumps [@types/react](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/react) from 18.0.33 to 18.0.34.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/react)

---
updated-dependencies:
- dependency-name: "@types/react"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bootstrapping preflight UI build. (#15138)

* Adding preflight build config and targets.

* Emitting and copying preflight assets from separate dir.

* Run preflight build from maven.

* Adding license headers.

* Removing duplicate `div#app-root`.

* Merging multiple plugin executions.

* Bump @typescript-eslint/parser (#15180)

Bumps [@typescript-eslint/parser](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/parser) from 5.57.1 to 5.58.0.
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/parser/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v5.58.0/packages/parser)

---
updated-dependencies:
- dependency-name: "@typescript-eslint/parser"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Security update to shiro-core 1.11.0 (#15187)

* Security update to shiro-core 1.11.0

Release notes:

- https://shiro.apache.org/blog/2023/01/13/apache-shiro-1110-released.html
- https://shiro.apache.org/blog/2022/11/19/apache-shiro-1101-released.html
- https://shiro.apache.org/blog/2022/10/10/apache-shiro-1100-released.html
- https://shiro.apache.org/blog/2022/06/28/apache-shiro-191-released.html
- https://shiro.apache.org/blog/2022/03/22/apache-shiro-190-released.html
- https://shiro.apache.org/blog/2021/v1.8.0.html
- https://shiro.apache.org/blog/2021/v1.7.1.html
- https://shiro.apache.org/blog/2020/v1.7.0.html
- https://shiro.apache.org/blog/2020/v1.6.0.html
- https://shiro.apache.org/blog/2020/05/03/apache-shiro-1.5.3-released.html

Fixed CVEs:

- CVE-2022-32532 - Doesn't affect Graylog because we don't run in a
  servlet container
- CVE-2022-40664 - Doesn't affect Graylog because we don't run in a
  servlet container
- CVE-2023-22602 - Doesn't affect Graylog because the issue is Spring
  Boot specific

* Add changelog

* Update netty from 4.1.84 to 4.1.91 (#15186)

netty-tcnative from 2.0.54 to 2.0.59

fixed CVEs

- HAProxyMessageDecoder Stack Exhaustion DoS (CVE-2022-41881)
- HTTP Response splitting from assigning header value iterator (CVE-2022-41915)

* 1.Fixed when using OSHI library to get cpu load is always 0. (#15183)

2.Fixed getting CPU frequency could be negative because CPU frequency could exceed Int max value.

* Update json-path to 2.8.0 to fix a security issue in json-smart (#15190)

* Update json-path to 2.8.0 to fix a security issue in json-smart

Fixes CVE-2023-1370 in the json-smart transitive dependency.

* Add changelog

* Fix "Add to query" value action for dashboard widgets. (#15182)

* Fix "Add to query" value action for dashboard widgets.

* Add few simple tests for "add to query" value action.

* Add test to ensure "Add to query" works on dashboards if global override is undefined.

* Cleanup code

* Fix error.

* Updating yarn lockfile (#15162)

Co-authored-by: Gary Bot <garybot2@graylog.com>

* Add regex matching to remove_field function (#15131)

* add regex matching to remove_field

* update CL

* Configuration Form Makeover (#15025)

* Add sidebar and only show selected configuration section

* Extract calls for Search and MessageProcessorsConfig into their component

* Migrate SidecarConfig to typescript and fetch config data in component

* Migrate EventsConfig to typescript and move fetch logic into component

* Move fetch logic to UrlWhitelist component

* Move fetch logic to Permissions component and use store in Searches config

* Use store in MessageProcessorConfig

* Use store in SidecarConfig

* Use store in EventsConfig

* Use store in UrlWhiteListConfig

* Use store in IndexSetsDefaultsConfig

* Use store in UserConfig

* Clean up ConfigurationPage component and create component for plugins

* Fix tests and make sections more narrow

* Fix SearchConfig after merge and add typing to MessageProcessorConfig

* Disable eslint warning for error boundary

* Adjust headlines

* Add missing license headers

* Add changelog

* Move changelog file to correct folder

* Add arrow to plugins nav item in configurations page

* Format with correct eslint config

* Use same text for submit and cancel button for IndexSetsDefaultsConfig

* Make sure the sidebars have the same width

* Extract Icons to styled components

* NodePingPeriodical registers node in init

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Gary Bot <garybot2@graylog.com>
Co-authored-by: Bernd Ahlers <bernd@users.noreply.github.com>
Co-authored-by: Laura <grotlue@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Zack King <91903901+kingzacko1@users.noreply.github.com>
Co-authored-by: Ezequiel Lopez <105670363+zeeklop@users.noreply.github.com>
Co-authored-by: Dennis Oelkers <dennis@graylog.com>
Co-authored-by: Marco Pfatschbacher <marco@graylog.com>
Co-authored-by: panxt <15639727428@163.com>
Co-authored-by: Linus Pahl <linus@graylog.com>
Co-authored-by: Patrick Mann <patrickmann@users.noreply.github.com>
 2023-04-12 13:51:37 +02:00
Bernd Ahlers
a2a2ee7af8 Don't pass JAVA_HOME environment variable to OpenSearch process (#15166) 
The OpenSearch process should use its bundled JVM version.
 2023-04-07 12:49:40 +02:00
Tomas Dvorak
eba2d54498 Extended preflight check, added logging and adapted state machine transitions (#15149) 
* Extended preflight check, added logging and adapted state machine transitions

* fixed opensearch preflight check

* fixed opensearch preflight check, add tests
 2023-04-05 18:14:15 +02:00
Bernd Ahlers
120fae5221 Refactor data-node download and assembly (#14963) 
* Refactor data-node download and assembly

- Download OpenSearch aarch64 distribution in addition to x64
- Extract OpenSearch distribution into "target/opensearch/*" instead of
  "bin/download".
- Include OpenSearch distributions in "dist/" instead of "bin/" in
  assembly

* Fix data-node test container creation

* Adjust OpenSearch location for architecture dependent locations

* If the configured location exists, use it

Otherwise, check for architecture dependent distribution directories.

* Switch default config file to locations that work with the tarball

* Rename datanode.jar to graylog-datanode.jar

* Rename datanodectl to graylog-datanode.sh

* Include graylog-datanode.sh in assembly as bin/graylog-datanode

* Add a jvm.options config file

* Add a production log4j2.xml config file

* Use bundled JVM if it exists

* fixed docker image for datanode integration tests

---------

Co-authored-by: Tomas Dvorak <tomas.dvorak@graylog.com>
 2023-03-28 10:49:49 +02:00
Tomas Dvorak
3f4dad6a7b Datanode periodical for NodeService pings (#15003) 
* Datanode periodical for NodeService pings

* node service markAsAlive update without additional query

* added unit test for NodePingPeriodical

* code cleanup

* simplified NodePingPeriodicalTest
 2023-03-23 09:02:35 +01:00
Jan Heise
94d8dc7802 Additional Logging to track down eventual errors in IT tests (#14863) 
* reverting preemptive auth using restassured

* add possibility to set a minimum error code (if you don't expect a 4xx for example)

* upped log level

* adding preemptive back in

* Update GraylogBackend.java

* remove preemptive()
 2023-03-09 12:14:05 +01:00
Jan Heise
0b5d6b0307 Add Preemptive Auth to IT Tests (#14833) 
* preemptive auth during IT tests everywhere

* removed unneccessary log
 2023-03-02 17:03:03 +01:00
Tomas Dvorak
21e55cbcb6 Initial Data Node version PR (#14417) 
Co-authored-by: Jan Heise <jan.heise@graylog.com>
 2023-02-27 13:11:24 +01:00