892 Commits

Author SHA1 Message Date
f76f426da3 Chore: Refactor Prometheus HTTP client middleware (#34473)
Following #33439 this refactors the Prometheus HTTP transport 
which is replaced by HTTP client middleware.
2021-05-27 12:43:21 +02:00
ef0fab9aa5 Expose folder UID in dashboards API response (#33991)
* expose folder UID in dashboards API response, import dashboards into folders by folder UID

* handle bad folder UID as 400 error

* 12591:Add tests for request with folderUid

* Use more descriptive error status for missing folders

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* return 400 when folder id is missing

* put error checking in the right place this time

* mention folderUid in the docs

* Clarify usage of folderUid and folderId when both present

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* Capitalise UID

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* mention folder UID in the metadata for a GET response

Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
2021-05-26 10:20:13 -04:00
91657dad18 HTTP Client: Make ResponseHeaderTimeout default timeout in http client (#34597)
* HTTP Client: Add `ResponseHeaderTimeout` - split from `DialContext` timeout

* Fixes according to reviewer's comments

* Use grafana-plugin-sdk-go v0.100.0
2021-05-25 11:32:41 +03:00
7204a64717 LibraryElements: Creates usage stats for panels and variables (#34476)
* LibraryPanels: Adds usage collection

* Refactor: renames Panel and Variable consts

* Chore: initialize stats

* Refactor: moves library element migrations to migration namespace
2021-05-24 06:11:01 +02:00
348e76fc8e Datasource: Shared HTTP client provider for core backend data sources and any data source using the data source proxy (#33439)
Uses new httpclient package from grafana-plugin-sdk-go introduced 
via grafana/grafana-plugin-sdk-go#328. 
Replaces the GetHTTPClient, GetTransport, GetTLSConfig methods defined 
on DataSource model.
Longer-term the goal is to migrate core HTTP backend data sources to use the 
SDK contracts and using httpclient.Provider for creating HTTP clients and such.

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2021-05-19 23:53:41 +02:00
e291203528 Alerting: a usage stat for ualerting (#34390)
collects `stats.alert_rules.count` , where alert_rules is the nextgen system
2021-05-19 10:31:25 -04:00
f2fcf721eb 32540: Add org users with pagination (#33788)
* Add model for search org user and add handler for dispatch

* 32540_org_users_with_pagination: Add endpoint for search org users

* 32540_org_users_with_pagination: Add test for org user search handler

* 32540_org_users_with_pagination: fix indentation

* 32540_org_users_with_pagination: Remove newline

* 32540_org_users_with_pagination: Remove empty line

* 32540_org_users_with_pagination: Fix indentation

* Update pkg/api/org_users.go

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* Update pkg/api/org_users.go

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* Update pkg/models/org_user.go

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* Update pkg/api/org_users_test.go

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* Update pkg/api/org_users_test.go

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* 32540_org_users_with_pagination: Use hs.SQLStore.SearchOrgUsers instead of bus

* Add model for search org user and add handler for dispatch

* 32540_org_users_with_pagination: Add endpoint for search org users

* 32540_org_users_with_pagination: Add test for org user search handler

* 32540_org_users_with_pagination: fix indentation

* 32540_org_users_with_pagination: Remove newline

* 32540_org_users_with_pagination: Remove empty line

* 32540_org_users_with_pagination: Fix indentation

* Update pkg/api/org_users.go

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* Update pkg/api/org_users.go

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* Update pkg/models/org_user.go

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* Update pkg/api/org_users_test.go

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* Update pkg/api/org_users_test.go

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* 32540_org_users_with_pagination: Use hs.SQLStore.SearchOrgUsers instead of bus

* 32540_org_users_with_pagination: Add test for the sqlstore

* 32540_org_users_with_pagination: Fix sqlstore test

* Update pkg/api/org_users.go

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* Update pkg/api/org_users_test.go

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* Update pkg/api/org_users_test.go

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* Update pkg/services/sqlstore/org_users.go

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* Update pkg/services/sqlstore/org_users.go

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* Update pkg/services/sqlstore/org_test.go

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* Update pkg/services/sqlstore/org_test.go

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* 32540: Fix search org users method

* 32540: Fix sqlstore test

* 32540: Fix go-lint

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2021-05-12 14:10:35 +02:00
95a356a840 Live: using organisation IDs throughout stack (#33714) 2021-05-11 22:03:04 +03:00
540f110220 [Alerting]: Extend quota service to optionally set limits on alerts (#33283)
* Quota: Extend service to set limit on alerts

* Add test for applying quota to alert rules

* Apply suggestions from code review

Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>

* Get used alert quota only if naglert is enabled

* Set alert limit to zero if nglalert is not enabled
Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>
2021-05-04 19:16:28 +03:00
0c2bcbf2bc Live: persisting last message in cache for broadcast scope (#32938) 2021-04-30 11:06:33 -07:00
748778fff0 new endpoint to post/get trimmed dashboard json (#33465)
* new endpoint to post/get trimed dashboard json

* add isdisabled check in dashboard.go
2021-04-28 19:38:33 +08:00
076e2ce06a Return 404 when deleting non-existing API key (#33346)
The server needs to return a HTTP 404 (Not Found) when an API key that does not exist is deleted.
2021-04-28 13:30:09 +02:00
1dd9e9b184 Live: support a dashboard gitops channel (#33291) 2021-04-23 21:55:31 +02:00
23c7e7ab60 Alerting: Various fixes for the alerts endpoint (#33182)
A set of fixes for the GET alert and groups endpoints.

- First, is the fact that the default values where not being for the query params. I've introduced a new method in the Grafana context that allow us to do this.
- Second, is the fact that alerts were never being transitioned to active. To my surprise this is actually done by the inhibitor in the pipeline - if an alert is not muted, or inhibited then it's active.
- Third, I have added an integration test to cover for regressions.

Signed-off-by: Josue Abreu <josue@grafana.com>
2021-04-21 06:34:42 -04:00
a92bcce9de Live: push scope to demultiplex incoming data to stream channels (#32808) 2021-04-08 12:40:06 -07:00
8e70d5d181 Add timeout option to datasource config (#31871) 2021-04-07 10:46:19 -04:00
93292f6eef Live: update Streaming plugin definitions, put frame schema in subscribe result data (#32561)
Co-authored-by: Torkel Ödegaard <torkel@grafana.org>
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2021-04-02 19:41:45 +03:00
39a3b0d0b0 Auth: support JWT Authentication (#29995) 2021-03-31 08:40:44 -07:00
f456c32f92 Chore: remove measurement.go (#32526) 2021-03-31 07:52:19 -07:00
da05b7a07b Live: http publish, refactor live interfaces (#32317) 2021-03-30 13:23:29 +03:00
20bd591bea Access control: Basic structure and functionality behind feature toggle (#31893)
Co-authored-by: Alexander Zobnin <alexander.zobnin@grafana.com>
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
Co-authored-by: Arve Knudsen <arve.knudsen@grafana.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@grafana.com>
2021-03-22 13:22:48 +01:00
5c07df9f4b Chore: Replace Command dispatches by explicit calls (#32131)
* Clean up GetProvisionedDashboardDataByIdQuery

* Clean up SaveProvisionedDashboardCommand

* Clean up & fix AddTeamMemberCommand usages

* Clean up & fix UpdateUserPermissionsCommand usages

* Lint imports
2021-03-19 09:14:14 +01:00
87c3a2b790 PluginManager: Make Plugins, Renderer and DataSources non-global (#31866)
* PluginManager: Make Plugins and DataSources non-global

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix integration tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Replace outdated command

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* DashboardService: Ensure it gets constructed with necessary parameters

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix build

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* DashboardService: Ensure it gets constructed with necessary parameters

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Remove dead code

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix test

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix test

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Remove FocusConvey

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix test

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Remove dead code

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Undo interface changes

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Backend: Move tsdbifaces.RequestHandler to plugins.DataRequestHandler

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Rename to DataSourceCount

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Consolidate dashboard interfaces into one

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix test

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix dashboard integration tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2021-03-17 16:06:10 +01:00
0c2045109e Chore: Collect elasticsearch version usage stats (#31787)
* Chore: Collect elasticsearch version usage stats

* Fix lint error

* use GetDataSources from sqlstore

* Apply review suggestions

* Return error if datasource type is not specified

* Update pkg/services/sqlstore/datasource.go

* fix undefined var
2021-03-17 09:14:53 +00:00
610999cfa2 Auth: Allow soft token revocation (#31601)
* Add revoked_at field to user auth token to allow soft revokes

* Allow soft token revocations

* Update token revocations and tests

* Return error info on revokedTokenErr

* Override session cookie only when no revokedErr nor API request

* Display modal on revoked token error

* Feedback: Refactor TokenRevokedModal to FC

* Add GetUserRevokedTokens into UserTokenService

* Backendsrv: adds tests and refactors soft token path

* Apply feedback

* Write redirect cookie on token revoked error

* Update TokenRevokedModal style

* Return meaningful error info

* Some UI changes

* Update backend_srv tests

* Minor style fix on backend_srv tests

* Replace deprecated method usage to publish events

* Fix backend_srv tests

* Apply suggestions from code review

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
Co-authored-by: Hugo Häggmark <hugo.haggmark@gmail.com>

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Hugo Häggmark <hugo.haggmark@gmail.com>

* Minor style fix after PR suggestion commit

* Apply suggestions from code review

Co-authored-by: Ursula Kallio <73951760+osg-grafana@users.noreply.github.com>

* Prettier fixes

Co-authored-by: Hugo Häggmark <hugo.haggmark@gmail.com>
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
Co-authored-by: Ursula Kallio <73951760+osg-grafana@users.noreply.github.com>
2021-03-16 17:44:02 +01:00
01b4048026 Usage Stats: Report usage for all signed plugins (#31545)
* Report usage for all the signed plugins

* Remove no longer used code

* Fix tests by manipulating the global state

* Minor test fix
2021-03-04 15:43:43 +01:00
663d44fe82 Login: handle custom token creation error messages (#31283)
* Login: handle custom token creation errors

* Login: handle external error for OAuth responses
2021-02-25 15:30:51 +01:00
6415d2802e Plugins: Requests validator (#30445)
* Introduce PluginRequestValidator abstraction with a NoOp implementation

* Update PluginRequestValidator abstraction to use the dsURL instead

* Inject PluginRequestValidator into the HTTPServer and validate requests going through data source proxy

* Inject PluginRequestValidator into the BackendPluginManager and validate requests going through it

* Validate requests going through QueryMetrics & QueryMetricsV2

* Validate BackendPluginManager health requests

* Fix backend plugins manager tests

* Validate requests going through alerting service

* Fix tests

* fix tests

* goimports

Co-authored-by: Leonard Gram <leo@xlson.com>
2021-02-03 20:47:45 +01:00
3b7b49a4aa Auth: Use SigV4 lib from grafana-aws-sdk (#30713)
* replace with lib

* remove test + apply feedback
2021-02-01 16:07:27 +01:00
c04bc805b5 CDN: Adds support for serving assets over a CDN (#30691)
* CDN: Initial poc support for serving assets over a CDN

* Minor fix

* added build path and test

* fix lint error

* Added edition to cdn path

* Move master builds to a separate path

* Added error handling for the url parsing, changed setting name, and added docs

* Updated sample.ini

* Some property renames

* updated

* Minor update to html

* index template improvements

* Update docs/sources/administration/configuration.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/administration/configuration.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Added ContentDeliveryPrefix to Licence service

* updated docs

* Updated test mock

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
2021-02-01 10:13:09 +01:00
1d399b89ea Usage Stats: Remove unused method for getting user stats (#30074)
* Usage Stats: Remove unused method for getting user stats

This was used in enterprise codebase, but is not longer in use, thus can be safely removed.

* Force refresh user stats in tests
2021-01-26 14:13:25 +01:00
2a70c73025 Auth: add expired token error and update CreateToken function (#30203)
* Auth: add error for expired token

* Auth: save token error into context data

* Auth: send full user and req context to CreateToken

* Auth: add token ID in context

* add TokenExpiredError struct

* update auth tests

* remove most of the changes to CreateToken func

* clean up

* Login: add requestURI in CreateToken ctx

* update RequestURIKey comment
2021-01-19 17:55:53 +01:00
af27dd7e7d Licensing: Send map of environment variables to plugins (#30347)
Replace hard-coded `GF_ENTERPRISE_LICENSE_TEXT` with a
`map[string]string` to allow the licensing service to allow sending any
environment variable through to the plugins for licensing purposes.
2021-01-18 14:42:39 +01:00
d6b468187e Plugins: Add Open Distro to the list of data sources supported by sigv4 (#30308) 2021-01-15 17:01:29 +03:00
a0a1422b2d Auth: Add missing request headers to SigV4 middleware allowlist (#30115)
* Auth: Add Content-Type to SigV4 header allowlist

* add MT headers to permitted headers

* add Kibana API header to allowlist

* add Content-Length header to allow list
2021-01-14 09:06:09 +01:00
1c1a800bc0 API: Add by UID routes for data sources (#29884)
- also add Get by UID+OrgID to datasource cache
- Refactor backend commands for Delete and Get to be unified
2021-01-13 13:16:27 -05:00
50b649a869 Middleware: Add CSP support (#29740)
* Middleware: Add support for CSP

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

Co-authored by @iOrcohen
2021-01-12 07:42:32 +01:00
2a9c625c9f Alerting: improve alerting default datasource search when extracting alerts (#29993)
* improve alerting search datasource

* correct the xorm get usage

* adding default datasource unittest
2021-01-07 21:33:17 +01:00
2d803ee57e Chore: Rewrite models datasource cache test to standard library (#30040) 2021-01-06 09:21:42 +01:00
d2b3bcf32f Chore: Rewrite models tags test to standard library (#30041) 2021-01-05 17:20:41 +01:00
71899a0ed7 Chore: Rewrite models dashboards test to standard library (#30023) 2021-01-01 13:29:40 +01:00
4f3a7bb0fa Chore: Rewrite models dashboard acl test to standard library (#30022) 2021-01-01 13:22:48 +01:00
8e1196e3ed Chore: Rewrite models alert test to standard library (#30021) 2021-01-01 13:18:47 +01:00
375e8e4fd0 SQLStore: customise the limit of retrieved datasources per organisation (#29358)
* SQLStore: customise the limit of retrieved datasources per organisation

* update all suggestions regarding nil or 0 as default

* Apply suggestions from code review

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>

* correct default.ini description + adding unittest

* Apply suggestions from code review

Co-authored-by: Sofia Papagiannaki <papagian@users.noreply.github.com>

* modify unittest name

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
Co-authored-by: Sofia Papagiannaki <papagian@users.noreply.github.com>
2020-12-28 12:24:42 +01:00
551f96089b Remove the bus from teamgroupsync (#29810)
* Add fake TeamSyncService

* Change SyncTeams dispatch to direct method call

* Remove SyncTeamsCommand struct

* Move TeamSyncService to a separate package

* Remove context from SyncTeams args

* Add comments to teamsyncgroup package

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>

* Replace TeamSyncService with TeamSyncFunc

* Remove unnecessary error check

* Add a test

* Fix test input

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
2020-12-23 14:42:20 +02:00
dd2d206d99 Backend: Remove more globals (#29644)
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-12-15 19:09:04 +01:00
c2cad26ca9 Chore: Disable default golangci-lint filter (#29751)
* Disable default golangci-lint filter

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Chore: Fix linter warnings

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-12-15 09:32:06 +01:00
12661e8a9d Move middleware context handler logic to service (#29605)
* middleware: Move context handler to own service

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

Co-authored-by: Emil Tullsted <sakjur@users.noreply.github.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2020-12-11 11:44:44 +01:00
f021f97c31 DataSourceSettings: Add servername field to DataSource TLS config (#29279)
* DataSourceSettings: Add servername field to DataSource TLS config

A DNS lookup URL can be provided in the DataSource URL field in order
to dynamically load balance between multiple instances of a DataSource.
When using mutual TLS, Golang's TLS config implementation checks that
the certificate's common name (< 1.15) or subject alternative name
(>= 1.15) has the same value as the domain being accessed. If the DNS
entry is dynamically generated for a specific environment, the
certificate cannot be generated with a name matching the dynamic
DNS URL. As such, Golang offers a servername field that can be set to
overwrite what value is used when checking against the certificate's
common name (or subject alternative name). Without this change, Skip
TLS Verify must be set to true in order for the DataSource to work,
removing some of the benefits gained by using mutual TLS.

This commit adds the ability to set Grafana's internal Golang TLS config
servername field from the UI or a provisioned DataSource.

The servername field is optional and the existing behavior is retained if
the field is not set.

Co-authored-by: Dana Pruitt <dpruitt@vmware.com>
Co-authored-by: Jeremy Alvis <jalvis@pivotal.io>

* Update docs with PR review changes

Co-authored-by: Jeremy Alvis <jalvis@pivotal.io>
Co-authored-by: Dana Pruitt <dpruitt@vmware.com>

* Update with additional PR requested changes

* Minor updates based on PR change requests

Co-authored-by: Dana Pruitt <dpruitt@vmware.com>
2020-12-10 16:07:05 +01:00
31d64d9074 Auth: Add SigV4 header allowlist to reduce chances of verification issues (#29650)
* enforce allowlist

* fix default auth selection

* add Host and comment
2020-12-09 10:45:57 +01:00