opensource/opentelemetry-python-contrib
1
0
Fork 0
mirror of https://github.com/open-telemetry/opentelemetry-python-contrib.git synced 2025-08-02 19:47:17 +08:00
Code Issues Packages Projects Releases Wiki Activity

Prometheus Remote Write Exporter (5/6) (#216)

Browse Source
This commit is contained in:
Azfaar Qureshi
2020-12-09 13:46:31 -05:00
committed by GitHub
parent ae70d5abe5
commit 6514f37177
5 changed files with 243 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

23
CHANGELOG.md
View File

@ -12,18 +12,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- `opentelemetry-sdk-extension-aws` Add method to return fields injected by propagator
([#226](https://github.com/open-telemetry/opentelemetry-python/pull/226))
- `opentelemetry-exporter-prometheus-remote-write` Prometheus Remote Write Exporter Setup
((#180)[https://github.com/open-telemetry/opentelemetry-python-contrib/pull/180])
- `opentelemetry-exporter-prometheus-remote-write` Add Exporter constructor validation methods
((#206)[https://github.com/open-telemetry/opentelemetry-python-contrib/pull/206])
- `opentelemetry-exporter-prometheus-remote-write` Add conversion to TimeSeries methods
((#207)[https://github.com/open-telemetry/opentelemetry-python-contrib/pull/207])
- `opentelemetry-exporter-prometheus-remote-write` Add request methods
((#212)[https://github.com/open-telemetry/opentelemetry-python-contrib/pull/212])
([#180](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/180))
- `opentelemetry-exporter-prometheus-remote-write` Add Exporter constructor validation methods in Prometheus Remote Write Exporter
([#206](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/206))
- `opentelemetry-exporter-prometheus-remote-write` Add conversion to TimeSeries methods in Prometheus Remote Write Exporter
([#207](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/207))
- `opentelemetry-exporter-prometheus-remote-write` Add request methods to Prometheus Remote Write Exporter
([#212](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/212))
- `opentelemetry-instrumentation-fastapi` Added support for excluding some routes with env var `OTEL_PYTHON_FASTAPI_EXCLUDED_URLS`
([#237](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/237))
- `opentelemetry-instrumentation-starlette` Added support for excluding some routes with env var `OTEL_PYTHON_STARLETTE_EXCLUDED_URLS`
([#237](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/237))
- Add Prometheus Remote Write Exporter integration tests in opentelemetry-docker-tests
([#216](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/216))
### Changed
- `opentelemetry-instrumentation-asgi`, `opentelemetry-instrumentation-wsgi` Return `None` for `CarrierGetter` if key not found
@ -92,7 +93,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
([#1239](https://github.com/open-telemetry/opentelemetry-python/pull/1239))
- `opentelemetry-instrumentation-django` Bugfix use request.path replace request.get_full_path(). It will get correct span name
([#1309](https://github.com/open-telemetry/opentelemetry-python/pull/1309#))
- `opentelemetry-instrumentation-django` Record span status and http.status_code attribute on exception
- `opentelemetry-instrumentation-django` Record span status and http.status_code attribute on exception
([#1257](https://github.com/open-telemetry/opentelemetry-python/pull/1257))
- `opentelemetry-instrumentation-grpc` Rewrite gRPC server interceptor
([#1171](https://github.com/open-telemetry/opentelemetry-python/pull/1171))
@ -120,7 +121,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- `opentelemetry-instrumentation-pymysql` Bumped version from 0.9.3 to 0.10.1
([#1228](https://github.com/open-telemetry/opentelemetry-python/pull/1228))
- `opentelemetry-instrumentation-django` Changed span name extraction from request to comply semantic convention
([#992](https://github.com/open-telemetry/opentelemetry-python/pull/992))
([#992](https://github.com/open-telemetry/opentelemetry-python/pull/992))
## [0.13b0](https://github.com/open-telemetry/opentelemetry-python/releases/tag/v0.13b0) - 2020-09-17
@ -264,7 +265,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [0.8b0](https://github.com/open-telemetry/opentelemetry-python/releases/tag/v0.8.0) - 2020-05-27
### Added
- `opentelemetry-ext-datadog` Add exporter to Datadog
- `opentelemetry-ext-datadog` Add exporter to Datadog
([#572](https://github.com/open-telemetry/opentelemetry-python/pull/572))
- `opentelemetry-ext-sqlite3` Initial release
- `opentelemetry-ext-psycopg2` Implement instrumentor interface, enabling auto-instrumentation

8
tests/opentelemetry-docker-tests/tests/docker-compose.yml
View File

@ -39,3 +39,11 @@ services:
- "16686:16686"
- "14268:14268"
- "9411:9411"
cortex:
image: quay.io/cortexproject/cortex:v1.5.0
command:
- -config.file=./config/cortex-config.yml
volumes:
- ./prometheus-remote-write-cortex/cortex-config.yml:/config/cortex-config.yml:ro
ports:
- 9009:9009

100
tests/opentelemetry-docker-tests/tests/prometheus-remote-write-cortex/cortex-config.yml Normal file
View File

@ -0,0 +1,100 @@
# This Cortex Config is copied from the Cortex Project documentation
# Source: https://github.com/cortexproject/cortex/blob/master/docs/configuration/single-process-config.yaml
# Configuration for running Cortex in single-process mode.
# This configuration should not be used in production.
# It is only for getting started and development.
# Disable the requirement that every request to Cortex has a
# X-Scope-OrgID header. `fake` will be substituted in instead.
auth_enabled: false
server:
http_listen_port: 9009
# Configure the server to allow messages up to 100MB.
grpc_server_max_recv_msg_size: 104857600
grpc_server_max_send_msg_size: 104857600
grpc_server_max_concurrent_streams: 1000
distributor:
shard_by_all_labels: true
pool:
health_check_ingesters: true
ingester_client:
grpc_client_config:
# Configure the client to allow messages up to 100MB.
max_recv_msg_size: 104857600
max_send_msg_size: 104857600
use_gzip_compression: true
ingester:
# We want our ingesters to flush chunks at the same time to optimise
# deduplication opportunities.
spread_flushes: true
chunk_age_jitter: 0
walconfig:
wal_enabled: true
recover_from_wal: true
wal_dir: /tmp/cortex/wal
lifecycler:
# The address to advertise for this ingester. Will be autodiscovered by
# looking up address on eth0 or en0; can be specified if this fails.
# address: 127.0.0.1
# We want to start immediately and flush on shutdown.
join_after: 0
min_ready_duration: 0s
final_sleep: 0s
num_tokens: 512
tokens_file_path: /tmp/cortex/wal/tokens
# Use an in memory ring store, so we don't need to launch a Consul.
ring:
kvstore:
store: inmemory
replication_factor: 1
# Use local storage - BoltDB for the index, and the filesystem
# for the chunks.
schema:
configs:
- from: 2019-07-29
store: boltdb
object_store: filesystem
schema: v10
index:
prefix: index_
period: 1w
storage:
boltdb:
directory: /tmp/cortex/index
filesystem:
directory: /tmp/cortex/chunks
delete_store:
store: boltdb
purger:
object_store_type: filesystem
frontend_worker:
# Configure the frontend worker in the querier to match worker count
# to max_concurrent on the queriers.
match_max_concurrent: true
# Configure the ruler to scan the /tmp/cortex/rules directory for prometheus
# rules: https://prometheus.io/docs/prometheus/latest/configuration/recording_rules/#recording-rules
ruler:
enable_api: true
enable_sharding: false
storage:
type: local
local:
directory: /tmp/cortex/rules

116
tests/opentelemetry-docker-tests/tests/prometheus-remote-write-cortex/prometheus_remote_write_cortex_test.py Normal file
View File

@ -0,0 +1,116 @@
# Copyright The OpenTelemetry Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from opentelemetry import metrics
from opentelemetry.exporter.prometheus_remote_write import (
PrometheusRemoteWriteMetricsExporter,
)
from opentelemetry.test.test_base import TestBase
def observer_callback(observer):
array = [1.0, 15.0, 25.0, 26.0]
for (index, usage) in enumerate(array):
labels = {"test_label": str(index)}
observer.observe(usage, labels)
class TestPrometheusRemoteWriteExporterCortex(TestBase):
def setUp(self):
super().setUp
self.exporter = PrometheusRemoteWriteMetricsExporter(
endpoint="http://localhost:9009/api/prom/push",
headers={"X-Scope-Org-ID": "5"},
)
self.labels = {"environment": "testing"}
self.meter = self.meter_provider.get_meter(__name__)
metrics.get_meter_provider().start_pipeline(
self.meter, self.exporter, 1,
)
def test_export_counter(self):
try:
requests_counter = self.meter.create_counter(
name="counter",
description="test_export_counter",
unit="1",
value_type=int,
)
requests_counter.add(25, self.labels)
except Exception as e:
self.fail(
"Export counter failed with unexpected error {}".format(e)
)
def test_export_valuerecorder(self):
try:
requests_size = self.meter.create_valuerecorder(
name="valuerecorder",
description="test_export_valuerecorder",
unit="1",
value_type=int,
)
requests_size.record(25, self.labels)
except Exception as e:
self.fail(
"Export valuerecorder failed with unexpected error {}".format(
e
)
)
def test_export_updowncounter(self):
try:
requests_size = self.meter.create_updowncounter(
name="updowncounter",
description="test_export_updowncounter",
unit="1",
value_type=int,
)
requests_size.add(-25, self.labels)
except Exception as e:
self.fail(
"Export updowncounter failed with unexpected error {}".format(
e
)
)
def test_export_sumobserver(self):
try:
self.meter.register_sumobserver(
callback=observer_callback,
name="sumobserver",
description="test_export_sumobserver",
unit="1",
value_type=float,
)
except Exception as e:
self.fail(
"Export sumobserver failed with unexpected error {}".format(e)
)
def test_export_updownsumobserver(self):
try:
self.meter.register_updownsumobserver(
callback=observer_callback,
name="updownsumobserver",
description="test_export_updownsumobserver",
unit="1",
value_type=float,
)
except Exception as e:
self.fail(
"Export updownsumobserver failed with unexpected error {}".format(
e
)
)

10
tox.ini
View File

@ -198,7 +198,7 @@ commands_pre =
test: pip install {toxinidir}/opentelemetry-python-core/opentelemetry-api {toxinidir}/opentelemetry-python-core/opentelemetry-sdk {toxinidir}/opentelemetry-python-core/tests/util
test: pip install {toxinidir}/opentelemetry-python-core/opentelemetry-instrumentation
celery: pip install {toxinidir}/instrumentation/opentelemetry-instrumentation-celery[test]
grpc: pip install {toxinidir}/instrumentation/opentelemetry-instrumentation-grpc[test]
@ -341,7 +341,8 @@ deps =
sqlalchemy ~= 1.3.16
redis ~= 3.3.11
celery ~= 4.0, != 4.4.4
protobuf>=3.13.0
requests==2.25.0
changedir =
tests/opentelemetry-docker-tests/tests
@ -361,7 +362,10 @@ commands_pre =
-e {toxinidir}/instrumentation/opentelemetry-instrumentation-aiopg \
-e {toxinidir}/instrumentation/opentelemetry-instrumentation-redis \
-e {toxinidir}/instrumentation/opentelemetry-instrumentation-system-metrics \
-e {toxinidir}/opentelemetry-python-core/exporter/opentelemetry-exporter-opencensus
-e {toxinidir}/opentelemetry-python-core/exporter/opentelemetry-exporter-opencensus \
-e {toxinidir}/exporter/opentelemetry-exporter-prometheus-remote-write
sudo apt-get install libsnappy-dev
pip install python-snappy
docker-compose up -d
python check_availability.py
commands =