opensource/opentelemetry-python-contrib
1
0
Fork 0
mirror of https://github.com/open-telemetry/opentelemetry-python-contrib.git synced 2025-08-02 11:31:52 +08:00
Code Issues Packages Projects Releases Wiki Activity

Add support for regular expression matching and sanitizing of headers in Django. (#1411)

Browse Source
This commit is contained in:
Dan Rogers
2022-11-01 23:54:33 -04:00
committed by GitHub
parent f58d16b47f
commit a8cf644960
5 changed files with 129 additions and 44 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
CHANGELOG.md
View File

@ -29,6 +29,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
([#1402](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1402))
- Add support for py3.11
([#1415](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1415))
- `opentelemetry-instrumentation-django` Add support for regular expression matching and sanitization of HTTP headers.
([#1411](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1411))
- `opentelemetry-instrumentation-falcon` Add support for regular expression matching and sanitization of HTTP headers.
([#1412](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1412))
- `opentelemetry-instrumentation-flask` Add support for regular expression matching and sanitization of HTTP headers.

98
instrumentation/opentelemetry-instrumentation-django/src/opentelemetry/instrumentation/django/__init__.py
View File

@ -94,8 +94,9 @@ Configuration
Exclude lists
*************
To exclude certain URLs from being tracked, set the environment variable ``OTEL_PYTHON_DJANGO_EXCLUDED_URLS``
(or ``OTEL_PYTHON_EXCLUDED_URLS`` as fallback) with comma delimited regexes representing which URLs to exclude.
To exclude certain URLs from tracking, set the environment variable ``OTEL_PYTHON_DJANGO_EXCLUDED_URLS``
(or ``OTEL_PYTHON_EXCLUDED_URLS`` to cover all instrumentations) to a string of comma delimited regexes that match the
URLs.
For example,
@ -107,8 +108,8 @@ will exclude requests such as ``https://site/client/123/info`` and ``https://sit
Request attributes
********************
To extract certain attributes from Django's request object and use them as span attributes, set the environment variable ``OTEL_PYTHON_DJANGO_TRACED_REQUEST_ATTRS`` to a comma
delimited list of request attribute names.
To extract attributes from Django's request object and use them as span attributes, set the environment variable
``OTEL_PYTHON_DJANGO_TRACED_REQUEST_ATTRS`` to a comma delimited list of request attribute names.
For example,
@ -116,14 +117,15 @@ For example,
export OTEL_PYTHON_DJANGO_TRACED_REQUEST_ATTRS='path_info,content_type'
will extract path_info and content_type attributes from every traced request and add them as span attritbues.
will extract the ``path_info`` and ``content_type`` attributes from every traced request and add them as span attributes.
Django Request object reference: https://docs.djangoproject.com/en/3.1/ref/request-response/#attributes
Request and Response hooks
***************************
The instrumentation supports specifying request and response hooks. These are functions that get called back by the instrumentation right after a Span is created for a request
and right before the span is finished while processing a response. The hooks can be configured as follows:
This instrumentation supports request and response hooks. These are functions that get called
right after a span is created for a request and right before the span is finished for the response.
The hooks can be configured as follows:
.. code:: python
@ -140,50 +142,94 @@ Django Response object: https://docs.djangoproject.com/en/3.1/ref/request-respon
Capture HTTP request and response headers
*****************************************
You can configure the agent to capture predefined HTTP headers as span attributes, according to the `semantic convention <https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/semantic_conventions/http.md#http-request-and-response-headers>`_.
You can configure the agent to capture specified HTTP headers as span attributes, according to the
`semantic convention <https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/semantic_conventions/http.md#http-request-and-response-headers>`_.
Request headers
***************
To capture predefined HTTP request headers as span attributes, set the environment variable ``OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST``
to a comma-separated list of HTTP header names.
To capture HTTP request headers as span attributes, set the environment variable
``OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST`` to a comma delimited list of HTTP header names.
For example,
::
export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST="content_type,custom_request_header"
export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST="content-type,custom_request_header"
will extract content_type and custom_request_header from request headers and add them as span attributes.
will extract ``content-type`` and ``custom_request_header`` from the request headers and add them as span attributes.
It is recommended that you should give the correct names of the headers to be captured in the environment variable.
Request header names in django are case insensitive. So, giving header name as ``CUStom_Header`` in environment variable will be able capture header with name ``custom-header``.
Request header names in Django are case-insensitive. So, giving the header name as ``CUStom-Header`` in the environment
variable will capture the header named ``custom-header``.
The name of the added span attribute will follow the format ``http.request.header.<header_name>`` where ``<header_name>`` being the normalized HTTP header name (lowercase, with - characters replaced by _ ).
The value of the attribute will be single item list containing all the header values.
Regular expressions may also be used to match multiple headers that correspond to the given pattern. For example:
::
Example of the added span attribute,
export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST="Accept.*,X-.*"
Would match all request headers that start with ``Accept`` and ``X-``.
To capture all request headers, set ``OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST`` to ``".*"``.
::
export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST=".*"
The name of the added span attribute will follow the format ``http.request.header.<header_name>`` where ``<header_name>``
is the normalized HTTP header name (lowercase, with ``-`` replaced by ``_``). The value of the attribute will be a
single item list containing all the header values.
For example:
``http.request.header.custom_request_header = ["<value1>,<value2>"]``
Response headers
****************
To capture predefined HTTP response headers as span attributes, set the environment variable ``OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE``
to a comma-separated list of HTTP header names.
To capture HTTP response headers as span attributes, set the environment variable
``OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE`` to a comma delimited list of HTTP header names.
For example,
::
export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE="content_type,custom_response_header"
export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE="content-type,custom_response_header"
will extract content_type and custom_response_header from response headers and add them as span attributes.
will extract ``content-type`` and ``custom_response_header`` from the response headers and add them as span attributes.
It is recommended that you should give the correct names of the headers to be captured in the environment variable.
Response header names captured in django are case insensitive. So, giving header name as ``CUStomHeader`` in environment variable will be able capture header with name ``customheader``.
Response header names in Django are case-insensitive. So, giving the header name as ``CUStom-Header`` in the environment
variable will capture the header named ``custom-header``.
The name of the added span attribute will follow the format ``http.response.header.<header_name>`` where ``<header_name>`` being the normalized HTTP header name (lowercase, with - characters replaced by _ ).
The value of the attribute will be single item list containing all the header values.
Regular expressions may also be used to match multiple headers that correspond to the given pattern. For example:
::
Example of the added span attribute,
export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE="Content.*,X-.*"
Would match all response headers that start with ``Content`` and ``X-``.
To capture all response headers, set ``OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE`` to ``".*"``.
::
export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE=".*"
The name of the added span attribute will follow the format ``http.response.header.<header_name>`` where ``<header_name>``
is the normalized HTTP header name (lowercase, with ``-`` replaced by ``_``). The value of the attribute will be a
single item list containing all the header values.
For example:
``http.response.header.custom_response_header = ["<value1>,<value2>"]``
Sanitizing headers
******************
In order to prevent storing sensitive data such as personally identifiable information (PII), session keys, passwords,
etc, set the environment variable ``OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS``
to a comma delimited list of HTTP header names to be sanitized. Regexes may be used, and all header names will be
matched in a case-insensitive manner.
For example,
::
export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS=".*session.*,set-cookie"
will replace the value of headers such as ``session-id`` and ``set-cookie`` with ``[REDACTED]`` in the span.
Note:
The environment variable names used to capture HTTP headers are still experimental, and thus are subject to change.
API
---

33
instrumentation/opentelemetry-instrumentation-django/tests/test_middleware.py
View File

@ -48,6 +48,7 @@ from opentelemetry.trace import (
format_trace_id,
)
from opentelemetry.util.http import (
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS,
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST,
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE,
_active_requests_count_attrs,
@ -530,6 +531,14 @@ class TestMiddlewareWithTracerProvider(WsgiTestBase):
)
@patch.dict(
"os.environ",
{
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS: ".*my-secret.*",
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3,Regex-Test-Header-.*,Regex-Invalid-Test-Header-.*,.*my-secret.*",
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3,my-custom-regex-header-.*,invalid-regex-header-.*,.*my-secret.*",
},
)
class TestMiddlewareWsgiWithCustomHeaders(WsgiTestBase):
@classmethod
def setUpClass(cls):
@ -542,18 +551,9 @@ class TestMiddlewareWsgiWithCustomHeaders(WsgiTestBase):
tracer_provider, exporter = self.create_tracer_provider()
self.exporter = exporter
_django_instrumentor.instrument(tracer_provider=tracer_provider)
self.env_patch = patch.dict(
"os.environ",
{
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3",
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3",
},
)
self.env_patch.start()
def tearDown(self):
super().tearDown()
self.env_patch.stop()
teardown_test_environment()
_django_instrumentor.uninstrument()
@ -570,10 +570,18 @@ class TestMiddlewareWsgiWithCustomHeaders(WsgiTestBase):
"http.request.header.custom_test_header_2": (
"test-header-value-2",
),
"http.request.header.regex_test_header_1": ("Regex Test Value 1",),
"http.request.header.regex_test_header_2": (
"RegexTestValue2,RegexTestValue3",
),
"http.request.header.my_secret_header": ("[REDACTED]",),
}
Client(
HTTP_CUSTOM_TEST_HEADER_1="test-header-value-1",
HTTP_CUSTOM_TEST_HEADER_2="test-header-value-2",
HTTP_REGEX_TEST_HEADER_1="Regex Test Value 1",
HTTP_REGEX_TEST_HEADER_2="RegexTestValue2,RegexTestValue3",
HTTP_MY_SECRET_HEADER="My Secret Value",
).get("/traced/")
spans = self.exporter.get_finished_spans()
self.assertEqual(len(spans), 1)
@ -607,6 +615,13 @@ class TestMiddlewareWsgiWithCustomHeaders(WsgiTestBase):
"http.response.header.custom_test_header_2": (
"test-header-value-2",
),
"http.response.header.my_custom_regex_header_1": (
"my-custom-regex-value-1,my-custom-regex-value-2",
),
"http.response.header.my_custom_regex_header_2": (
"my-custom-regex-value-3,my-custom-regex-value-4",
),
"http.response.header.my_secret_header": ("[REDACTED]",),
}
Client().get("/traced_custom_header/")
spans = self.exporter.get_finished_spans()

33
instrumentation/opentelemetry-instrumentation-django/tests/test_middleware_asgi.py
View File

@ -43,6 +43,7 @@ from opentelemetry.trace import (
format_trace_id,
)
from opentelemetry.util.http import (
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS,
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST,
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE,
get_excluded_urls,
@ -424,6 +425,14 @@ class TestMiddlewareAsgiWithTracerProvider(SimpleTestCase, TestBase):
)
@patch.dict(
"os.environ",
{
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS: ".*my-secret.*",
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3,Regex-Test-Header-.*,Regex-Invalid-Test-Header-.*,.*my-secret.*",
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3,my-custom-regex-header-.*,invalid-regex-header-.*,.*my-secret.*",
},
)
class TestMiddlewareAsgiWithCustomHeaders(SimpleTestCase, TestBase):
@classmethod
def setUpClass(cls):
@ -437,18 +446,9 @@ class TestMiddlewareAsgiWithCustomHeaders(SimpleTestCase, TestBase):
tracer_provider, exporter = self.create_tracer_provider()
self.exporter = exporter
_django_instrumentor.instrument(tracer_provider=tracer_provider)
self.env_patch = patch.dict(
"os.environ",
{
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3",
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3",
},
)
self.env_patch.start()
def tearDown(self):
super().tearDown()
self.env_patch.stop()
teardown_test_environment()
_django_instrumentor.uninstrument()
@ -465,12 +465,20 @@ class TestMiddlewareAsgiWithCustomHeaders(SimpleTestCase, TestBase):
"http.request.header.custom_test_header_2": (
"test-header-value-2",
),
"http.request.header.regex_test_header_1": ("Regex Test Value 1",),
"http.request.header.regex_test_header_2": (
"RegexTestValue2,RegexTestValue3",
),
"http.request.header.my_secret_header": ("[REDACTED]",),
}
await self.async_client.get(
"/traced/",
**{
"custom-test-header-1": "test-header-value-1",
"custom-test-header-2": "test-header-value-2",
"Regex-Test-Header-1": "Regex Test Value 1",
"regex-test-header-2": "RegexTestValue2,RegexTestValue3",
"My-Secret-Header": "My Secret Value",
},
)
spans = self.exporter.get_finished_spans()
@ -510,6 +518,13 @@ class TestMiddlewareAsgiWithCustomHeaders(SimpleTestCase, TestBase):
"http.response.header.custom_test_header_2": (
"test-header-value-2",
),
"http.response.header.my_custom_regex_header_1": (
"my-custom-regex-value-1,my-custom-regex-value-2",
),
"http.response.header.my_custom_regex_header_2": (
"my-custom-regex-value-3,my-custom-regex-value-4",
),
"http.response.header.my_secret_header": ("[REDACTED]",),
}
await self.async_client.get("/traced_custom_header/")
spans = self.exporter.get_finished_spans()

7
instrumentation/opentelemetry-instrumentation-django/tests/views.py
View File

@ -35,6 +35,13 @@ def response_with_custom_header(request):
response = HttpResponse()
response["custom-test-header-1"] = "test-header-value-1"
response["custom-test-header-2"] = "test-header-value-2"
response[
"my-custom-regex-header-1"
] = "my-custom-regex-value-1,my-custom-regex-value-2"
response[
"my-custom-regex-header-2"
] = "my-custom-regex-value-3,my-custom-regex-value-4"
response["my-secret-header"] = "my-secret-value"
return response