chore(crashtracking): reconfigure after identity refresh by litianningdatadog · Pull Request #19823 · DataDog/dd-trace-py

litianningdatadog · 2026-08-23T21:52:15Z

Stacked PRs:

chore(runtime): add explicit identity refresh #19778
- chore(flask): emit MicroVM request-starting event #19816

Description

Split from #19780.

Crashtracking configuration includes runtime identity. Reconfigure crashtracking after runtime identity refresh so crash reports from a restored MicroVM instance use the refreshed runtime id.

Testing

scripts/lint format_check ddtrace/internal/core/crashtracking.py ddtrace/internal/native/_native.pyi tests/crashtracker/test_crashtracker.py
git diff --check

Stack

Draft split branch. Stacked on #19816.

cit-pr-commenter-54b7da · 2026-08-23T21:52:23Z

Circular import analysis

⚠️ Existing circular imports

There are 3 circular imports that already exist on the base branch and have not been changed by this PR.

ddtrace.errortracking._handled_exceptions.bytecode_injector -> ddtrace.errortracking._handled_exceptions.callbacks -> ddtrace.errortracking._handled_exceptions.collector -> ddtrace.errortracking._handled_exceptions.bytecode_reporting -> ddtrace.errortracking._handled_exceptions.bytecode_injector
ddtrace.llmobs -> ddtrace.llmobs._evaluators -> ddtrace.llmobs._evaluators.format -> ddtrace.llmobs._experiment -> ddtrace.llmobs
ddtrace.appsec._asm_request_context -> ddtrace.appsec._iast._iast_request_context_base -> ddtrace.appsec._iast._iast_env -> ddtrace.appsec._iast.reporter -> ddtrace.appsec._exploit_prevention.stack_traces -> ddtrace.appsec._asm_request_context

cit-pr-commenter-54b7da · 2026-08-23T21:52:27Z

Dependency direction analysis

⚠️ Existing dependency direction violations

There are 250 dependency direction violations that already exist on the base branch and have not been changed by this PR.

Show existing violations (showing 5 of 250 highest severity)

ddtrace.internal.tracemethods -×-> ddtrace.trace  (internal-core -> product:tracing, score=135)
ddtrace.llmobs._utils -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=133)
ddtrace.llmobs._integrations.vllm -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=133)
ddtrace.appsec._contrib.django -×-> ddtrace.trace  (product:appsec -> product:tracing, score=133)
ddtrace.profiling.collector.pytorch -×-> ddtrace.trace  (product:profiling -> product:tracing, score=133)

To see all violations, download the layers-base.json and layers-pr.json artifacts from this CI job and run:
uv run --script scripts/import-analysis/layers.py compare layers-base.json layers-pr.json

cit-pr-commenter-54b7da · 2026-08-23T21:52:47Z

Codeowners resolved as

Resolved from the full PR diff against tianning.li/2-flask-web-request-starting-event using the target branch CODEOWNERS file.
CODEOWNERS team requests not listed below are not required by the current file set.

ddtrace/contrib/_events/web_framework.py                                @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/contrib/internal/flask/patch.py                                 @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/internal/core/crashtracking.py                                  @DataDog/profiling-python @DataDog/apm-core-python
ddtrace/internal/core/event_names.py                                    @DataDog/apm-core-python
ddtrace/internal/native/_native.pyi                                     @DataDog/apm-core-python
src/native/crashtracker.rs                                              @DataDog/profiling-python @DataDog/apm-core-python
src/native/lib.rs                                                       @DataDog/apm-core-python
tests/contrib/flask/test_microvm_identity_refresh.py                    @DataDog/apm-core-python @DataDog/apm-idm-python
tests/crashtracker/test_crashtracker.py                                 @DataDog/profiling-python

datadog-datadog-prod-us1-2 · 2026-08-23T22:00:44Z

⚠️ Warnings

❌ Your PR has failed checks. Please review the issues below and take necessary action before merging.

🚦 5 Pipeline jobs failed

DataDog/apm-reliability/dd-trace-py | build linux serverless: [arm64, cp315-cp315, v113741357-d2b8243-manylinux2014_aarch64, 1] — 🔧 Needs a code fix, caused by this PR

View more details · View in GitLab

DataDog/apm-reliability/dd-trace-py | build linux: [amd64, cp315-cp315, v113741238-d2b8243-manylinux2014_x86_64] — 🔧 Needs a code fix, caused by this PR

View more details · View in GitLab

DataDog/apm-reliability/dd-trace-py | build linux serverless: [amd64, cp315-cp315, v113741238-d2b8243-manylinux2014_x86_64, 1]

View more details · View in GitLab

View all 5 failed jobs.

🧪 1 Test failed in 1 job

DataDog/apm-reliability/dd-trace-py | contrib/kafka 3/4 — ❌ 1 test failed

View more details

❌ test_service_override_env_var[py3.11] from test_kafka.py

At request &lt;Request GET /test/session/snapshot &gt;:
   At snapshot (token=&#39;tests.contrib.kafka.test_kafka.test_service_override_env_var&#39;):
    - Directory: /go/src/github.com/DataDog/apm-reliability/dd-trace-py/tests/snapshots
    - CI mode: 1
    - Trace File: /go/src/github.com/DataDog/apm-reliability/dd-trace-py/tests/snapshots/tests.contrib.kafka.test_kafka.test_service_override_env_var.json
    - Stats File: /go/src/github.com/DataDog/apm-reliability/dd-trace-py/tests/snapshots/tests.contrib.kafka.test_kafka.test_service_override_env_var_tracestats.json
    At compare of 2 expected trace(s) to 2 received trace(s):
     At trace &#39;kafka.consume&#39; (1 spans):
      At snapshot compare of span &#39;kafka.consume&#39; at position 1 in trace:
       - Expected span:
...

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

🔄 Datadog retried 9 tests - 8 passed on retry

🔄 Datadog auto-retried 5 jobs - 5 passed on retry

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 6e3841d | Docs | View more details | Give us feedback!}

pr-commenter · 2026-08-23T22:26:59Z

Benchmarks

Benchmark execution time: 2026-08-25 18:04:10

Comparing candidate commit 6e3841d in PR branch tianning.li/3-6-crashtracking-identity-refresh with baseline commit 3bb9ecd in branch main.

📊 Benchmarking dashboard

Found 0 performance improvements and 10 performance regressions! Performance is the same for 608 metrics, 10 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

🟩 = significantly better candidate vs. baseline
🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:httppropagationextract-empty_headers

🟥 execution_time [+74.858ns; +90.109ns] or [+7.663%; +9.224%]

scenario:httppropagationinject-ids_only

🟥 execution_time [+1.833µs; +2.002µs] or [+9.658%; +10.549%]

scenario:iast_aspects-re_sub_aspect

🟥 execution_time [+26.860µs; +32.622µs] or [+7.176%; +8.715%]

scenario:iastaspects-add_aspect

🟥 execution_time [+12.109µs; +14.480µs] or [+12.279%; +14.683%]

scenario:iastaspects-translate_aspect

🟥 execution_time [+95.441µs; +108.356µs] or [+22.335%; +25.357%]

scenario:iastaspectsospath-ospathbasename_aspect

🟥 execution_time [+146.087µs; +154.987µs] or [+37.641%; +39.934%]

scenario:iastaspectssplit-rsplit_aspect

🟥 execution_time [+44.552µs; +50.497µs] or [+32.961%; +37.359%]

scenario:span-start

🟥 execution_time [+1.257ms; +1.454ms] or [+8.552%; +9.892%]

scenario:telemetryaddmetric-1-count-metric-1-times

🟥 execution_time [+206.297ns; +265.175ns] or [+7.577%; +9.739%]

scenario:tracer-small

🟥 execution_time [+28.752µs; +30.651µs] or [+8.579%; +9.146%]

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:coreapiscenario-context_with_data_listeners

unstable execution_time [-672.591ns; +817.056ns] or [-6.074%; +7.378%]

scenario:coreapiscenario-core_dispatch_1_listener

unstable execution_time [-29.029ns; +37.820ns] or [-4.754%; +6.194%]

scenario:coreapiscenario-core_dispatch_50_listeners

unstable execution_time [-1757.444ns; +1575.300ns] or [-10.241%; +9.180%]

scenario:coreapiscenario-core_dispatch_exception_listeners

unstable execution_time [-1249.545ns; +1224.189ns] or [-9.622%; +9.427%]

scenario:coreapiscenario-core_dispatch_listeners

unstable execution_time [-310.551ns; +343.458ns] or [-8.441%; +9.335%]

scenario:coreapiscenario-core_dispatch_no_args_listeners

unstable execution_time [-220.002ns; +289.666ns] or [-7.551%; +9.942%]

scenario:coreapiscenario-core_dispatch_with_results_1_listener

unstable execution_time [-45.042ns; +103.490ns] or [-3.955%; +9.086%]

scenario:coreapiscenario-core_dispatch_with_results_50_listeners

unstable execution_time [-4239.166ns; +3621.324ns] or [-10.457%; +8.933%]

scenario:coreapiscenario-core_dispatch_with_results_listeners

unstable execution_time [-808.329ns; +770.274ns] or [-9.989%; +9.519%]

scenario:packagesupdateimporteddependencies-import_many_stdlib_cached

unstable execution_time [-63.583µs; +58.189µs] or [-9.863%; +9.027%]

Define web request event names in ddtrace.internal.core.event_names so integrations and lower-level runtime hooks can depend on the same event strings without duplicating literals. The MicroVM runtime hook needs WEB_REQUEST_STARTING during ddtrace bootstrap. Importing ddtrace.contrib from that path can load trace handlers that import top-level ddtrace.config before __init__ has exported it, producing a circular import in MicroVM images. Keeping the names in a core constants-only module gives both contrib and runtime a safe dependency point.

Copilot

Pull request overview

This PR ensures crashtracking metadata stays in sync with refreshed runtime identity (e.g., after MicroVM restore) by adding a native “reconfigure” entrypoint and wiring crashtracking to re-run configuration when identity changes.

Changes:

Add a new native binding crashtracker_reconfigure and expose it to Python.
Register an identity-change callback in ddtrace.internal.core.crashtracking to rebuild crashtracker args/metadata on identity refresh.
Add a subprocess test asserting reconfiguration happens after runtime.refresh_identity().

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
`ddtrace/internal/core/crashtracking.py`	Registers runtime-id change callback and triggers crashtracker reconfiguration with refreshed metadata.
`ddtrace/internal/native/_native.pyi`	Adds typing stub for the new `crashtracker_reconfigure` native function.
`src/native/lib.rs`	Exposes `crashtracker_reconfigure` from the Rust extension module.
`src/native/crashtracker.rs`	Implements the new `crashtracker_reconfigure` pyfunction forwarding into `libdd_crashtracker`.
`tests/crashtracker/test_crashtracker.py`	Adds regression test validating identity refresh triggers crashtracker reconfiguration.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+def _reconfigure_for_identity_refresh(additional_tags: Optional[dict[str, str]]) -> None:
+    if not is_started():
+        return
+
+    config, receiver_config, metadata = _get_args(additional_tags)
+    if config is None or receiver_config is None or metadata is None:
+        log.error("Failed to reconfigure crashtracker after identity refresh: failed to construct configuration")
+        return
+    crashtracker_reconfigure(config, receiver_config, metadata)
+


            crashtracker_on_fork(config, receiver_config, metadata)

        forksafe.register(crashtracker_fork_handler)
+        _identity_refresh_additional_tags = additional_tags



        forksafe.register(crashtracker_fork_handler)
+        _identity_refresh_additional_tags = additional_tags
+        on_runtime_id_change(_on_identity_refresh)


litianningdatadog · 2026-08-31T18:32:12Z

close it as it is out of the scope

litianningdatadog added changelog/no-changelog A changelog entry is not required for this PR. aws-microvm Work related to AWS MicroVM onboarding labels Aug 23, 2026

litianningdatadog changed the title ~~fix(crashtracking): reconfigure after identity refresh~~ chore(crashtracking): reconfigure after identity refresh Aug 23, 2026

litianningdatadog force-pushed the tianning.li/2-flask-web-request-starting-event branch from d59e112 to 16a5332 Compare August 24, 2026 02:23

litianningdatadog force-pushed the tianning.li/3-6-crashtracking-identity-refresh branch from 1176485 to 96bb152 Compare August 24, 2026 02:24

litianningdatadog force-pushed the tianning.li/2-flask-web-request-starting-event branch from 16a5332 to 8dd7e8e Compare August 24, 2026 02:31

litianningdatadog force-pushed the tianning.li/3-6-crashtracking-identity-refresh branch from 96bb152 to c760e6d Compare August 24, 2026 02:31

litianningdatadog force-pushed the tianning.li/2-flask-web-request-starting-event branch 5 times, most recently from cde3045 to a0e3c42 Compare August 24, 2026 23:56

litianningdatadog force-pushed the tianning.li/3-6-crashtracking-identity-refresh branch from c760e6d to 426f6b5 Compare August 25, 2026 13:23

litianningdatadog requested a lite review from Copilot August 25, 2026 13:36

Copilot started reviewing on behalf of litianningdatadog August 25, 2026 13:36 View session

Copilot AI reviewed Aug 25, 2026

View reviewed changes

fix(crashtracking): reconfigure after identity refresh

6e3841d

litianningdatadog force-pushed the tianning.li/3-6-crashtracking-identity-refresh branch from 426f6b5 to 6e3841d Compare August 25, 2026 17:35

litianningdatadog mentioned this pull request Aug 27, 2026

chore(serverless): publish web request-start event #19898

Open

litianningdatadog closed this Aug 31, 2026

Navigation Menu

chore(crashtracking): reconfigure after identity refresh - #19823

chore(crashtracking): reconfigure after identity refresh#19823
litianningdatadog wants to merge 2 commits into
tianning.li/2-flask-web-request-starting-eventfrom
tianning.li/3-6-crashtracking-identity-refresh

litianningdatadog commented Aug 23, 2026 •

edited

Loading

Uh oh!

cit-pr-commenter-54b7da Bot commented Aug 23, 2026

Uh oh!

cit-pr-commenter-54b7da Bot commented Aug 23, 2026 •

edited

Loading

Uh oh!

cit-pr-commenter-54b7da Bot commented Aug 23, 2026 •

edited

Loading

Uh oh!

datadog-datadog-prod-us1-2 Bot commented Aug 23, 2026 •

edited by datadog-datadog-prod-us1 Bot

Loading

Uh oh!

pr-commenter Bot commented Aug 23, 2026 •

edited

Loading

Explanation

More details about the CI and significant changes

Unstable benchmarks

scenario:coreapiscenario-context_with_data_listeners

scenario:coreapiscenario-core_dispatch_1_listener

scenario:coreapiscenario-core_dispatch_50_listeners

scenario:coreapiscenario-core_dispatch_exception_listeners

scenario:coreapiscenario-core_dispatch_listeners

scenario:coreapiscenario-core_dispatch_no_args_listeners

scenario:coreapiscenario-core_dispatch_with_results_1_listener

scenario:coreapiscenario-core_dispatch_with_results_50_listeners

scenario:coreapiscenario-core_dispatch_with_results_listeners

scenario:packagesupdateimporteddependencies-import_many_stdlib_cached

Uh oh!

Copilot AI left a comment

Uh oh!

litianningdatadog commented Aug 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

litianningdatadog commented Aug 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Testing

Stack

Uh oh!

cit-pr-commenter-54b7da Bot commented Aug 23, 2026

Circular import analysis

⚠️ Existing circular imports

Uh oh!

cit-pr-commenter-54b7da Bot commented Aug 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Dependency direction analysis

⚠️ Existing dependency direction violations

Uh oh!

cit-pr-commenter-54b7da Bot commented Aug 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codeowners resolved as

Uh oh!

datadog-datadog-prod-us1-2 Bot commented Aug 23, 2026 • edited by datadog-datadog-prod-us1 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Warnings

ℹ️ Info

Uh oh!

pr-commenter Bot commented Aug 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Explanation

More details about the CI and significant changes

scenario:httppropagationextract-empty_headers

scenario:httppropagationinject-ids_only

scenario:iast_aspects-re_sub_aspect

scenario:iastaspects-add_aspect

scenario:iastaspects-translate_aspect

scenario:iastaspectsospath-ospathbasename_aspect

scenario:iastaspectssplit-rsplit_aspect

scenario:span-start

scenario:telemetryaddmetric-1-count-metric-1-times

scenario:tracer-small

Unstable benchmarks

scenario:coreapiscenario-context_with_data_listeners

scenario:coreapiscenario-core_dispatch_1_listener

scenario:coreapiscenario-core_dispatch_50_listeners

scenario:coreapiscenario-core_dispatch_exception_listeners

scenario:coreapiscenario-core_dispatch_listeners

scenario:coreapiscenario-core_dispatch_no_args_listeners

scenario:coreapiscenario-core_dispatch_with_results_1_listener

scenario:coreapiscenario-core_dispatch_with_results_50_listeners

scenario:coreapiscenario-core_dispatch_with_results_listeners

scenario:packagesupdateimporteddependencies-import_many_stdlib_cached

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

litianningdatadog commented Aug 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

litianningdatadog commented Aug 23, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Aug 23, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Aug 23, 2026 •

edited

Loading

datadog-datadog-prod-us1-2 Bot commented Aug 23, 2026 •

edited by datadog-datadog-prod-us1 Bot

Loading

pr-commenter Bot commented Aug 23, 2026 •

edited

Loading