Skip to content

[Feature Flags] Add privacy-protected flagevaluation EVP events through the Agent - #9351

Draft
vjfridge wants to merge 14 commits into
masterfrom
vickie/ffe-evp-pii-dotnet
Draft

[Feature Flags] Add privacy-protected flagevaluation EVP events through the Agent#9351
vjfridge wants to merge 14 commits into
masterfrom
vickie/ffe-evp-pii-dotnet

Conversation

@vjfridge

@vjfridge vjfridge commented Oct 1, 2026 •
edited by atlassian Bot
Loading

Copy link
Copy Markdown

Summary of changes

  • Adds aggregated flagevaluation EVP events for instrumented Datadog OpenFeature evaluations, including unsuccessful evaluations. These events show how flags are evaluated without sending raw targeting keys or evaluation context by default.
  • Ships Agent-only event delivery. Configuration can still come from Remote Config or the existing Agentless configuration source; event delivery always goes through the configured Agent.
  • Targets master independently of #9235. Replaces the approach in the stale #9139 draft; it does not import or require Leo’s transport work.

Reason for change

Provide evaluation visibility while protecting customer data by default and keeping telemetry failures separate from flag evaluation behavior. Follows the merged implementations in Go, Java, Python, and Ruby, with PII guidance and the JS implementation as additional references.

Implementation details

Key decisions / assumptions

  • Consent belongs to the evaluation. Capture the configuration’s observeFullEvaluationData value and evaluation timestamp when evaluating the flag. Later configuration changes must not change that observation’s privacy treatment. Only boolean true opts into full data.
  • Protect output independently. Protected events omit context and hash non-empty, valid targeting keys when building output. Full-consent events may include the raw key and a bounded context snapshot. Invalid targeting keys are omitted, not used to discard the observation. Errors use an allowlist of stable codes, never raw messages.
  • Keep expensive processing off the evaluation call. The caller captures metadata and, only with consent, bounded context, then attempts queue admission without waiting for room. A dedicated worker aggregates, serializes, compresses, and sends. It sleeps when idle, flushes every 10 seconds or on explicit flush, and has a bounded shutdown wait. Caller-side capture and synchronization still have a measurable cost.
  • Bound retained entries, not total process memory. Queue: 16,384 observations; full aggregation: 10,000 buckets per flag / 131,072 globally; degraded aggregation: 32,768 buckets. Degraded events combine observations without targeting key or context; consent is not part of their identity. Once capacity is exhausted, telemetry can be dropped without changing evaluation results. These are not byte budgets or an RSS guarantee.
  • Bound consented context work. Capture at most 256 fields with depth, text, container, and total-visit limits. We do not scan/sort an arbitrarily large input just to choose fields. Rare hash collisions in rebuilt over-limit maps can change the retained subset and split telemetry buckets; flag results and protected-mode events are unaffected.
  • Keep transport scoped. Send gzip JSON through the Agent’s existing EVP v2 route. No direct-intake fallback, route discovery, or retries. Missing/incompatible Agents and failed sends can lose telemetry, but must not break evaluations. Agent 7.84.0 was verified to preserve SDK origin/version headers; 7.83.1 stripped those headers despite delivering events.

Measured performance overhead

These are local engineering measurements, not final-head measurements or a claim of performance equivalence. Baseline: 2514c73b48; measured implementation: 4de2e6c67c (same production code in 3cef2ec0d2). They predate the two review fixes and latest-master integration.

Native ARM64, Linux/musl, .NET 10.0.12; real OpenFeature/native-instrumentation path and a responsive local Agent fixture. Each burst run measured 100,000 evaluations, 64 subjects and eight attributes after warm-up. Values are medians of three serial runs. CPUs mean affinity slots, not container quotas. Caller latency excludes final flush; CPU and allocation include all process threads and final flush.

CPUs Mode Evaluation p50 / p95 (µs) CPU (µs/evaluation) Allocation (bytes/evaluation)
1 Baseline 46.12 / 56.25 47.03 6,088
1 Events disabled 53.05 / 75.64 57.05 6,584
1 Default protected 65.94 / 105.22 75.11 6,837
1 Full consent 71.98 / 134.29 89.92 11,472
2 Baseline 10.72 / 11.24 11.79 6,088
2 Events disabled 11.82 / 12.40 12.96 6,584
2 Default protected 22.53 / 26.23 30.64 6,836
2 Full consent 27.17 / 30.05 41.71 11,470
  • Protected-mode median latency increased by about 20 µs on one CPU / 12 µs on two CPUs, with process CPU per evaluation up approximately 60% / 160%. Disabling events avoids the writer, but not all added provider/metadata work: it is not a zero-overhead path.
  • Protected burst peak RSS was 142.10 vs 133.38 MiB baseline on one CPU and 136.50 vs 135.30 MiB on two CPUs. These are whole-process observations, not queue-only memory or worst-case bounds; allocation above measures garbage produced, not retained memory.
  • Paced (~1,000/s before processing cost), sparse (~100/s), high-cardinality and oversized-context cases were also exercised. These are test workloads, not customer averages. High-cardinality enabled runs reached roughly 167–185 MiB peak RSS and 134–638 ms final flush times; full-consent oversized contexts add substantial allocation despite bounded capture.
  • Profiling was followed by three worker-signaling experiments. None established a broad win across one/two CPUs, so none is retained; the original AutoResetEvent design stays. Provider hook-list reuse is a separate general optimization tracked in FFL-3356.

Test coverage

  • Merged tree, committed as 7160ad3b03: 788 targeted tracer tests + 74 provider tests passed; one platform-specific skip. All four tracer and four provider framework builds, plus master’s startup-hook project, passed without warnings/errors. These checks ran on the resolved tree before its signed merge commit.
  • Pre-merge 02b6a03698: full tracer suite 13,390 passed / 70 skipped / zero failed; regression coverage for worker request-context retention and metadata-only error codes.
  • External validation at 3cef2ec0d2: system-tests 43 passed / zero failed / eight existing XPASS / 53 deselected, including all 12 Agent EVP cases. Identical baseline harness had 31 passes and 12 expected failures for absent emission. Direct-routing cases were outside scope, not reported as passing.
  • Dogfooding at 3cef2ec0d2, staging dev, Agent 7.84.0: baseline/candidate results matched for 81 live and 24 controlled requests, plus six no-Agent checks. Verified emitted observations, SDK identity, strict consent, and unchanged evaluation behavior.
  • Performance correctness: all 3,237,402 expected enabled observations across 88 trials were audited, including protected payloads and overflow degradation. This proves delivery to the responsive test fixture, not guaranteed delivery under failures.

Reviewer QA

  • With the updated provider and tracer plus a configured Agent, compare flag values with emission on/off. Default/missing/false/malformed consent must omit context and protect targeting keys; boolean true may include bounded full data.
  • Set DD_FLAGGING_EVALUATION_COUNTS_ENABLED=false, or disconnect the Agent: evaluations must still work. Disabled emission must produce no new flagevaluation events. Check that existing exposure/span-enrichment behavior remains unchanged.

Other details

Blast radius

  • Changes the provider/tracer evaluation metadata bridge and adds a hook, background writer, Agent requests, and diagnostic metrics. Complete functionality requires both updated components; mixed-version and uninstrumented-provider behavior has dedicated tests.
  • Events are enabled by default when feature flagging is active; DD_FLAGGING_EVALUATION_COUNTS_ENABLED=false disables them. Existing feature-flag enable/source settings still apply.
  • Does not change targeting rules, returned flag values, existing exposure events, or configuration-fetching transport. No dependency on [Feature Flags] Add agentless EVP fallback transport #9235 and no new direct-intake credentials/path.
  • Adds CPU, allocation, a background worker, and best-effort shutdown work. Consented snapshots intentionally retain more customer data than protected mode. Count limits, snapshot limits and payload splitting reduce resource risk but do not guarantee a fixed memory ceiling.

Remaining before ready for review / merge

  • Review this description and release-note wording.
  • Select provider/tracer release versions through the normal release process.
  • Sign and commit the resolved latest-master merge (7160ad3b03), then push. No PR merge will be performed by the agent.
  • Rerun final-head Agent-only system-tests/dogfooding, publish concise Note to self results, and replace historical evidence above with links to those comments. Publish/coordinate the separately signed test-harness changes needed for reproducibility.
  • Refresh the final-artifact performance summary and have the .NET reviewers assess the residual overhead; successful correctness tests do not establish acceptable performance.
  • Pass relevant framework/platform CI and address any new high-priority review findings. The previous whole-branch review’s two P2 findings are fixed.

@dd-trace-dotnet-ci-bot

dd-trace-dotnet-ci-bot Bot commented Oct 1, 2026 •
edited
Loading

Copy link
Copy Markdown

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing This PR (9351) and master.

✅ No regressions detected

📄 View the full report (charts + all metrics) →

@pr-commenter

pr-commenter Bot commented Oct 1, 2026 •
edited
Loading

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-10-02 01:55:47

Comparing candidate commit 7584f5a in PR branch vickie/ffe-evp-pii-dotnet with baseline commit 0610b13 in branch master.

📊 Benchmarking dashboard

Found 0 performance improvements and 10 performance regressions! Performance is the same for 62 metrics, 0 unstable metrics, 75 known flaky benchmarks, 48 flaky benchmarks without significant changes.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery net472

  • 🟥 allocated_mem [+95 bytes; +96 bytes] or [+10.133%; +10.144%]
  • 🟥 throughput [-32428.082op/s; -29392.191op/s] or [-9.133%; -8.278%]

scenario:Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery net6.0

  • 🟥 allocated_mem [+95 bytes; +96 bytes] or [+9.750%; +9.761%]

scenario:Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery netcoreapp3.1

  • 🟥 allocated_mem [+95 bytes; +96 bytes] or [+9.830%; +9.841%]
  • 🟥 throughput [-39149.523op/s; -31208.723op/s] or [-9.677%; -7.714%]

scenario:Benchmarks.Trace.HttpClientBenchmark.SendAsync net472

  • 🟥 allocated_mem [+271 bytes; +272 bytes] or [+9.338%; +9.348%]
  • 🟥 throughput [-11907.666op/s; -11460.400op/s] or [-13.593%; -13.083%]

scenario:Benchmarks.Trace.HttpClientBenchmark.SendAsync net6.0

  • 🟥 allocated_mem [+199 bytes; +200 bytes] or [+9.426%; +9.438%]

scenario:Benchmarks.Trace.HttpClientBenchmark.SendAsync netcoreapp3.1

  • 🟥 allocated_mem [+199 bytes; +200 bytes] or [+7.569%; +7.581%]
  • 🟥 throughput [-13771.775op/s; -12406.518op/s] or [-10.929%; -9.845%]

Known flaky benchmarks

These benchmarks are marked as flaky and will not trigger a failure. Modify FLAKY_BENCHMARKS_REGEX to control which benchmarks are marked as flaky.

scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.ActivityBenchmark.StartSpan_SetAttributes_Sampled net472

  • 🟥 throughput [-13383.480op/s; -10948.852op/s] or [-8.521%; -6.971%]

scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.ActivityBenchmark.StartSpan_UpdateName_Sampled net472

  • 🟥 throughput [-15566.273op/s; -11387.680op/s] or [-8.187%; -5.989%]

scenario:Benchmarks.Trace.ActivityBenchmark.StartStopWithChild net472

  • 🟥 throughput [-8536.978op/s; -7751.756op/s] or [-10.122%; -9.191%]

scenario:Benchmarks.Trace.ActivityBenchmark.StartStopWithChild net6.0

  • 🟥 throughput [-9872.533op/s; -7366.162op/s] or [-8.298%; -6.192%]

scenario:Benchmarks.Trace.ActivityBenchmark.StartStopWithChild netcoreapp3.1

  • 🟥 throughput [-11558.193op/s; -10403.952op/s] or [-11.752%; -10.579%]

scenario:Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces net472

  • 🟥 allocated_mem [+1.541KB; +1.541KB] or [+46.817%; +46.831%]
  • 🟥 execution_time [+309.199ms; +313.130ms] or [+153.435%; +155.386%]
  • 🟥 throughput [-72.838op/s; -65.160op/s] or [-13.105%; -11.724%]

scenario:Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces net6.0

  • 🟥 allocated_mem [+1.012KB; +1.012KB] or [+37.524%; +37.537%]
  • 🟥 execution_time [+376.006ms; +379.238ms] or [+297.068%; +299.621%]
  • 🟩 throughput [+70.390op/s; +75.562op/s] or [+9.281%; +9.963%]

scenario:Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces netcoreapp3.1

  • 🟥 allocated_mem [+1.088KB; +1.088KB] or [+40.343%; +40.355%]
  • 🟥 execution_time [+394.283ms; +398.648ms] or [+348.925%; +352.788%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody net472

  • 🟥 allocated_mem [+4.758KB; +4.758KB] or [+100.159%; +100.174%]
  • 🟥 throughput [-60483.335op/s; -60083.696op/s] or [-47.059%; -46.748%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody net6.0

  • 🟥 allocated_mem [+3.880KB; +3.880KB] or [+82.058%; +82.072%]
  • 🟩 execution_time [-16.327ms; -12.150ms] or [-7.625%; -5.675%]
  • 🟥 throughput [-61895.073op/s; -59133.755op/s] or [-45.180%; -43.164%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody netcoreapp3.1

  • 🟥 allocated_mem [+4.608KB; +4.608KB] or [+99.641%; +99.654%]
  • 🟥 throughput [-48419.317op/s; -46174.114op/s] or [-43.777%; -41.747%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody net472

  • 🟥 allocated_mem [+1.380KB; +1.380KB] or [+111.648%; +111.662%]
  • 🟥 throughput [-300853.284op/s; -297839.575op/s] or [-30.719%; -30.411%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody net6.0

  • 🟥 allocated_mem [+543 bytes; +544 bytes] or [+44.437%; +44.446%]
  • 🟩 execution_time [-26.697ms; -21.851ms] or [-11.906%; -9.745%]
  • 🟥 throughput [-99741.400op/s; -77162.411op/s] or [-10.656%; -8.243%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody netcoreapp3.1

  • 🟥 allocated_mem [+1.344KB; +1.344KB] or [+111.253%; +111.267%]
  • 🟥 throughput [-173093.094op/s; -156805.502op/s] or [-24.870%; -22.530%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorMoreComplexBody net472

  • 🟥 allocated_mem [+3.378KB; +3.378KB] or [+89.003%; +89.017%]
  • 🟥 throughput [-90739.855op/s; -84252.168op/s] or [-61.067%; -56.701%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorMoreComplexBody net6.0

  • 🟥 allocated_mem [+3.336KB; +3.336KB] or [+88.150%; +88.161%]
  • 🟥 throughput [-75010.240op/s; -72135.385op/s] or [-47.728%; -45.899%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorMoreComplexBody netcoreapp3.1

  • 🟥 allocated_mem [+3.264KB; +3.264KB] or [+88.493%; +88.506%]
  • 🟥 throughput [-55964.404op/s; -53349.028op/s] or [-44.583%; -42.499%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody net472

  • 🟥 throughput [-582198.768op/s; -309786.329op/s] or [-17.711%; -9.424%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody net6.0

  • 🟩 throughput [+243565.933op/s; +267940.768op/s] or [+8.122%; +8.934%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody netcoreapp3.1

  • 🟩 execution_time [-18.880ms; -14.550ms] or [-8.703%; -6.707%]

scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeArgs net472

  • 🟩 allocated_mem [-13.759KB; -13.757KB] or [-42.326%; -42.318%]
  • 🟥 execution_time [+298.574ms; +300.198ms] or [+149.187%; +149.999%]
  • 🟩 throughput [+994.560op/s; +1013.043op/s] or [+10.985%; +11.189%]

scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeArgs net6.0

  • 🟩 allocated_mem [-13.722KB; -13.718KB] or [-42.341%; -42.329%]
  • 🟥 execution_time [+298.625ms; +308.192ms] or [+150.597%; +155.422%]
  • 🟩 throughput [+2213.445op/s; +2521.248op/s] or [+16.930%; +19.284%]

scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeArgs netcoreapp3.1

  • 🟩 allocated_mem [-13.722KB; -13.718KB] or [-42.341%; -42.329%]
  • 🟥 execution_time [+300.756ms; +304.279ms] or [+151.497%; +153.272%]
  • 🟩 throughput [+1790.274op/s; +1931.749op/s] or [+17.284%; +18.650%]

scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeLegacyArgs net472

  • 🟥 execution_time [+295.394ms; +296.558ms] or [+145.086%; +145.658%]
  • 🟩 throughput [+532.041op/s; +545.598op/s] or [+14.105%; +14.464%]

scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeLegacyArgs net6.0

  • 🟥 execution_time [+297.986ms; +299.384ms] or [+145.674%; +146.358%]
  • 🟩 throughput [+2687.914op/s; +2761.434op/s] or [+39.050%; +40.118%]

scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeLegacyArgs netcoreapp3.1

  • 🟥 execution_time [+301.324ms; +302.878ms] or [+150.601%; +151.378%]
  • 🟩 throughput [+1386.874op/s; +1409.190op/s] or [+27.528%; +27.971%]

scenario:Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmark net472

  • 🟩 execution_time [-145.766µs; -142.022µs] or [-29.928%; -29.159%]
  • 🟩 throughput [+849.424op/s; +873.206op/s] or [+41.371%; +42.529%]

scenario:Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmark net6.0

  • 🟩 execution_time [-132.654µs; -105.901µs] or [-30.424%; -24.289%]
  • 🟩 throughput [+796.208op/s; +920.282op/s] or [+34.616%; +40.010%]

scenario:Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmark netcoreapp3.1

  • 🟩 execution_time [-142.691µs; -120.725µs] or [-30.572%; -25.866%]
  • 🟩 throughput [+777.729op/s; +859.986op/s] or [+35.902%; +39.699%]

scenario:Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmarkWithAttack net472

  • 🟩 execution_time [-125.421µs; -120.699µs] or [-33.863%; -32.588%]
  • 🟩 throughput [+1316.234op/s; +1374.850op/s] or [+48.747%; +50.918%]

scenario:Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmarkWithAttack net6.0

  • 🟩 execution_time [-96.321µs; -71.746µs] or [-30.750%; -22.905%]
  • 🟩 throughput [+1052.946op/s; +1278.526op/s] or [+32.823%; +39.855%]

scenario:Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmarkWithAttack netcoreapp3.1

  • 🟩 execution_time [-131.823µs; -104.061µs] or [-36.062%; -28.467%]
  • 🟩 throughput [+1177.547op/s; +1403.382op/s] or [+42.257%; +50.362%]

scenario:Benchmarks.Trace.AspNetCoreBenchmark.SendRequest net472

  • 🟥 execution_time [+299.795ms; +300.576ms] or [+149.629%; +150.018%]

scenario:Benchmarks.Trace.AspNetCoreBenchmark.SendRequest net6.0

  • 🟥 execution_time [+415.011ms; +421.650ms] or [+450.927%; +458.140%]

scenario:Benchmarks.Trace.AspNetCoreBenchmark.SendRequest netcoreapp3.1

  • unstable execution_time [+295.669ms; +348.793ms] or [+224.499%; +264.835%]

scenario:Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces net472

  • unstable execution_time [+243.532ms; +309.766ms] or [+111.974%; +142.428%]
  • 🟥 throughput [-540.326op/s; -492.479op/s] or [-48.959%; -44.623%]

scenario:Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces net6.0

  • unstable execution_time [+208.393ms; +341.716ms] or [+88.808%; +145.625%]
  • 🟥 throughput [-674.546op/s; -591.011op/s] or [-44.992%; -39.421%]

scenario:Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces netcoreapp3.1

  • 🟥 execution_time [+330.933ms; +339.166ms] or [+197.936%; +202.861%]
  • 🟥 throughput [-389.988op/s; -352.999op/s] or [-27.154%; -24.579%]

scenario:Benchmarks.Trace.CharSliceBenchmark.OriginalCharSlice net6.0

  • 🟩 execution_time [-175.184µs; -148.513µs] or [-8.874%; -7.523%]
  • 🟩 throughput [+41.644op/s; +49.574op/s] or [+8.221%; +9.786%]

scenario:Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch net472

  • 🟥 allocated_mem [+96 bytes; +97 bytes] or [+12.092%; +12.103%]
  • 🟥 execution_time [+301.772ms; +304.490ms] or [+151.967%; +153.335%]
  • 🟥 throughput [-30820.947op/s; -27410.767op/s] or [-9.917%; -8.820%]

scenario:Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch net6.0

  • 🟥 allocated_mem [+95 bytes; +96 bytes] or [+12.114%; +12.124%]
  • 🟥 execution_time [+300.518ms; +301.938ms] or [+150.590%; +151.302%]

scenario:Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch netcoreapp3.1

  • 🟥 allocated_mem [+95 bytes; +96 bytes] or [+12.114%; +12.124%]
  • 🟥 execution_time [+302.686ms; +305.933ms] or [+152.056%; +153.688%]

scenario:Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync net472

  • 🟥 allocated_mem [+95 bytes; +96 bytes] or [+11.165%; +11.177%]
  • 🟥 execution_time [+300.077ms; +302.168ms] or [+150.688%; +151.739%]

scenario:Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync net6.0

  • 🟥 allocated_mem [+95 bytes; +96 bytes] or [+12.493%; +12.506%]
  • 🟥 execution_time [+297.843ms; +315.332ms] or [+147.270%; +155.918%]
  • 🟥 throughput [-61008.856op/s; -45038.764op/s] or [-9.830%; -7.257%]

scenario:Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync netcoreapp3.1

  • 🟥 allocated_mem [+95 bytes; +96 bytes] or [+11.422%; +11.434%]
  • 🟥 execution_time [+303.191ms; +307.979ms] or [+153.671%; +156.097%]
  • 🟥 throughput [-32785.682op/s; -24081.050op/s] or [-7.080%; -5.200%]

scenario:Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync net472

  • 🟥 allocated_mem [+96 bytes; +97 bytes] or [+11.197%; +11.208%]
  • 🟥 execution_time [+301.269ms; +303.529ms] or [+151.210%; +152.344%]
  • 🟥 throughput [-46931.987op/s; -41931.116op/s] or [-12.175%; -10.878%]

scenario:Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync net6.0

  • 🟥 allocated_mem [+95 bytes; +96 bytes] or [+10.618%; +10.631%]
  • 🟥 execution_time [+300.682ms; +303.921ms] or [+149.863%; +151.477%]
  • 🟥 throughput [-31531.941op/s; -25595.918op/s] or [-6.261%; -5.083%]

scenario:Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync netcoreapp3.1

  • 🟥 allocated_mem [+95 bytes; +96 bytes] or [+10.618%; +10.631%]
  • 🟥 execution_time [+299.261ms; +302.660ms] or [+148.880%; +150.571%]
  • 🟥 throughput [-29532.889op/s; -23887.883op/s] or [-6.991%; -5.654%]

scenario:Benchmarks.Trace.ILoggerBenchmark.EnrichedLog net472

  • 🟥 allocated_mem [+96 bytes; +97 bytes] or [+6.002%; +6.011%]
  • 🟥 throughput [-22188.259op/s; -19549.436op/s] or [-8.922%; -7.861%]

scenario:Benchmarks.Trace.ILoggerBenchmark.EnrichedLog net6.0

  • 🟥 allocated_mem [+95 bytes; +96 bytes] or [+5.627%; +5.636%]
  • 🟩 execution_time [-16.744ms; -13.083ms] or [-7.786%; -6.084%]

scenario:Benchmarks.Trace.ILoggerBenchmark.EnrichedLog netcoreapp3.1

  • 🟥 allocated_mem [+95 bytes; +96 bytes] or [+5.605%; +5.617%]

scenario:Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark net6.0

  • 🟩 allocated_mem [-20.337KB; -20.313KB] or [-7.419%; -7.410%]
  • unstable execution_time [-28.181µs; +30.944µs] or [-5.570%; +6.116%]
  • unstable throughput [-107.797op/s; +99.027op/s] or [-5.379%; +4.941%]

scenario:Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark netcoreapp3.1

  • unstable execution_time [-83.365µs; -20.900µs] or [-14.447%; -3.622%]
  • unstable throughput [+89.482op/s; +270.692op/s] or [+5.112%; +15.465%]

scenario:Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark net6.0

  • 🟥 execution_time [+7.387µs; +11.311µs] or [+17.461%; +26.737%]
  • 🟥 throughput [-5195.898op/s; -3420.742op/s] or [-21.873%; -14.400%]

scenario:Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark netcoreapp3.1

  • unstable execution_time [-13.350µs; -5.320µs] or [-20.711%; -8.255%]
  • unstable throughput [+1296.331op/s; +3029.148op/s] or [+7.953%; +18.585%]

scenario:Benchmarks.Trace.Log4netBenchmark.EnrichedLog net472

  • 🟥 execution_time [+304.048ms; +305.508ms] or [+153.683%; +154.421%]

scenario:Benchmarks.Trace.Log4netBenchmark.EnrichedLog net6.0

  • 🟥 execution_time [+301.136ms; +304.109ms] or [+153.277%; +154.791%]

scenario:Benchmarks.Trace.Log4netBenchmark.EnrichedLog netcoreapp3.1

  • 🟥 execution_time [+299.846ms; +302.130ms] or [+150.110%; +151.253%]

scenario:Benchmarks.Trace.RedisBenchmark.SendReceive net472

  • 🟥 allocated_mem [+95 bytes; +96 bytes] or [+9.197%; +9.208%]
  • 🟥 throughput [-28614.474op/s; -24783.174op/s] or [-7.921%; -6.861%]

scenario:Benchmarks.Trace.RedisBenchmark.SendReceive net6.0

  • 🟥 allocated_mem [+95 bytes; +96 bytes] or [+9.226%; +9.238%]

scenario:Benchmarks.Trace.RedisBenchmark.SendReceive netcoreapp3.1

  • 🟥 allocated_mem [+95 bytes; +96 bytes] or [+9.226%; +9.238%]
  • 🟥 throughput [-30903.288op/s; -21988.832op/s] or [-7.314%; -5.204%]

scenario:Benchmarks.Trace.SerilogBenchmark.EnrichedLog net472

  • 🟥 execution_time [+300.042ms; +301.971ms] or [+149.544%; +150.505%]

scenario:Benchmarks.Trace.SerilogBenchmark.EnrichedLog net6.0

  • 🟥 allocated_mem [+96 bytes; +96 bytes] or [+6.000%; +6.010%]
  • 🟥 execution_time [+300.179ms; +306.836ms] or [+150.736%; +154.078%]

scenario:Benchmarks.Trace.SerilogBenchmark.EnrichedLog netcoreapp3.1

  • 🟥 allocated_mem [+95 bytes; +96 bytes] or [+5.819%; +5.828%]
  • 🟥 execution_time [+302.392ms; +304.770ms] or [+153.354%; +154.560%]

scenario:Benchmarks.Trace.SingleSpanAspNetCoreBenchmark.SingleSpanAspNetCore net472

  • 🟥 execution_time [+299.783ms; +300.459ms] or [+149.534%; +149.870%]
  • 🟩 throughput [+65312763.437op/s; +65572591.502op/s] or [+47.565%; +47.754%]

scenario:Benchmarks.Trace.SingleSpanAspNetCoreBenchmark.SingleSpanAspNetCore net6.0

  • 🟥 execution_time [+420.196ms; +425.878ms] or [+522.588%; +529.655%]

scenario:Benchmarks.Trace.SingleSpanAspNetCoreBenchmark.SingleSpanAspNetCore netcoreapp3.1

  • 🟥 execution_time [+299.102ms; +300.312ms] or [+149.185%; +149.789%]
  • 🟩 throughput [+18210132.003op/s; +19186173.411op/s] or [+8.066%; +8.498%]

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishScope net472

  • 🟥 allocated_mem [+31 bytes; +32 bytes] or [+5.244%; +5.256%]
  • 🟥 throughput [-91722.672op/s; -79475.305op/s] or [-10.236%; -8.869%]

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishSpan net472

  • 🟥 allocated_mem [+31 bytes; +32 bytes] or [+6.037%; +6.047%]
  • 🟥 throughput [-131880.000op/s; -129401.167op/s] or [-12.072%; -11.845%]

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishSpan net6.0

  • 🟥 allocated_mem [+31 bytes; +32 bytes] or [+6.057%; +6.065%]

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishSpan netcoreapp3.1

  • 🟥 allocated_mem [+31 bytes; +32 bytes] or [+6.059%; +6.069%]

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishTwoScopes net472

  • 🟥 throughput [-32419.277op/s; -27232.629op/s] or [-7.223%; -6.067%]

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishTwoScopes net6.0

  • 🟩 throughput [+32752.002op/s; +38813.554op/s] or [+5.947%; +7.048%]

scenario:Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin net472

  • 🟥 allocated_mem [+95 bytes; +96 bytes] or [+15.733%; +15.745%]
  • 🟥 throughput [-43626.584op/s; -39500.268op/s] or [-6.385%; -5.781%]

scenario:Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin net6.0

  • 🟥 allocated_mem [+95 bytes; +96 bytes] or [+14.810%; +14.823%]

Known flaky benchmarks without significant changes:

  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.ActivityBenchmark.StartSpan net472
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.ActivityBenchmark.StartSpan net6.0
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.ActivityBenchmark.StartSpan netcoreapp3.1
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.ActivityBenchmark.StartSpan_AddEvent_Sampled net472
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.ActivityBenchmark.StartSpan_AddEvent_Sampled net6.0
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.ActivityBenchmark.StartSpan_AddEvent_Sampled netcoreapp3.1
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.ActivityBenchmark.StartSpan_GetContext_Sampled net472
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.ActivityBenchmark.StartSpan_GetContext_Sampled net6.0
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.ActivityBenchmark.StartSpan_GetContext_Sampled netcoreapp3.1
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.ActivityBenchmark.StartSpan_SetAttributes_Sampled net6.0
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.ActivityBenchmark.StartSpan_SetAttributes_Sampled netcoreapp3.1
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.ActivityBenchmark.StartSpan_SetStatus_Sampled net472
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.ActivityBenchmark.StartSpan_SetStatus_Sampled net6.0
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.ActivityBenchmark.StartSpan_SetStatus_Sampled netcoreapp3.1
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.ActivityBenchmark.StartSpan_UpdateName_Sampled net6.0
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.ActivityBenchmark.StartSpan_UpdateName_Sampled netcoreapp3.1
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.TelemetrySpanBenchmark.StartSpan net472
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.TelemetrySpanBenchmark.StartSpan net6.0
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.TelemetrySpanBenchmark.StartSpan netcoreapp3.1
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.TelemetrySpanBenchmark.StartSpan_AddEvent_Sampled net472
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.TelemetrySpanBenchmark.StartSpan_AddEvent_Sampled net6.0
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.TelemetrySpanBenchmark.StartSpan_AddEvent_Sampled netcoreapp3.1
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.TelemetrySpanBenchmark.StartSpan_GetContext_Sampled net472
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.TelemetrySpanBenchmark.StartSpan_GetContext_Sampled net6.0
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.TelemetrySpanBenchmark.StartSpan_GetContext_Sampled netcoreapp3.1
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.TelemetrySpanBenchmark.StartSpan_RecordException_Sampled net472
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.TelemetrySpanBenchmark.StartSpan_RecordException_Sampled net6.0
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.TelemetrySpanBenchmark.StartSpan_RecordException_Sampled netcoreapp3.1
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.TelemetrySpanBenchmark.StartSpan_SetAttributes_Sampled net472
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.TelemetrySpanBenchmark.StartSpan_SetAttributes_Sampled net6.0
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.TelemetrySpanBenchmark.StartSpan_SetAttributes_Sampled netcoreapp3.1
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.TelemetrySpanBenchmark.StartSpan_SetStatus_Sampled net472
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.TelemetrySpanBenchmark.StartSpan_SetStatus_Sampled net6.0
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.TelemetrySpanBenchmark.StartSpan_SetStatus_Sampled netcoreapp3.1
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.TelemetrySpanBenchmark.StartSpan_UpdateName_Sampled net472
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.TelemetrySpanBenchmark.StartSpan_UpdateName_Sampled net6.0
  • scenario:Benchmarks.OpenTelemetry.InstrumentedApi.Trace.TelemetrySpanBenchmark.StartSpan_UpdateName_Sampled netcoreapp3.1
  • scenario:Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice net472
  • scenario:Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice net6.0
  • scenario:Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice netcoreapp3.1
  • scenario:Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool net472
  • scenario:Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool net6.0
  • scenario:Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool netcoreapp3.1
  • scenario:Benchmarks.Trace.CharSliceBenchmark.OriginalCharSlice net472
  • scenario:Benchmarks.Trace.CharSliceBenchmark.OriginalCharSlice netcoreapp3.1
  • scenario:Benchmarks.Trace.SpanBenchmark.StartFinishScope net6.0
  • scenario:Benchmarks.Trace.SpanBenchmark.StartFinishScope netcoreapp3.1
  • scenario:Benchmarks.Trace.SpanBenchmark.StartFinishTwoScopes netcoreapp3.1

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

area:ffe type:new-feature

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@vjfridge