Fix flaky KafkaClientDataStreamsDisabledForkedTest batch consume test by bm1549 · Pull Request #10797 · DataDog/dd-trace-java

bm1549 · 2026-03-10T19:54:29Z

What Does This Do

Fixes the flaky test spring kafka template produce and batch consume test in KafkaClientDataStreamsDisabledForkedTest by replacing the positional index-based consumer-to-producer span mapping with dynamic parent lookup.

Motivation

The test uses SORT_TRACES_BY_ID which sorts traces by their root span's spanId (a random value). In the else branch (when hasQueueSpan() is false), the test assumed consumer traces at indices 1, 2, 3 would map to specific producer spans at trace(0)[6], trace(0)[4], trace(0)[2] respectively. Since the sort order is non-deterministic, this mapping was incorrect when trace IDs happened to sort the consumer traces in a different order, causing assert span.parentId == parent.spanId failures.

The fix dynamically finds each consumer span's actual parent producer span by matching the consumer span's parentId against the producer spans' spanId, eliminating any dependency on sort order.

Additional Notes

Only the else branch (no queue span, used by KafkaClientDataStreamsDisabledForkedTest) is modified
The offset range is relaxed to 0..1 for all consumer spans since the consumer-to-producer mapping is no longer positional
The hasQueueSpan() branch is left unchanged as it was not reported as flaky

Contributor Checklist

Format the title according to the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any other useful labels
Avoid using close, fix, or any linking keywords when referencing an issue

🤖 Generated with Claude Code

pr-commenter · 2026-03-10T20:27:19Z

Kafka / producer-benchmark

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	brian.marks/fix-kafka-dsm-disabled-flaky-test-v2
git_commit_date	1773169596	1773172392
git_commit_sha	`cc12228`	`79b8bde`

See matching parameters

	Baseline	Candidate
ci_job_date	1773173344	1773173344
ci_job_id	1494433095	1494433095
ci_pipeline_id	101729027	101729027
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion	11.0.25	11.0.25
jmhVersion	1.36	1.36
jvm	/usr/lib/jvm/java-11-openjdk-amd64/bin/java	/usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName	OpenJDK 64-Bit Server VM	OpenJDK 64-Bit Server VM
vmVersion	11.0.25+9-post-Ubuntu-1ubuntu122.04	11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results

scenario	Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce	same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce	same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce	unsure [-4588.554op/s; -731.962op/s] or [-3.053%; -0.487%]

pr-commenter · 2026-03-10T20:38:59Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	brian.marks/fix-kafka-dsm-disabled-flaky-test-v2
git_commit_date	1773169596	1773175790
git_commit_sha	`cc12228`	`106ceb0`
release_version	1.61.0-SNAPSHOT~cc122288e5	1.61.0-SNAPSHOT~106ceb0206

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1773177525	1773177525
ci_job_id	1494686490	1494686490
ci_pipeline_id	101744899	101744899
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-77qh36ol 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-77qh36ol 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 63 metrics, 8 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.61.0-SNAPSHOT~106ceb0206, baseline=1.61.0-SNAPSHOT~cc122288e5

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.056 s) : 0, 1055563
Total [baseline] (11.044 s) : 0, 11044077
Agent [candidate] (1.059 s) : 0, 1059303
Total [candidate] (10.988 s) : 0, 10987651
section appsec
Agent [baseline] (1.242 s) : 0, 1242029
Total [baseline] (11.156 s) : 0, 11155989
Agent [candidate] (1.246 s) : 0, 1246091
Total [candidate] (11.13 s) : 0, 11130115
section iast
Agent [baseline] (1.227 s) : 0, 1226734
Total [baseline] (11.377 s) : 0, 11377036
Agent [candidate] (1.246 s) : 0, 1245535
Total [candidate] (11.328 s) : 0, 11328229
section profiling
Agent [baseline] (1.18 s) : 0, 1180139
Total [baseline] (10.939 s) : 0, 10938520
Agent [candidate] (1.181 s) : 0, 1181001
Total [candidate] (11.005 s) : 0, 11004739

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.056 s	-
Agent	appsec	1.242 s	186.467 ms (17.7%)
Agent	iast	1.227 s	171.172 ms (16.2%)
Agent	profiling	1.18 s	124.576 ms (11.8%)
Total	tracing	11.044 s	-
Total	appsec	11.156 s	111.912 ms (1.0%)
Total	iast	11.377 s	332.959 ms (3.0%)
Total	profiling	10.939 s	-105.557 ms (-1.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.059 s	-
Agent	appsec	1.246 s	186.787 ms (17.6%)
Agent	iast	1.246 s	186.232 ms (17.6%)
Agent	profiling	1.181 s	121.697 ms (11.5%)
Total	tracing	10.988 s	-
Total	appsec	11.13 s	142.464 ms (1.3%)
Total	iast	11.328 s	340.579 ms (3.1%)
Total	profiling	11.005 s	17.089 ms (0.2%)

gantt
    title petclinic - break down per module: candidate=1.61.0-SNAPSHOT~106ceb0206, baseline=1.61.0-SNAPSHOT~cc122288e5

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.2 ms) : 0, 1200
crashtracking [candidate] (1.185 ms) : 0, 1185
BytebuddyAgent [baseline] (626.758 ms) : 0, 626758
BytebuddyAgent [candidate] (629.283 ms) : 0, 629283
AgentMeter [baseline] (29.08 ms) : 0, 29080
AgentMeter [candidate] (29.179 ms) : 0, 29179
GlobalTracer [baseline] (256.072 ms) : 0, 256072
GlobalTracer [candidate] (256.993 ms) : 0, 256993
AppSec [baseline] (31.44 ms) : 0, 31440
AppSec [candidate] (31.583 ms) : 0, 31583
Debugger [baseline] (59.266 ms) : 0, 59266
Debugger [candidate] (59.469 ms) : 0, 59469
Remote Config [baseline] (582.081 µs) : 0, 582
Remote Config [candidate] (596.716 µs) : 0, 597
Telemetry [baseline] (8.591 ms) : 0, 8591
Telemetry [candidate] (8.59 ms) : 0, 8590
Flare Poller [baseline] (6.491 ms) : 0, 6491
Flare Poller [candidate] (6.4 ms) : 0, 6400
section appsec
crashtracking [baseline] (1.189 ms) : 0, 1189
crashtracking [candidate] (1.187 ms) : 0, 1187
BytebuddyAgent [baseline] (655.574 ms) : 0, 655574
BytebuddyAgent [candidate] (658.604 ms) : 0, 658604
AgentMeter [baseline] (12.047 ms) : 0, 12047
AgentMeter [candidate] (11.998 ms) : 0, 11998
GlobalTracer [baseline] (257.753 ms) : 0, 257753
GlobalTracer [candidate] (258.268 ms) : 0, 258268
IAST [baseline] (23.871 ms) : 0, 23871
IAST [candidate] (23.97 ms) : 0, 23970
AppSec [baseline] (176.615 ms) : 0, 176615
AppSec [candidate] (177.13 ms) : 0, 177130
Debugger [baseline] (65.429 ms) : 0, 65429
Debugger [candidate] (65.487 ms) : 0, 65487
Remote Config [baseline] (563.764 µs) : 0, 564
Remote Config [candidate] (567.703 µs) : 0, 568
Telemetry [baseline] (9.084 ms) : 0, 9084
Telemetry [candidate] (9.009 ms) : 0, 9009
Flare Poller [baseline] (3.632 ms) : 0, 3632
Flare Poller [candidate] (3.581 ms) : 0, 3581
section iast
crashtracking [baseline] (1.185 ms) : 0, 1185
crashtracking [candidate] (1.211 ms) : 0, 1211
BytebuddyAgent [baseline] (795.759 ms) : 0, 795759
BytebuddyAgent [candidate] (810.425 ms) : 0, 810425
AgentMeter [baseline] (11.334 ms) : 0, 11334
AgentMeter [candidate] (11.736 ms) : 0, 11736
GlobalTracer [baseline] (247.034 ms) : 0, 247034
GlobalTracer [candidate] (249.362 ms) : 0, 249362
IAST [baseline] (25.184 ms) : 0, 25184
IAST [candidate] (25.369 ms) : 0, 25369
AppSec [baseline] (26.48 ms) : 0, 26480
AppSec [candidate] (26.612 ms) : 0, 26612
Debugger [baseline] (63.439 ms) : 0, 63439
Debugger [candidate] (64.636 ms) : 0, 64636
Remote Config [baseline] (524.967 µs) : 0, 525
Remote Config [candidate] (529.328 µs) : 0, 529
Telemetry [baseline] (14.864 ms) : 0, 14864
Telemetry [candidate] (14.443 ms) : 0, 14443
Flare Poller [baseline] (4.904 ms) : 0, 4904
Flare Poller [candidate] (4.751 ms) : 0, 4751
section profiling
crashtracking [baseline] (1.167 ms) : 0, 1167
crashtracking [candidate] (1.184 ms) : 0, 1184
BytebuddyAgent [baseline] (681.222 ms) : 0, 681222
BytebuddyAgent [candidate] (682.483 ms) : 0, 682483
AgentMeter [baseline] (8.577 ms) : 0, 8577
AgentMeter [candidate] (8.616 ms) : 0, 8616
GlobalTracer [baseline] (214.855 ms) : 0, 214855
GlobalTracer [candidate] (215.366 ms) : 0, 215366
AppSec [baseline] (31.925 ms) : 0, 31925
AppSec [candidate] (31.802 ms) : 0, 31802
Debugger [baseline] (62.598 ms) : 0, 62598
Debugger [candidate] (62.727 ms) : 0, 62727
Remote Config [baseline] (586.064 µs) : 0, 586
Remote Config [candidate] (586.971 µs) : 0, 587
Telemetry [baseline] (9.774 ms) : 0, 9774
Telemetry [candidate] (9.715 ms) : 0, 9715
Flare Poller [baseline] (4.317 ms) : 0, 4317
Flare Poller [candidate] (4.254 ms) : 0, 4254
ProfilingAgent [baseline] (94.48 ms) : 0, 94480
ProfilingAgent [candidate] (93.599 ms) : 0, 93599
Profiling [baseline] (95.058 ms) : 0, 95058
Profiling [candidate] (94.157 ms) : 0, 94157

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.61.0-SNAPSHOT~106ceb0206, baseline=1.61.0-SNAPSHOT~cc122288e5

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.067 s) : 0, 1066745
Total [baseline] (8.877 s) : 0, 8876526
Agent [candidate] (1.06 s) : 0, 1059749
Total [candidate] (8.803 s) : 0, 8803193
section iast
Agent [baseline] (1.229 s) : 0, 1228833
Total [baseline] (9.544 s) : 0, 9543983
Agent [candidate] (1.236 s) : 0, 1236047
Total [candidate] (9.551 s) : 0, 9551375

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.067 s	-
Agent	iast	1.229 s	162.087 ms (15.2%)
Total	tracing	8.877 s	-
Total	iast	9.544 s	667.456 ms (7.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.06 s	-
Agent	iast	1.236 s	176.297 ms (16.6%)
Total	tracing	8.803 s	-
Total	iast	9.551 s	748.182 ms (8.5%)

gantt
    title insecure-bank - break down per module: candidate=1.61.0-SNAPSHOT~106ceb0206, baseline=1.61.0-SNAPSHOT~cc122288e5

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.199 ms) : 0, 1199
crashtracking [candidate] (1.215 ms) : 0, 1215
BytebuddyAgent [baseline] (632.901 ms) : 0, 632901
BytebuddyAgent [candidate] (628.688 ms) : 0, 628688
AgentMeter [baseline] (29.357 ms) : 0, 29357
AgentMeter [candidate] (29.077 ms) : 0, 29077
GlobalTracer [baseline] (258.769 ms) : 0, 258769
GlobalTracer [candidate] (257.254 ms) : 0, 257254
AppSec [baseline] (31.81 ms) : 0, 31810
AppSec [candidate] (31.648 ms) : 0, 31648
Debugger [baseline] (59.156 ms) : 0, 59156
Debugger [candidate] (58.641 ms) : 0, 58641
Remote Config [baseline] (592.656 µs) : 0, 593
Remote Config [candidate] (581.397 µs) : 0, 581
Telemetry [baseline] (8.781 ms) : 0, 8781
Telemetry [candidate] (8.654 ms) : 0, 8654
Flare Poller [baseline] (7.996 ms) : 0, 7996
Flare Poller [candidate] (7.964 ms) : 0, 7964
section iast
crashtracking [baseline] (1.19 ms) : 0, 1190
crashtracking [candidate] (1.203 ms) : 0, 1203
BytebuddyAgent [baseline] (797.997 ms) : 0, 797997
BytebuddyAgent [candidate] (803.297 ms) : 0, 803297
AgentMeter [baseline] (11.35 ms) : 0, 11350
AgentMeter [candidate] (11.586 ms) : 0, 11586
GlobalTracer [baseline] (248.004 ms) : 0, 248004
GlobalTracer [candidate] (248.693 ms) : 0, 248693
AppSec [baseline] (26.314 ms) : 0, 26314
AppSec [candidate] (26.604 ms) : 0, 26604
Debugger [baseline] (62.65 ms) : 0, 62650
Debugger [candidate] (62.736 ms) : 0, 62736
Remote Config [baseline] (524.558 µs) : 0, 525
Remote Config [candidate] (550.346 µs) : 0, 550
Telemetry [baseline] (14.862 ms) : 0, 14862
Telemetry [candidate] (14.81 ms) : 0, 14810
Flare Poller [baseline] (4.842 ms) : 0, 4842
Flare Poller [candidate] (4.967 ms) : 0, 4967
IAST [baseline] (25.116 ms) : 0, 25116
IAST [candidate] (25.432 ms) : 0, 25432

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	brian.marks/fix-kafka-dsm-disabled-flaky-test-v2
git_commit_date	1773169596	1773175790
git_commit_sha	`cc12228`	`106ceb0`
release_version	1.61.0-SNAPSHOT~cc122288e5	1.61.0-SNAPSHOT~106ceb0206

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1773178007	1773178007
ci_job_id	1494686492	1494686492
ci_pipeline_id	101744899	101744899
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-gato387k 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-gato387k 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 1 performance regressions! Performance is the same for 18 metrics, 17 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:petclinic:appsec:high_load	worse [+0.599ms; +1.407ms] or [+3.292%; +7.732%]	unsure [+0.363ms; +1.693ms] or [+1.213%; +5.662%]	unstable [-39.659op/s; +15.346op/s] or [-15.826%; +6.124%]	19.196ms	30.937ms	238.438op/s	18.193ms	29.908ms	250.594op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~106ceb0206, baseline=1.61.0-SNAPSHOT~cc122288e5
    dateFormat X
    axisFormat %s
section baseline
no_agent (19.05 ms) : 18856, 19243
.   : milestone, 19050,
appsec (18.624 ms) : 18433, 18816
.   : milestone, 18624,
code_origins (17.692 ms) : 17517, 17868
.   : milestone, 17692,
iast (18.01 ms) : 17829, 18192
.   : milestone, 18010,
profiling (18.931 ms) : 18743, 19119
.   : milestone, 18931,
tracing (17.898 ms) : 17720, 18076
.   : milestone, 17898,
section candidate
no_agent (19.679 ms) : 19476, 19881
.   : milestone, 19679,
appsec (19.577 ms) : 19377, 19778
.   : milestone, 19577,
code_origins (17.726 ms) : 17550, 17903
.   : milestone, 17726,
iast (17.527 ms) : 17352, 17702
.   : milestone, 17527,
profiling (19.51 ms) : 19310, 19711
.   : milestone, 19510,
tracing (18.016 ms) : 17834, 18197
.   : milestone, 18016,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	19.05 ms [18.856 ms, 19.243 ms]	-
appsec	18.624 ms [18.433 ms, 18.816 ms]	-425.552 µs (-2.2%)
code_origins	17.692 ms [17.517 ms, 17.868 ms]	-1.357 ms (-7.1%)
iast	18.01 ms [17.829 ms, 18.192 ms]	-1.039 ms (-5.5%)
profiling	18.931 ms [18.743 ms, 19.119 ms]	-118.748 µs (-0.6%)
tracing	17.898 ms [17.72 ms, 18.076 ms]	-1.152 ms (-6.0%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	19.679 ms [19.476 ms, 19.881 ms]	-
appsec	19.577 ms [19.377 ms, 19.778 ms]	-101.3 µs (-0.5%)
code_origins	17.726 ms [17.55 ms, 17.903 ms]	-1.952 ms (-9.9%)
iast	17.527 ms [17.352 ms, 17.702 ms]	-2.152 ms (-10.9%)
profiling	19.51 ms [19.31 ms, 19.711 ms]	-168.322 µs (-0.9%)
tracing	18.016 ms [17.834 ms, 18.197 ms]	-1.663 ms (-8.5%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~106ceb0206, baseline=1.61.0-SNAPSHOT~cc122288e5
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.17 ms) : 1159, 1182
.   : milestone, 1170,
iast (3.348 ms) : 3298, 3398
.   : milestone, 3348,
iast_FULL (5.858 ms) : 5799, 5916
.   : milestone, 5858,
iast_GLOBAL (3.541 ms) : 3482, 3601
.   : milestone, 3541,
profiling (2.067 ms) : 2049, 2086
.   : milestone, 2067,
tracing (1.779 ms) : 1764, 1794
.   : milestone, 1779,
section candidate
no_agent (1.183 ms) : 1172, 1195
.   : milestone, 1183,
iast (3.302 ms) : 3263, 3342
.   : milestone, 3302,
iast_FULL (5.834 ms) : 5776, 5892
.   : milestone, 5834,
iast_GLOBAL (3.618 ms) : 3560, 3675
.   : milestone, 3618,
profiling (2.053 ms) : 2034, 2072
.   : milestone, 2053,
tracing (1.732 ms) : 1718, 1745
.   : milestone, 1732,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.17 ms [1.159 ms, 1.182 ms]	-
iast	3.348 ms [3.298 ms, 3.398 ms]	2.178 ms (186.1%)
iast_FULL	5.858 ms [5.799 ms, 5.916 ms]	4.688 ms (400.5%)
iast_GLOBAL	3.541 ms [3.482 ms, 3.601 ms]	2.371 ms (202.6%)
profiling	2.067 ms [2.049 ms, 2.086 ms]	896.969 µs (76.6%)
tracing	1.779 ms [1.764 ms, 1.794 ms]	608.508 µs (52.0%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.183 ms [1.172 ms, 1.195 ms]	-
iast	3.302 ms [3.263 ms, 3.342 ms]	2.119 ms (179.1%)
iast_FULL	5.834 ms [5.776 ms, 5.892 ms]	4.65 ms (393.0%)
iast_GLOBAL	3.618 ms [3.56 ms, 3.675 ms]	2.434 ms (205.7%)
profiling	2.053 ms [2.034 ms, 2.072 ms]	869.517 µs (73.5%)
tracing	1.732 ms [1.718 ms, 1.745 ms]	548.42 µs (46.3%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	brian.marks/fix-kafka-dsm-disabled-flaky-test-v2
git_commit_date	1773169596	1773175790
git_commit_sha	`cc12228`	`106ceb0`
release_version	1.61.0-SNAPSHOT~cc122288e5	1.61.0-SNAPSHOT~106ceb0206

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1773177722	1773177722
ci_job_id	1494686494	1494686494
ci_pipeline_id	101744899	101744899
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-bd3uf914 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-bd3uf914 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~106ceb0206, baseline=1.61.0-SNAPSHOT~cc122288e5
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.471 ms) : 1460, 1483
.   : milestone, 1471,
appsec (3.76 ms) : 3543, 3976
.   : milestone, 3760,
iast (2.269 ms) : 2199, 2339
.   : milestone, 2269,
iast_GLOBAL (2.3 ms) : 2229, 2370
.   : milestone, 2300,
profiling (2.108 ms) : 2051, 2165
.   : milestone, 2108,
tracing (2.058 ms) : 2004, 2112
.   : milestone, 2058,
section candidate
no_agent (1.473 ms) : 1461, 1484
.   : milestone, 1473,
appsec (3.809 ms) : 3588, 4031
.   : milestone, 3809,
iast (2.259 ms) : 2189, 2328
.   : milestone, 2259,
iast_GLOBAL (2.308 ms) : 2238, 2379
.   : milestone, 2308,
profiling (2.117 ms) : 2059, 2175
.   : milestone, 2117,
tracing (2.068 ms) : 2013, 2123
.   : milestone, 2068,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.471 ms [1.46 ms, 1.483 ms]	-
appsec	3.76 ms [3.543 ms, 3.976 ms]	2.288 ms (155.5%)
iast	2.269 ms [2.199 ms, 2.339 ms]	797.814 µs (54.2%)
iast_GLOBAL	2.3 ms [2.229 ms, 2.37 ms]	828.239 µs (56.3%)
profiling	2.108 ms [2.051 ms, 2.165 ms]	637.016 µs (43.3%)
tracing	2.058 ms [2.004 ms, 2.112 ms]	586.745 µs (39.9%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.473 ms [1.461 ms, 1.484 ms]	-
appsec	3.809 ms [3.588 ms, 4.031 ms]	2.337 ms (158.7%)
iast	2.259 ms [2.189 ms, 2.328 ms]	786.214 µs (53.4%)
iast_GLOBAL	2.308 ms [2.238 ms, 2.379 ms]	835.972 µs (56.8%)
profiling	2.117 ms [2.059 ms, 2.175 ms]	644.593 µs (43.8%)
tracing	2.068 ms [2.013 ms, 2.123 ms]	595.281 µs (40.4%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~106ceb0206, baseline=1.61.0-SNAPSHOT~cc122288e5
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.308 s) : 15308000, 15308000
.   : milestone, 15308000,
appsec (14.994 s) : 14994000, 14994000
.   : milestone, 14994000,
iast (18.274 s) : 18274000, 18274000
.   : milestone, 18274000,
iast_GLOBAL (17.832 s) : 17832000, 17832000
.   : milestone, 17832000,
profiling (14.657 s) : 14657000, 14657000
.   : milestone, 14657000,
tracing (15.013 s) : 15013000, 15013000
.   : milestone, 15013000,
section candidate
no_agent (15.488 s) : 15488000, 15488000
.   : milestone, 15488000,
appsec (14.599 s) : 14599000, 14599000
.   : milestone, 14599000,
iast (18.509 s) : 18509000, 18509000
.   : milestone, 18509000,
iast_GLOBAL (17.303 s) : 17303000, 17303000
.   : milestone, 17303000,
profiling (14.9 s) : 14900000, 14900000
.   : milestone, 14900000,
tracing (15.0 s) : 15000000, 15000000
.   : milestone, 15000000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.308 s [15.308 s, 15.308 s]	-
appsec	14.994 s [14.994 s, 14.994 s]	-314.0 ms (-2.1%)
iast	18.274 s [18.274 s, 18.274 s]	2.966 s (19.4%)
iast_GLOBAL	17.832 s [17.832 s, 17.832 s]	2.524 s (16.5%)
profiling	14.657 s [14.657 s, 14.657 s]	-651.0 ms (-4.3%)
tracing	15.013 s [15.013 s, 15.013 s]	-295.0 ms (-1.9%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.488 s [15.488 s, 15.488 s]	-
appsec	14.599 s [14.599 s, 14.599 s]	-889.0 ms (-5.7%)
iast	18.509 s [18.509 s, 18.509 s]	3.021 s (19.5%)
iast_GLOBAL	17.303 s [17.303 s, 17.303 s]	1.815 s (11.7%)
profiling	14.9 s [14.9 s, 14.9 s]	-588.0 ms (-3.8%)
tracing	15.0 s [15.0 s, 15.0 s]	-488.0 ms (-3.2%)

pr-commenter · 2026-03-10T20:39:39Z

Kafka / consumer-benchmark

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	brian.marks/fix-kafka-dsm-disabled-flaky-test-v2
git_commit_date	1773169596	1773175790
git_commit_sha	`cc12228`	`106ceb0`

See matching parameters

	Baseline	Candidate
ci_job_date	1773176849	1773176849
ci_job_id	1494686498	1494686498
ci_pipeline_id	101744899	101744899
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion	11.0.25	11.0.25
jmhVersion	1.36	1.36
jvm	/usr/lib/jvm/java-11-openjdk-amd64/bin/java	/usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName	OpenJDK 64-Bit Server VM	OpenJDK 64-Bit Server VM
vmVersion	11.0.25+9-post-Ubuntu-1ubuntu122.04	11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results

scenario	Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume	same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume	same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume	same

The test mapped consumer traces to producer spans by positional index after SORT_TRACES_BY_ID sorting. Since trace IDs are random, the consumer-to-producer mapping was non-deterministic, causing intermittent `span.parentId == parent.spanId` assertion failures. Fix by dynamically finding each consumer span's actual parent producer span via parentId matching instead of relying on sort order. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

bm1549 added type: bug Bug report and fix tag: flaky test Flaky tests inst: kafka Kafka instrumentation tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes labels Mar 10, 2026

bm1549 force-pushed the brian.marks/fix-kafka-dsm-disabled-flaky-test-v2 branch from 79b8bde to 106ceb0 Compare March 10, 2026 20:50

bm1549 marked this pull request as ready for review March 10, 2026 22:00

bm1549 requested review from a team as code owners March 10, 2026 22:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix flaky KafkaClientDataStreamsDisabledForkedTest batch consume test#10797

Fix flaky KafkaClientDataStreamsDisabledForkedTest batch consume test#10797
bm1549 wants to merge 1 commit intomasterfrom
brian.marks/fix-kafka-dsm-disabled-flaky-test-v2

bm1549 commented Mar 10, 2026

Uh oh!

pr-commenter bot commented Mar 10, 2026

Uh oh!

pr-commenter bot commented Mar 10, 2026 •

edited

Loading

Uh oh!

pr-commenter bot commented Mar 10, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

bm1549 commented Mar 10, 2026

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenter bot commented Mar 10, 2026

Kafka / producer-benchmark

Parameters

Summary

Uh oh!

pr-commenter bot commented Mar 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

pr-commenter bot commented Mar 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Kafka / consumer-benchmark

Parameters

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

pr-commenter bot commented Mar 10, 2026 •

edited

Loading

pr-commenter bot commented Mar 10, 2026 •

edited

Loading