Add trace tests for 49 previously uncovered functions and patterns#186
Open
PhilippGrulich wants to merge 3 commits intomainfrom
Open
Add trace tests for 49 previously uncovered functions and patterns#186PhilippGrulich wants to merge 3 commits intomainfrom
PhilippGrulich wants to merge 3 commits intomainfrom
Conversation
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Tracing Benchmark
Details
| Benchmark suite | Current: 07cfaa9 | Previous: a94c041 | Ratio |
|---|---|---|---|
ssa_add |
182.643 ns (± 8.4346) |
189.854 ns (± 8.8728) |
0.96 |
ssa_ifThenElse |
469.96 ns (± 38.3387) |
470.398 ns (± 33.5247) |
1.00 |
ssa_deeplyNestedIfElse |
1.1703 us (± 89.6353) |
1.18264 us (± 67.8288) |
0.99 |
ssa_loop |
501.23 ns (± 34.7407) |
500.311 ns (± 24.3476) |
1.00 |
ssa_ifInsideLoop |
896.329 ns (± 66.119) |
934.871 ns (± 68.4223) |
0.96 |
ssa_loopDirectCall |
512.978 ns (± 52.447) |
544.303 ns (± 58.6122) |
0.94 |
ssa_pointerLoop |
598.624 ns (± 62.2146) |
602.57 ns (± 47.5605) |
0.99 |
ssa_staticLoop |
444.797 ns (± 76.2064) |
417.082 ns (± 20.4485) |
1.07 |
ssa_fibonacci |
529.624 ns (± 57.5476) |
522.363 ns (± 33.9407) |
1.01 |
ssa_gcd |
483.202 ns (± 70.9081) |
454.516 ns (± 22.3378) |
1.06 |
ir_add |
904.667 ns (± 161.002) |
886.753 ns (± 155.841) |
1.02 |
ir_ifThenElse |
2.46831 us (± 391.754) |
2.37082 us (± 144.788) |
1.04 |
ir_deeplyNestedIfElse |
6.15193 us (± 652.115) |
6.4817 us (± 594.719) |
0.95 |
ir_loop |
2.82825 us (± 834.095) |
2.76964 us (± 259.632) |
1.02 |
ir_ifInsideLoop |
5.17998 us (± 355.277) |
5.47838 us (± 506.981) |
0.95 |
ir_loopDirectCall |
2.9788 us (± 274.15) |
3.1426 us (± 310.146) |
0.95 |
ir_pointerLoop |
3.42842 us (± 266.816) |
3.62442 us (± 253.194) |
0.95 |
ir_staticLoop |
2.127 us (± 137.922) |
2.11912 us (± 112.156) |
1.00 |
ir_fibonacci |
2.91352 us (± 262.197) |
2.95137 us (± 236.135) |
0.99 |
ir_gcd |
2.47554 us (± 224.994) |
2.49399 us (± 222.552) |
0.99 |
ir_nestedIf10 |
18.3502 us (± 1.50946) |
14.8752 us (± 962.385) |
1.23 |
ir_nestedIf100 |
192.198 us (± 9.93458) |
180.715 us (± 6.96718) |
1.06 |
ir_chainedIf10 |
29.5478 us (± 5.59015) |
27.9073 us (± 1.88501) |
1.06 |
ir_chainedIf100 |
329.311 us (± 10.3282) |
369.384 us (± 21.5391) |
0.89 |
trace_add |
2.28224 us (± 197.73) |
2.31269 us (± 152.08) |
0.99 |
completing_trace_add |
2.21619 us (± 177.186) |
2.27177 us (± 297.347) |
0.98 |
trace_ifThenElse |
11.7562 us (± 2.89423) |
11.201 us (± 1.48931) |
1.05 |
completing_trace_ifThenElse |
4.85773 us (± 572.247) |
4.89995 us (± 502.813) |
0.99 |
trace_deeplyNestedIfElse |
33.7941 us (± 6.5534) |
33.8433 us (± 6.70751) |
1.00 |
completing_trace_deeplyNestedIfElse |
14.5922 us (± 1.97736) |
13.977 us (± 1.69294) |
1.04 |
trace_loop |
11.1788 us (± 1.70195) |
10.8828 us (± 1.629) |
1.03 |
completing_trace_loop |
5.22053 us (± 702.698) |
5.01563 us (± 471.123) |
1.04 |
trace_ifInsideLoop |
21.6251 us (± 2.42365) |
21.5118 us (± 2.69996) |
1.01 |
completing_trace_ifInsideLoop |
9.56774 us (± 998.573) |
9.24287 us (± 1.02439) |
1.04 |
trace_loopDirectCall |
11.0503 us (± 1.43088) |
11.0114 us (± 1.58471) |
1.00 |
completing_trace_loopDirectCall |
5.32524 us (± 838.596) |
4.91857 us (± 542.048) |
1.08 |
trace_pointerLoop |
16.5846 us (± 3.35546) |
16.3969 us (± 2.52489) |
1.01 |
completing_trace_pointerLoop |
10.2084 us (± 1.5457) |
16.8165 us (± 1.85588) |
0.61 |
trace_staticLoop |
8.89481 us (± 903.15) |
10.6353 us (± 1.3183) |
0.84 |
completing_trace_staticLoop |
8.90035 us (± 938.797) |
9.60258 us (± 1.03259) |
0.93 |
trace_fibonacci |
12.5626 us (± 2.62949) |
12.2836 us (± 1.65846) |
1.02 |
completing_trace_fibonacci |
6.27422 us (± 874.009) |
6.21119 us (± 669.975) |
1.01 |
trace_gcd |
10.4198 us (± 1.65894) |
10.1338 us (± 1.52478) |
1.03 |
completing_trace_gcd |
4.39216 us (± 439.094) |
4.25056 us (± 426.246) |
1.03 |
trace_nestedIf10 |
57.0881 us (± 7.32208) |
57.3211 us (± 6.28067) |
1.00 |
completing_trace_nestedIf10 |
54.0545 us (± 5.20989) |
58.8817 us (± 6.66274) |
0.92 |
trace_nestedIf100 |
1.76038 ms (± 83.125) |
1.73114 ms (± 61.9958) |
1.02 |
completing_trace_nestedIf100 |
1.766 ms (± 35.6703) |
1.78391 ms (± 58.1985) |
0.99 |
trace_chainedIf10 |
137.125 us (± 9.70342) |
135.901 us (± 11.1646) |
1.01 |
completing_trace_chainedIf10 |
67.9714 us (± 4.66549) |
68.0422 us (± 7.32388) |
1.00 |
trace_chainedIf100 |
5.06259 ms (± 56.9399) |
5.07229 ms (± 53.4982) |
1.00 |
completing_trace_chainedIf100 |
2.67391 ms (± 61.846) |
2.72815 ms (± 29.9088) |
0.98 |
exec_mlir_exceptionBasedTracing_add |
3.58602 ns (± 1.11636) |
3.47926 ns (± 0.419387) |
1.03 |
comp_mlir_add |
8.12171 ms (± 81.6181) |
8.76337 ms (± 359.136) |
0.93 |
comp_mlir_ifThenElse |
9.33151 ms (± 257.246) |
9.59782 ms (± 491.213) |
0.97 |
comp_mlir_deeplyNestedIfElse |
7.82208 ms (± 490.516) |
7.85152 ms (± 257.963) |
1.00 |
comp_mlir_loop |
9.58261 ms (± 153.044) |
9.9616 ms (± 384.277) |
0.96 |
comp_mlir_ifInsideLoop |
30.8986 ms (± 290.6) |
32.828 ms (± 574.216) |
0.94 |
comp_mlir_loopDirectCall |
14.2431 ms (± 206.483) |
15.5591 ms (± 501.153) |
0.92 |
comp_mlir_pointerLoop |
29.975 ms (± 385.347) |
31.5597 ms (± 813.897) |
0.95 |
comp_mlir_staticLoop |
7.45952 ms (± 133.977) |
7.99904 ms (± 470.107) |
0.93 |
comp_mlir_fibonacci |
12.8255 ms (± 165.414) |
13.8717 ms (± 740.229) |
0.92 |
comp_mlir_gcd |
11.7905 ms (± 235.963) |
12.8624 ms (± 521.317) |
0.92 |
comp_mlir_nestedIf10 |
12.7264 ms (± 103.665) |
13.5905 ms (± 395.93) |
0.94 |
comp_mlir_nestedIf100 |
26.6397 ms (± 242.015) |
29.4807 ms (± 803.834) |
0.90 |
comp_mlir_chainedIf10 |
11.919 ms (± 95.7374) |
13.0733 ms (± 490.62) |
0.91 |
comp_mlir_chainedIf100 |
22.2775 ms (± 195.751) |
23.6901 ms (± 684.597) |
0.94 |
comp_cpp_add |
26.3455 ms (± 283.727) |
29.5027 ms (± 1.26729) |
0.89 |
comp_cpp_ifThenElse |
27.9573 ms (± 1.07411) |
29.2713 ms (± 1.34548) |
0.96 |
comp_cpp_deeplyNestedIfElse |
28.8019 ms (± 798.098) |
30.0089 ms (± 884.798) |
0.96 |
comp_cpp_loop |
27.8627 ms (± 752.144) |
29.9151 ms (± 1.45235) |
0.93 |
comp_cpp_ifInsideLoop |
28.9249 ms (± 643.305) |
30.7793 ms (± 1.22846) |
0.94 |
comp_cpp_loopDirectCall |
28.5606 ms (± 677.504) |
30.1899 ms (± 1.09701) |
0.95 |
comp_cpp_pointerLoop |
29.2105 ms (± 1.05559) |
29.7113 ms (± 1.08924) |
0.98 |
comp_cpp_staticLoop |
28.478 ms (± 790.543) |
29.6982 ms (± 1.34717) |
0.96 |
comp_cpp_fibonacci |
28.7512 ms (± 674.76) |
29.3696 ms (± 1.30053) |
0.98 |
comp_cpp_gcd |
28.11 ms (± 716.213) |
28.9277 ms (± 1.15231) |
0.97 |
comp_cpp_nestedIf10 |
30.6879 ms (± 614.476) |
33.4369 ms (± 988.297) |
0.92 |
comp_cpp_nestedIf100 |
64.5128 ms (± 3.40206) |
66.8613 ms (± 1.72143) |
0.96 |
comp_cpp_chainedIf10 |
32.6219 ms (± 719.417) |
35.7876 ms (± 1.28741) |
0.91 |
comp_cpp_chainedIf100 |
94.4732 ms (± 1.03826) |
96.7916 ms (± 1.54812) |
0.98 |
comp_bc_add |
1.18067 us (± 85.3733) |
1.16043 us (± 68.7809) |
1.02 |
comp_bc_ifThenElse |
2.46136 us (± 195.489) |
2.5821 us (± 935.645) |
0.95 |
comp_bc_deeplyNestedIfElse |
5.75764 us (± 471.981) |
5.95746 us (± 645.183) |
0.97 |
comp_bc_loop |
2.49048 us (± 210.965) |
2.44889 us (± 141.26) |
1.02 |
comp_bc_ifInsideLoop |
4.39165 us (± 383.508) |
4.50943 us (± 397.636) |
0.97 |
comp_bc_loopDirectCall |
2.66989 us (± 233.195) |
2.60618 us (± 159.847) |
1.02 |
comp_bc_pointerLoop |
3.10681 us (± 322.457) |
3.19648 us (± 611.096) |
0.97 |
comp_bc_staticLoop |
2.65185 us (± 286.901) |
2.55605 us (± 150.218) |
1.04 |
comp_bc_fibonacci |
2.78549 us (± 415.996) |
2.66599 us (± 240.094) |
1.04 |
comp_bc_gcd |
2.27638 us (± 295.998) |
2.26902 us (± 186.223) |
1.00 |
comp_bc_nestedIf10 |
18.3603 us (± 1.53275) |
19.6324 us (± 2.59418) |
0.94 |
comp_bc_nestedIf100 |
174.287 us (± 6.94119) |
182.875 us (± 21.9627) |
0.95 |
comp_bc_chainedIf10 |
27.6244 us (± 2.60862) |
27.6953 us (± 1.81753) |
1.00 |
comp_bc_chainedIf100 |
284.671 us (± 9.3706) |
294.781 us (± 9.41496) |
0.97 |
This comment was automatically generated by workflow using github-action-benchmark.
PhilippGrulich
force-pushed
the
claude/trace-tests-coverage-6UcJT
branch
2 times, most recently
from
07cfaa9 to
23798bd
Compare
Coverage gaps found and addressed: Expression tests (35 new): - 8 existing functions missing from trace tests: subInt8AndInt8TruncateResult, chainedPrefixIncrement/Decrement, unaryPlus/Minus, moveAssignment, prefixIncrement/DecrementReturnValue - Wider type coverage: shiftLeft/Right for i32/i64, negate for i32/i64 - New standalone arithmetic: sub, mul, div, mod for int32 - New bitwise operators: AND, OR, XOR - New comparison operators: ==, !=, <, <=, >, >= - New unsigned arithmetic: uint32 add/sub/mul/div, uint64 add - New complex expressions: complexArithmetic, multipleAssignments, expressionWithConstants Bool tests (2 new): - boolComplexOps, boolProbabilityTest Runtime call tests (5 new): - directCallWithNestedCalls, loopDirectCall2, directCallComplexFunction, callCountFuncCall, callMemberFunction Loop tests (6 new functions + registrations): - forContinue, whileMultiCondition, nestedLoopBreak, loopCounterModification, doWhileMinOnce, power Static loop tests (1 new): - staticLoopWithIf Also applies clang-format fixes to existing files. https://claude.ai/code/session_01B68pzSQ7d6Pe6f2tYdDKLV
…gaps New trace test categories (5 new TEST_CASEs): - Bit intrinsics (12): countl_zero, countr_zero, popcount, byteswap, rotl, rotr for both 32-bit and 64-bit types - CMath intrinsics (19): sin, cos, sqrt, exp, log, ceil, floor, fabs, tan, asin, acos, atan, pow, atan2, fmin, fmax, fmod, copysign, fma - Memory intrinsics (6): memcpy, memmove, memset and variants - Profile/assume (3): assumeFunction, assumeComplexCondition, assumeAlignedFunction - Cross-type arithmetic (13): mulAWithB, addAWithB, subAWithB, divAWithB with mixed type combinations (i8/i32, i32/i64, float/double) Additional registrations in existing TEST_CASEs: - Pointer tests (6): isNotNullptr, pointerAddNegativeOffset, pointerRoundTrip, pointerMultiStep, pointerLessThanAfterAdd, pointerGreaterThanAfterAdd - Control flow (1): withBranchProbability - Runtime calls (2): incrementFuncCallFiveTimesWithModRef, incrementFuncCallFiveTimesWithRef https://claude.ai/code/session_01B68pzSQ7d6Pe6f2tYdDKLV
PhilippGrulich
force-pushed
the
claude/trace-tests-coverage-6UcJT
branch
from
1aa2927 to
e8dba2a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Coverage gaps found and addressed:
Expression tests (35 new):
chainedPrefixIncrement/Decrement, unaryPlus/Minus, moveAssignment,
prefixIncrement/DecrementReturnValue
expressionWithConstants
Bool tests (2 new):
Runtime call tests (5 new):
callCountFuncCall, callMemberFunction
Loop tests (6 new functions + registrations):
loopCounterModification, doWhileMinOnce, power
Static loop tests (1 new):
Also applies clang-format fixes to existing files.
https://claude.ai/code/session_01B68pzSQ7d6Pe6f2tYdDKLV