Clean up CI test matrix: 13 entries down to 8#241
Open
PhilippGrulich wants to merge 2 commits intomainfrom
Open
Clean up CI test matrix: 13 entries down to 8#241PhilippGrulich wants to merge 2 commits intomainfrom
PhilippGrulich wants to merge 2 commits intomainfrom
Conversation
- Drop all ubuntu-22.04 entries (gcc-10, clang-15) - Replace clang-18/19 with clang-19/21 (project default) - Consolidate GCC to gcc-14 only, use gcc-14 for ASan - Enable ASMJIT backend by default in CMakeLists.txt, removing 3 explicit -DENABLE_ASMJIT_BACKEND=ON entries - Consolidate feature-toggle tests into one gcc-14 entry - Make clang install step generic (extract version from matrix) https://claude.ai/code/session_01AGaSTB8p6VGJ2eA6negxaz
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Tracing Benchmark
Details
| Benchmark suite | Current: 6620ff3 | Previous: 6b27760 | Ratio |
|---|---|---|---|
exec_bc_addOne |
35.8952 ns (± 1.01826) |
31.5133 ns (± 1.47662) |
1.14 |
exec_mlir_addOne |
276.846 ns (± 16.0575) |
426.959 ns (± 10.9997) |
0.65 |
exec_cpp_addOne |
3.76903 ns (± 0.6056) |
3.8169 ns (± 0.42368) |
0.99 |
exec_interpreted_addOne |
38.3961 ns (± 1.76675) |
36.1075 ns (± 2.80713) |
1.06 |
comp_mlir_add |
8.98315 ms (± 179.773) |
9.80556 ms (± 203.615) |
0.92 |
comp_mlir_ifThenElse |
9.5703 ms (± 159.472) |
10.2887 ms (± 222.287) |
0.93 |
comp_mlir_deeplyNestedIfElse |
8.4204 ms (± 139.21) |
9.21899 ms (± 396.158) |
0.91 |
comp_mlir_loop |
10.8344 ms (± 789.666) |
11.3512 ms (± 209.74) |
0.95 |
comp_mlir_ifInsideLoop |
32.535 ms (± 1.15501) |
31.6105 ms (± 409.372) |
1.03 |
comp_mlir_loopDirectCall |
15.5911 ms (± 838.56) |
15.6318 ms (± 269.541) |
1.00 |
comp_mlir_pointerLoop |
31.4113 ms (± 323.87) |
30.8828 ms (± 344.173) |
1.02 |
comp_mlir_staticLoop |
8.38096 ms (± 147.764) |
8.9691 ms (± 166.093) |
0.93 |
comp_mlir_fibonacci |
14.1355 ms (± 269.533) |
14.6547 ms (± 168.476) |
0.96 |
comp_mlir_gcd |
12.887 ms (± 288.013) |
13.5406 ms (± 188.742) |
0.95 |
comp_mlir_nestedIf10 |
13.9721 ms (± 527.165) |
14.3597 ms (± 266.738) |
0.97 |
comp_mlir_nestedIf100 |
28.5227 ms (± 483.287) |
28.5056 ms (± 234.978) |
1.00 |
comp_mlir_chainedIf10 |
13.1633 ms (± 166.001) |
14.996 ms (± 254.85) |
0.88 |
comp_mlir_chainedIf100 |
24.1678 ms (± 146.806) |
63.2918 ms (± 415.841) |
0.38 |
comp_cpp_add |
27.0288 ms (± 2.10824) |
23.1127 ms (± 428.831) |
1.17 |
comp_cpp_ifThenElse |
27.4182 ms (± 379.818) |
23.5572 ms (± 441.557) |
1.16 |
comp_cpp_deeplyNestedIfElse |
28.315 ms (± 231.201) |
24.6668 ms (± 513.626) |
1.15 |
comp_cpp_loop |
27.4204 ms (± 294.608) |
23.9817 ms (± 390.824) |
1.14 |
comp_cpp_ifInsideLoop |
28.4926 ms (± 271.613) |
24.3761 ms (± 342.933) |
1.17 |
comp_cpp_loopDirectCall |
27.8953 ms (± 235.001) |
24.2439 ms (± 544.239) |
1.15 |
comp_cpp_pointerLoop |
27.9799 ms (± 272.404) |
24.5815 ms (± 1.82614) |
1.14 |
comp_cpp_staticLoop |
27.2002 ms (± 230.388) |
23.8184 ms (± 357.042) |
1.14 |
comp_cpp_fibonacci |
27.7048 ms (± 690.566) |
24.2473 ms (± 425.092) |
1.14 |
comp_cpp_gcd |
27.9029 ms (± 1.86183) |
23.8836 ms (± 1.10049) |
1.17 |
comp_cpp_nestedIf10 |
30.5588 ms (± 632.842) |
26.9719 ms (± 491.927) |
1.13 |
comp_cpp_nestedIf100 |
64.1913 ms (± 1.70229) |
59.1255 ms (± 427.274) |
1.09 |
comp_cpp_chainedIf10 |
32.7767 ms (± 251.698) |
28.7604 ms (± 390.515) |
1.14 |
comp_cpp_chainedIf100 |
93.7645 ms (± 1.19487) |
87.8231 ms (± 655.714) |
1.07 |
comp_bc_add |
14.2906 us (± 2.87599) |
11.202 us (± 2.2074) |
1.28 |
comp_bc_ifThenElse |
15.6729 us (± 2.41481) |
13.2939 us (± 2.29983) |
1.18 |
comp_bc_deeplyNestedIfElse |
20.5209 us (± 4.13902) |
17.9851 us (± 2.87267) |
1.14 |
comp_bc_loop |
15.8691 us (± 2.50382) |
13.8129 us (± 2.85219) |
1.15 |
comp_bc_ifInsideLoop |
18.4125 us (± 3.38475) |
15.3559 us (± 2.53001) |
1.20 |
comp_bc_loopDirectCall |
16.1503 us (± 2.8388) |
13.3069 us (± 2.57466) |
1.21 |
comp_bc_pointerLoop |
17.6511 us (± 3.75154) |
14.5097 us (± 2.66856) |
1.22 |
comp_bc_staticLoop |
16.3204 us (± 2.44534) |
12.5812 us (± 2.72478) |
1.30 |
comp_bc_fibonacci |
16.0397 us (± 2.36996) |
13.3013 us (± 3.02441) |
1.21 |
comp_bc_gcd |
15.5666 us (± 2.40751) |
12.796 us (± 2.81743) |
1.22 |
comp_bc_nestedIf10 |
32.2803 us (± 4.30963) |
32.7855 us (± 5.39414) |
0.98 |
comp_bc_nestedIf100 |
177.718 us (± 9.71818) |
173.779 us (± 7.54629) |
1.02 |
comp_bc_chainedIf10 |
43.91 us (± 7.84442) |
40.6492 us (± 6.371) |
1.08 |
comp_bc_chainedIf100 |
287.031 us (± 20.3453) |
289.262 us (± 10.4213) |
0.99 |
comp_asmjit_add |
17.9309 us (± 3.92904) |
15.1187 us (± 4.95528) |
1.19 |
comp_asmjit_ifThenElse |
28.4033 us (± 6.56909) |
27.8613 us (± 7.11287) |
1.02 |
comp_asmjit_deeplyNestedIfElse |
51.5177 us (± 9.87248) |
45.1688 us (± 7.19196) |
1.14 |
comp_asmjit_loop |
30.658 us (± 6.8443) |
29.4438 us (± 8.29152) |
1.04 |
comp_asmjit_ifInsideLoop |
51.5627 us (± 10.6835) |
61.816 us (± 26.9624) |
0.83 |
comp_asmjit_loopDirectCall |
34.1053 us (± 6.58721) |
32.8387 us (± 7.17729) |
1.04 |
comp_asmjit_pointerLoop |
36.3224 us (± 6.12612) |
35.9059 us (± 8.01682) |
1.01 |
comp_asmjit_staticLoop |
24.198 us (± 5.06413) |
23.2977 us (± 6.63763) |
1.04 |
comp_asmjit_fibonacci |
32.6306 us (± 5.74386) |
31.8168 us (± 7.3914) |
1.03 |
comp_asmjit_gcd |
30.2035 us (± 6.68157) |
28.7574 us (± 7.41072) |
1.05 |
comp_asmjit_nestedIf10 |
104.31 us (± 13.3799) |
94.6042 us (± 11.146) |
1.10 |
comp_asmjit_nestedIf100 |
1.2011 ms (± 159.297) |
1.11692 ms (± 34.2849) |
1.08 |
comp_asmjit_chainedIf10 |
163.522 us (± 15.1621) |
146.493 us (± 14.2674) |
1.12 |
comp_asmjit_chainedIf100 |
2.34514 ms (± 23.5749) |
2.33025 ms (± 74.2677) |
1.01 |
trace_add |
2.39707 us (± 248.96) |
2.92672 us (± 367.994) |
0.82 |
completing_trace_add |
2.36592 us (± 198.285) |
2.8078 us (± 323.643) |
0.84 |
trace_ifThenElse |
10.671 us (± 1.93845) |
11.8504 us (± 2.41156) |
0.90 |
completing_trace_ifThenElse |
5.09934 us (± 576.431) |
5.93214 us (± 898.825) |
0.86 |
trace_deeplyNestedIfElse |
33.5635 us (± 8.62212) |
34.9611 us (± 6.07696) |
0.96 |
completing_trace_deeplyNestedIfElse |
15.683 us (± 2.07227) |
16.5467 us (± 2.56945) |
0.95 |
trace_loop |
10.5875 us (± 2.1698) |
11.8755 us (± 2.40443) |
0.89 |
completing_trace_loop |
5.19525 us (± 849.566) |
5.90652 us (± 780.209) |
0.88 |
trace_ifInsideLoop |
21.5391 us (± 4.08598) |
22.9887 us (± 6.00503) |
0.94 |
completing_trace_ifInsideLoop |
9.96845 us (± 1.49676) |
11.0473 us (± 2.0469) |
0.90 |
trace_loopDirectCall |
10.798 us (± 2.2004) |
11.9257 us (± 2.74476) |
0.91 |
completing_trace_loopDirectCall |
5.1974 us (± 701.896) |
5.94598 us (± 882.872) |
0.87 |
trace_pointerLoop |
17.3421 us (± 4.48682) |
18.5742 us (± 3.27291) |
0.93 |
completing_trace_pointerLoop |
11.5413 us (± 2.52973) |
12.8333 us (± 2.2561) |
0.90 |
trace_staticLoop |
9.51949 us (± 1.53279) |
9.84307 us (± 1.73462) |
0.97 |
completing_trace_staticLoop |
9.19551 us (± 1.16935) |
9.76063 us (± 1.9155) |
0.94 |
trace_fibonacci |
12.1079 us (± 2.2192) |
13.2585 us (± 2.36739) |
0.91 |
completing_trace_fibonacci |
6.54431 us (± 910.266) |
7.54262 us (± 1.26503) |
0.87 |
trace_gcd |
9.76586 us (± 1.64181) |
10.9047 us (± 2.29828) |
0.90 |
completing_trace_gcd |
4.35526 us (± 539.734) |
4.90646 us (± 712.302) |
0.89 |
trace_nestedIf10 |
58.5997 us (± 9.49304) |
57.4293 us (± 8.88093) |
1.02 |
completing_trace_nestedIf10 |
57.8954 us (± 8.78165) |
56.0183 us (± 6.98204) |
1.03 |
trace_nestedIf100 |
1.86057 ms (± 65.4996) |
1.71437 ms (± 30.9252) |
1.09 |
completing_trace_nestedIf100 |
1.86547 ms (± 35.3822) |
1.72187 ms (± 29.9083) |
1.08 |
trace_chainedIf10 |
132.99 us (± 12.7448) |
134.517 us (± 11.214) |
0.99 |
completing_trace_chainedIf10 |
72.3087 us (± 9.91755) |
71.9399 us (± 8.3763) |
1.01 |
trace_chainedIf100 |
4.90352 ms (± 55.0402) |
4.69625 ms (± 47.2122) |
1.04 |
completing_trace_chainedIf100 |
2.72552 ms (± 34.9307) |
2.86492 ms (± 31.2368) |
0.95 |
tiered_compile_addOne |
36.969 us (± 13.7753) |
30.8157 us (± 9.91006) |
1.20 |
single_compile_mlir_addOne |
6.85109 ms (± 143.514) |
7.09873 ms (± 235.664) |
0.97 |
single_compile_cpp_addOne |
26.9259 ms (± 1.40413) |
22.9638 ms (± 446.082) |
1.17 |
single_compile_bc_addOne |
37.6323 us (± 12.8153) |
32.9177 us (± 9.6468) |
1.14 |
tiered_compile_sumLoop |
51.2421 us (± 17.0713) |
49.4098 us (± 11.3711) |
1.04 |
single_compile_mlir_sumLoop |
8.97991 ms (± 353.721) |
9.54604 ms (± 218.921) |
0.94 |
single_compile_cpp_sumLoop |
27.4645 ms (± 267.802) |
23.4215 ms (± 469.559) |
1.17 |
single_compile_bc_sumLoop |
52.3447 us (± 17.4227) |
49.2555 us (± 12.1802) |
1.06 |
exec_mlir_add |
9.66933 ns (± 0.622973) |
11.7884 ns (± 1.1242) |
0.82 |
exec_mlir_fibonacci |
16.2399 us (± 1.06752) |
29.6758 us (± 2.66025) |
0.55 |
exec_mlir_sum |
562.696 us (± 61.7245) |
698.659 us (± 29.5044) |
0.81 |
exec_cpp_add |
4.49078 ns (± 0.476642) |
5.47998 ns (± 0.634902) |
0.82 |
exec_cpp_fibonacci |
107.272 us (± 5.28776) |
135.152 us (± 13.0396) |
0.79 |
exec_cpp_sum |
23.5333 ms (± 53.9286) |
25.6715 ms (± 158.084) |
0.92 |
exec_bc_add |
43.914 ns (± 5.82827) |
39.4171 ns (± 3.62026) |
1.11 |
exec_bc_fibonacci |
719.894 us (± 15.3589) |
552.308 us (± 6.99609) |
1.30 |
exec_bc_sum |
170.741 ms (± 16.9434) |
118.724 ms (± 3.31959) |
1.44 |
exec_asmjit_add |
3.23517 ns (± 0.285946) |
2.63871 ns (± 0.182011) |
1.23 |
exec_asmjit_fibonacci |
22.24 us (± 2.44215) |
31.5449 us (± 3.79249) |
0.71 |
exec_asmjit_sum |
5.12213 ms (± 24.7944) |
6.25776 ms (± 44.1308) |
0.82 |
ir_add |
827.866 ns (± 60.6575) |
887.068 ns (± 68.9307) |
0.93 |
ir_ifThenElse |
2.36961 us (± 140.117) |
2.84031 us (± 177.491) |
0.83 |
ir_deeplyNestedIfElse |
6.77354 us (± 523.711) |
7.92117 us (± 483.223) |
0.86 |
ir_loop |
2.87092 us (± 192.199) |
3.38614 us (± 198.951) |
0.85 |
ir_ifInsideLoop |
5.73276 us (± 467.29) |
6.78355 us (± 548.756) |
0.85 |
ir_loopDirectCall |
3.08791 us (± 210.778) |
3.74419 us (± 352.131) |
0.82 |
ir_pointerLoop |
3.76996 us (± 234.045) |
4.47156 us (± 432.502) |
0.84 |
ir_staticLoop |
2.25919 us (± 139.141) |
2.51218 us (± 144.496) |
0.90 |
ir_fibonacci |
3.04248 us (± 220.907) |
3.68307 us (± 293.835) |
0.83 |
ir_gcd |
2.58602 us (± 326.066) |
3.09055 us (± 229.195) |
0.84 |
ir_nestedIf10 |
15.8519 us (± 994.274) |
18.5992 us (± 855.122) |
0.85 |
ir_nestedIf100 |
190.054 us (± 10.051) |
200.22 us (± 3.30138) |
0.95 |
ir_chainedIf10 |
29.2792 us (± 1.81923) |
36.2247 us (± 2.28135) |
0.81 |
ir_chainedIf100 |
358.113 us (± 11.6868) |
383.811 us (± 7.76252) |
0.93 |
ssa_add |
184.856 ns (± 19.7006) |
204.278 ns (± 40.1528) |
0.90 |
ssa_ifThenElse |
425.813 ns (± 26.8578) |
448.014 ns (± 23.8545) |
0.95 |
ssa_deeplyNestedIfElse |
1.10217 us (± 71.9002) |
1.30584 us (± 102.385) |
0.84 |
ssa_loop |
448.452 ns (± 24.0782) |
480.298 ns (± 35.269) |
0.93 |
ssa_ifInsideLoop |
848.397 ns (± 57.2421) |
1054.81 ns (± 50848) |
0.80 |
ssa_loopDirectCall |
452.738 ns (± 32.4057) |
496.389 ns (± 40.2814) |
0.91 |
ssa_pointerLoop |
559.398 ns (± 37.0205) |
593.302 ns (± 40.7084) |
0.94 |
ssa_staticLoop |
468.155 ns (± 29.564) |
457.995 ns (± 36.7854) |
1.02 |
ssa_fibonacci |
479.414 ns (± 34.4128) |
499.376 ns (± 25.9612) |
0.96 |
ssa_gcd |
429.42 ns (± 30.3677) |
450.108 ns (± 30.3266) |
0.95 |
e2e_tiered_bc_to_mlir |
37.2996 us (± 13.9946) |
30.9276 us (± 9.2304) |
1.21 |
e2e_single_mlir |
8.88377 ms (± 146.03) |
9.14683 ms (± 252.099) |
0.97 |
This comment was automatically generated by workflow using github-action-benchmark.
The project's strict -pedantic-errors flag causes anonymous struct errors in asmjit when built with GCC. The previous Clang-specific -Wno-* flags had no effect on GCC. Use the existing disable_target_warnings() macro (already used for Catch2 and fmt) to suppress all warnings for the third-party asmjit library. https://claude.ai/code/session_01AGaSTB8p6VGJ2eA6negxaz
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
removing 3 explicit -DENABLE_ASMJIT_BACKEND=ON entries
https://claude.ai/code/session_01AGaSTB8p6VGJ2eA6negxaz