Profile-Guided Optimization (PGO) results on parsing JSON files

[zamazan4ik]

Hi!

I am doing research about Profile-Guided Optimization (PGO) effects on different kinds of software - my current results are available here. I decided to PGO-optimize json-benchmark and try to estimate the PGO performance boost for that kind of workload.

My test environment is Macbook M1 Pro, Ventura 13.4, with a connected charger (it's important since it affects the results). The results are collected with multiple runs of cargo run --release and choosing the greater results (since the same way is described in the README file). PGO optimization was done with cargo-pgo. My results are the following.

Release:

======= serde_json ======= parse|stringify ===== parse|stringify ====
data/canada.json         440 MB/s   710 MB/s   650 MB/s   510 MB/s
data/citm_catalog.json   830 MB/s   940 MB/s  1400 MB/s  1210 MB/s
data/twitter.json        510 MB/s  1360 MB/s   970 MB/s  1490 MB/s

==== rustc_serialize ===== parse|stringify ===== parse|stringify ====
data/canada.json         250 MB/s   140 MB/s   210 MB/s   100 MB/s
data/citm_catalog.json   350 MB/s   390 MB/s   270 MB/s   470 MB/s
data/twitter.json        200 MB/s   600 MB/s   150 MB/s   670 MB/s

======= simd-json ======== parse|stringify ===== parse|stringify ====
data/canada.json         710 MB/s   770 MB/s   890 MB/s
data/citm_catalog.json  1890 MB/s  1070 MB/s  2290 MB/s
data/twitter.json       1820 MB/s  1630 MB/s  1420 MB/s

PGO-optimized:

======= serde_json ======= parse|stringify ===== parse|stringify ====
data/canada.json         590 MB/s   720 MB/s  1150 MB/s   530 MB/s
data/citm_catalog.json  1100 MB/s  1090 MB/s  1620 MB/s  1210 MB/s
data/twitter.json        600 MB/s  1370 MB/s   990 MB/s  1370 MB/s

==== rustc_serialize ===== parse|stringify ===== parse|stringify ====
data/canada.json         280 MB/s   150 MB/s   230 MB/s   110 MB/s
data/citm_catalog.json   450 MB/s   470 MB/s   340 MB/s   530 MB/s
data/twitter.json        220 MB/s   740 MB/s   170 MB/s   780 MB/s

======= simd-json ======== parse|stringify ===== parse|stringify ====
data/canada.json         740 MB/s   780 MB/s   990 MB/s
data/citm_catalog.json  1900 MB/s  1190 MB/s  2520 MB/s
data/twitter.json       1900 MB/s  1700 MB/s  1620 MB/s

According to these tests, PGO allows us to achieve better performance with parsing JSON files. Hope my tests can help someone. Probably would be a good idea to add the information about PGO to the libraries' documentation somewhere in a "Performance tuning" section.