Skip to content

refactor(codec): expose base64 encode/decode#1545

Open
nabetti1720 wants to merge 1 commit into
awslabs:mainfrom
nabetti1720:refactor/expose-base64-codec
Open

refactor(codec): expose base64 encode/decode#1545
nabetti1720 wants to merge 1 commit into
awslabs:mainfrom
nabetti1720:refactor/expose-base64-codec

Conversation

@nabetti1720
Copy link
Copy Markdown
Contributor

@nabetti1720 nabetti1720 commented May 5, 2026

Issue # (if available)

n/a

Description of changes

  • When bundling the @aws-sdk, LLRT replaces @smithy/util-base64 with a native implementation.
  • Previously, native implementations used Buffer.from(), but by replacing it with the simd-compatible base64 encode/decode already provided by llrt_encoding, we can further improve speed.
  • Note that while Uint8Array.[from|to]Base64() appears to perform better when the input is a Uint8Array, its performance drops significantly as the input length increases. In contrast, llrt:codec.[decodeFrom|encodeTo]Base64() exhibits relatively minor degradation, so llrt:codec.[decodeFrom|encodeTo]Base64() was chosen for this pull request.

Benchmark result

benchmarks/mitata/decode_from_base64.js
% llrt benchmarks/mitata/decode_from_base64.js
Buffer [ 117, 115, 101, 114, 58, 112, 97, 115, 115 ]
Uint8Array [ 117, 115, 101, 114, 58, 112, 97, 115, 115 ]
Uint8Array [ 117, 115, 101, 114, 58, 112, 97, 115, 115 ]
clk: ~0.04 GHz
cpu: Apple M2
runtime: llrt 0.8.1-beta (arm64-darwin)

benchmark                    avg (min … max) p75 / p99    (min … top 1%)
-------------------------------------------- -------------------------------
• small
-------------------------------------------- -------------------------------
Buffer.from()                 337.23 ns/iter 339.36 ns        █▅            
                     (326.90 ns … 366.94 ns) 357.18 ns ▃▇▅▂▁▃▄██▅▅▂▂▂▂▁▁▁▁▁▁
llrt:codec.decodeFromBase64() 195.06 ns/iter 195.56 ns       ██             
                     (188.48 ns … 216.31 ns) 208.01 ns ▂▅▂▁▂▂██▃▃▂▃▂▂▁▁▁▁▁▁▁
Uint8Array.fromBase64()       163.14 ns/iter 163.57 ns       █▃             
                     (157.47 ns … 193.60 ns) 175.05 ns ▂▄▂▂▁▂██▂▂▂▂▂▁▂▁▁▁▁▁▁

summary
  Uint8Array.fromBase64()
   1.2x faster than llrt:codec.decodeFromBase64()
   2.07x faster than Buffer.from()

• middle
-------------------------------------------- -------------------------------
Buffer.from()                 342.16 ns/iter 343.75 ns         █            
                     (330.81 ns … 371.58 ns) 359.38 ns ▁▄▃▂▂▂▃██▅▄▃▂▂▂▁▁▁▁▁▁
llrt:codec.decodeFromBase64() 207.91 ns/iter 208.01 ns      █               
                     (200.68 ns … 260.01 ns) 225.34 ns ▂▃▂▁▅█▆▂▂▂▃▁▁▁▁▁▁▁▁▁▁
Uint8Array.fromBase64()       191.42 ns/iter 191.65 ns       █▂             
                     (184.57 ns … 214.36 ns) 204.59 ns ▂▄▁▁▂▆██▃▃▃▂▂▂▂▁▁▁▁▁▁

summary
  Uint8Array.fromBase64()
   1.09x faster than llrt:codec.decodeFromBase64()
   1.79x faster than Buffer.from()

• large
-------------------------------------------- -------------------------------
Buffer.from()                 373.19 ns/iter 374.76 ns         █            
                     (361.08 ns … 405.03 ns) 391.36 ns ▂▅▃▂▂▁▂██▆▅▂▂▃▂▁▂▁▂▁▁
llrt:codec.decodeFromBase64() 228.91 ns/iter 229.25 ns        █             
                     (221.44 ns … 259.03 ns) 241.21 ns ▂▃▂▁▁▁▅██▂▂▂▂▁▁▁▁▁▁▁▁
Uint8Array.fromBase64()       236.43 ns/iter 236.08 ns        █             
                     (228.52 ns … 273.68 ns) 248.78 ns ▂▁▁▁▁▁▃█▂▂▁▂▂▂▁▁▁▁▁▁▁

summary
  llrt:codec.decodeFromBase64()
   1.03x faster than Uint8Array.fromBase64()
   1.63x faster than Buffer.from()
benchmarks/mitata/encode_to_base64.js
% llrt benchmarks/mitata/encode_to_base64.js 
aGVsbG8gd29ybGQ=
aGVsbG8gd29ybGQ=
aGVsbG8gd29ybGQ=
aGVsbG8gd29ybGQ=
aGVsbG8gd29ybGQ=
clk: ~0.04 GHz
cpu: Apple M2
runtime: llrt 0.8.1-beta (arm64-darwin)

benchmark                   avg (min … max) p75 / p99    (min … top 1%)
------------------------------------------- -------------------------------
• small
------------------------------------------- -------------------------------
Buffer.from().toString()     430.54 ns/iter 433.84 ns  ▄     █             
                    (418.46 ns … 471.68 ns) 458.50 ns ▅█▃▃▄▂▇██▃▃▂▂▂▁▂▂▁▂▁▁
llrt:codec.encodeToBase64()  117.01 ns/iter 117.68 ns      █               
                    (113.28 ns … 135.50 ns) 126.71 ns ▃▅▃▁▁█▄▆▂▂▁▁▂▁▁▁▁▁▁▁▁
Uint8Array.toBase64()         56.57 ns/iter  56.40 ns      █               
                      (54.20 ns … 71.04 ns)  63.96 ns ▁▃▂▃██▅▂▂▁▁▁▁▁▁▁▁▁▁▁▁

summary
  Uint8Array.toBase64()
   2.07x faster than llrt:codec.encodeToBase64()
   7.61x faster than Buffer.from().toString()

• middle
------------------------------------------- -------------------------------
Buffer.from().toString()     443.18 ns/iter 445.07 ns          █▂          
                    (430.18 ns … 483.15 ns) 458.01 ns ▂▅▃▂▂▁▂▁▄██▆▃▃▂▂▂▁▁▂▁
llrt:codec.encodeToBase64()  122.57 ns/iter 123.05 ns      ▂█              
                    (118.65 ns … 137.70 ns) 131.84 ns ▅▃▂▁▁███▂▂▂▂▁▂▁▁▁▁▁▁▁
Uint8Array.toBase64()         73.94 ns/iter  74.22 ns     █                
                      (71.29 ns … 93.02 ns)  81.54 ns ▃▁▁▁█▂▅▁▂▂▁▁▁▁▁▁▁▁▁▁▁

summary
  Uint8Array.toBase64()
   1.66x faster than llrt:codec.encodeToBase64()
   5.99x faster than Buffer.from().toString()

• large
------------------------------------------- -------------------------------
Buffer.from().toString()     451.75 ns/iter 453.61 ns         ▄█           
                    (437.99 ns … 480.96 ns) 468.26 ns ▂▃▂▂▁▂▂▂███▅▃▃▂▂▂▂▁▁▁
llrt:codec.encodeToBase64()  132.16 ns/iter 132.08 ns       █▃             
                    (127.44 ns … 148.93 ns) 141.60 ns ▂▂▂▁▁▆██▂▂▂▂▂▂▁▁▁▁▁▁▁
Uint8Array.toBase64()         96.76 ns/iter  97.17 ns     ██▄              
                     (93.51 ns … 124.51 ns) 105.71 ns ▅▁▃▁███▂▂▂▂▁▂▁▁▁▁▁▁▁▁

summary
  Uint8Array.toBase64()
   1.37x faster than llrt:codec.encodeToBase64()
   4.67x faster than Buffer.from().toString()

• small
------------------------------------------- -------------------------------
Buffer.from().toString()     370.98 ns/iter 372.56 ns         █▆           
                    (359.86 ns … 404.30 ns) 387.21 ns ▂▇▂▂▂▂▂▃██▃▃▃▃▂▂▁▂▁▁▁
llrt:codec.encodeToBase64()  112.38 ns/iter 112.55 ns      █               
                    (108.40 ns … 132.32 ns) 120.85 ns ▂▂▂▁▁██▇▂▂▁▁▁▁▁▁▁▁▁▁▁

summary
  llrt:codec.encodeToBase64()
   3.3x faster than Buffer.from().toString()

• middle
------------------------------------------- -------------------------------
Buffer.from().toString()     376.26 ns/iter 376.95 ns         ▆█           
                    (364.26 ns … 402.34 ns) 389.65 ns ▁▂▁▁▁▁▁▂██▅▃▃▂▂▂▂▁▂▁▁
llrt:codec.encodeToBase64()  116.64 ns/iter 116.70 ns      █▂              
                    (112.55 ns … 138.92 ns) 126.95 ns ▂▂▂▁▄██▂▁▁▁▁▁▁▁▁▁▁▁▁▁

summary
  llrt:codec.encodeToBase64()
   3.23x faster than Buffer.from().toString()

• large
------------------------------------------- -------------------------------
Buffer.from().toString()     394.88 ns/iter 395.75 ns      █               
                    (381.10 ns … 434.81 ns) 424.80 ns ▂▂▁▁▂██▃▂▃▂▂▂▁▁▁▁▁▁▁▁
llrt:codec.encodeToBase64()  132.18 ns/iter 132.32 ns      ▄█▂             
                    (127.69 ns … 146.97 ns) 141.60 ns ▃▃▂▁▁███▂▂▂▂▂▁▁▂▁▁▁▁▁

summary
  llrt:codec.encodeToBase64()
   2.99x faster than Buffer.from().toString()

Checklist

  • Created unit tests in tests/unit and/or in Rust for my feature if needed
  • Ran make fix to format JS and apply Clippy auto fixes
  • Made sure my code didn't add any additional warnings: make check
  • Added relevant type info in types/ directory
  • Updated documentation if needed (API.md/README.md/Other)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@nabetti1720 nabetti1720 force-pushed the refactor/expose-base64-codec branch from b86a896 to b6ad8ba Compare May 5, 2026 23:25
@nabetti1720 nabetti1720 force-pushed the refactor/expose-base64-codec branch from b6ad8ba to 6478154 Compare May 5, 2026 23:28
@richarddavison
Copy link
Copy Markdown
Collaborator

Thanks for the PR! There are some timezone related changes that i think sliped in. Could you please also run criterion benches as well to see raw performance since a lot of this is called from rust fns.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants