Skip to content

Further optimization #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
youknowone merged 26 commits into from
Apr 20, 2026
Merged

Further optimization #6

Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
a57a504
Decouple liblzma-sys backend from bindgen
youknowone Apr 13, 2026
c6236a1
trim
youknowone Apr 13, 2026
5142a3d
Refine xz internals and restore systest perf builds
youknowone Apr 13, 2026
1f964de
Tidy trivial c2rust naming remnants
youknowone Apr 13, 2026
3627f85
Tidy remaining low-risk c2rust naming
youknowone Apr 13, 2026
1ebdc7b
Use ctest 0.5.1 raw identifier support
youknowone Apr 13, 2026
a93e21b
Reduce mutable statics and narrow unsafe helpers
youknowone Apr 13, 2026
d8d77f1
revert crc64
youknowone Apr 14, 2026
2d7323e
ifix
youknowone Apr 14, 2026
19e4fe3
cleanup
youknowone Apr 14, 2026
ce93d55
Reduce internal panic paths
youknowone Apr 16, 2026
0314bc7
Refine LZMA hot paths and backend comparison tooling
youknowone Apr 16, 2026
834e3fd
Tune performance probes and hot paths
youknowone Apr 17, 2026
c4b047d
Trim hot-path function-pointer checks
youknowone Apr 17, 2026
2211802
remove redundant
youknowone Apr 17, 2026
dc080a9
edition 2024
youknowone Apr 17, 2026
1c627e0
xz 5.8.3
youknowone Apr 18, 2026
146611d
Optino<fn>
youknowone Apr 18, 2026
e8a96e6
Replace c2rust state hashes with named control flow
youknowone Apr 18, 2026
b36bdd0
Refine allocator defaults and optimum helper flow
youknowone Apr 18, 2026
d1ae9b8
refine
youknowone Apr 19, 2026
db7ed70
Revert top-level src changes to origin
youknowone Apr 20, 2026
1a94603
1.88
youknowone Apr 20, 2026
674511a
fix systest
youknowone Apr 20, 2026
e944c6f
let-else
youknowone Apr 20, 2026
7a41ac9
Restore cold literal-after helpers in optimum encoder
youknowone Apr 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ jobs:
matrix:
include:
- os: ubuntu-latest
rust: '1.63' # msrv
rust: '1.85' # msrv
- os: ubuntu-latest
rust: stable
- os: ubuntu-latest
Expand Down Expand Up @@ -59,7 +59,7 @@ jobs:
run: echo LZMA_API_STATIC=1 >> $GITHUB_ENV
shell: bash
- name: copy Cargo.lock for msrv check
if: matrix.rust == '1.63'
if: matrix.rust == '1.85'
run: |
cp Cargo.lock.msrv Cargo.lock
- run: |
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/webassembly.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ jobs:
name: Run test
run: |
curl -L --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/cargo-bins/cargo-binstall/main/install-from-binstall-release.sh | bash
cargo binstall -y --force wasm-bindgen-cli@0.2.115
cargo binstall -y --force wasm-bindgen-cli@0.2.118
cargo test --locked --target ${{ matrix.target }} --features wasm --release
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # For ease GitHub API rate limits
Expand Down
82 changes: 41 additions & 41 deletions Cargo.lock.msrv
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 5 additions & 2 deletions Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ in-memory encoding/decoding. forked from xz2.
"""
categories = ["compression", "api-bindings"]
edition = "2021"
rust-version = "1.63"
rust-version = "1.85"
exclude = [".github/"]

[workspace]
Expand Down Expand Up @@ -51,7 +51,7 @@ wasm-bindgen-test = "0.3"
default = ["xz"]
xz = ["dep:xz"]
xz-sys = ["dep:xz-sys"]
liblzma-sys = ["dep:liblzma-sys", "liblzma-sys/bindgen"]
liblzma-sys = ["dep:liblzma-sys"]
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🌐 Web query:

In Cargo, when the same crate appears in [dependencies]and[dev-dependencies], are features unified for benches/examples/tests, and do default features from the dev-dependency stay enabled?

💡 Result:

Yes, when the same crate appears in both [dependencies] and [dev-dependencies], Cargo unifies the features for builds that include dev-dependencies (benches, examples, tests). This uses the union of features specified in both sections. Default features from the dev-dependency declaration remain enabled unless explicitly disabled there (via default-features = false). For normal builds (cargo build on lib/bin), only [dependencies] features apply (dev-deps ignored). This behavior depends on the feature resolver version (default "2" for edition 2021+, controlled by top-level resolver field). Resolver v2 avoids unwanted unification but includes dev-dep features when building dev targets like tests/examples/benches. Example from docs: [dependencies] serde = {version = "1.0", default-features = false} [dev-dependencies] serde = {version = "1.0", features = ["std"]} - cargo build: serde without features (defaults off). - cargo test: serde with "std" (union, defaults from dev-dep enabled since not disabled).

Citations:

🏁 Script executed:

# Check the structure of Cargo.toml, particularly around the lines mentioned
sed -n '20,60p' Cargo.toml | cat -n

Repository: youknowone/xz-rs

Length of output: 1680

liblzma-sys still enables bindgen for dev targets.

Line 45 defines a target-specific dev-dependency liblzma-sys = { path = "liblzma-sys" } without default-features = false. When benches/examples/tests are built with the liblzma-sys feature, Cargo unifies features across dependencies and dev-dependencies, re-enabling the default bindgen feature from the dev-dependency. This defeats the goal of disabling it at Line 25 and keeps the libclang requirement around for those builds.

♻️ Proposed fix 
 [target.'cfg(not(target_family = "wasm"))'.dev-dependencies]
 xz-sys = { path = "xz-sys" }
-liblzma-sys = { path = "liblzma-sys" }
+liblzma-sys = { path = "liblzma-sys", default-features = false }
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@Cargo.toml` at line 54, The dev-dependency entry for liblzma-sys still pulls
in its default features (re-enabling bindgen); update the liblzma-sys dependency
declarations so they set default-features = false (both the target-specific
dev-dependency and the liblzma-sys = ["dep:liblzma-sys"] mapping) to prevent the
bindgen feature from being activated for benches/examples/tests; look for the
liblzma-sys entries and add default-features = false (and adjust to a table form
if needed) to ensure the bindgen feature stays disabled.

static = ["liblzma-sys?/static"]
parallel = ["xz-sys?/parallel", "liblzma-sys?/parallel", "num_cpus"]
bindgen = ["liblzma-sys?/bindgen"] # only affects liblzma-sys
Expand All @@ -64,3 +64,6 @@ thin-lto = ["liblzma-sys?/thin-lto"]

[package.metadata.docs.rs]
features = ["parallel"]

[lints.rust]
unsafe_op_in_unsafe_fn = "allow"
35 changes: 24 additions & 11 deletions benches/backend_comparison.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,21 +16,22 @@ compile_error!("backend_comparison bench requires `xz`, `xz-sys`, or `liblzma-sy
#[cfg(feature = "liblzma-sys")]
use liblzma_sys::{
lzma_crc32, lzma_crc64, lzma_easy_buffer_encode, lzma_stream_buffer_bound,
lzma_stream_buffer_decode, LZMA_CHECK_CRC64,
lzma_stream_buffer_decode, LZMA_CHECK_CRC64, LZMA_OK,
};
#[cfg(feature = "xz")]
use xz::check::{crc32_fast::lzma_crc32, crc64_fast::lzma_crc64};
#[cfg(feature = "xz")]
use xz::common::{
easy_buffer_encoder::lzma_easy_buffer_encode, stream_buffer_decoder::lzma_stream_buffer_decode,
stream_buffer_encoder::lzma_stream_buffer_bound,
use xz::{
check::{crc32_fast::lzma_crc32, crc64_fast::lzma_crc64},
common::{
easy_buffer_encoder::lzma_easy_buffer_encode,
stream_buffer_decoder::lzma_stream_buffer_decode,
stream_buffer_encoder::lzma_stream_buffer_bound,
},
types::{LZMA_CHECK_CRC64, LZMA_OK},
};
#[cfg(feature = "xz")]
use xz::types::LZMA_CHECK_CRC64;
#[cfg(feature = "xz-sys")]
use xz_sys::{
lzma_crc32, lzma_crc64, lzma_easy_buffer_encode, lzma_stream_buffer_bound,
lzma_stream_buffer_decode, LZMA_CHECK_CRC64,
lzma_stream_buffer_decode, LZMA_CHECK_CRC64, LZMA_OK,
};

#[cfg(feature = "xz")]
Expand All @@ -56,7 +57,7 @@ unsafe fn backend_encode(input: &[u8]) -> Vec<u8> {
let bound = lzma_stream_buffer_bound(input.len());
let mut out = vec![0u8; bound];
let mut out_pos: usize = 0;
lzma_easy_buffer_encode(
let ret = lzma_easy_buffer_encode(
6,
LZMA_CHECK_CRC64,
ptr::null(),
Expand All @@ -66,6 +67,7 @@ unsafe fn backend_encode(input: &[u8]) -> Vec<u8> {
&mut out_pos,
out.len(),
);
assert_eq!(ret, LZMA_OK, "{BACKEND_NAME} encode failed with {ret}");
out.truncate(out_pos);
out
}
Expand All @@ -75,7 +77,7 @@ unsafe fn backend_decode(compressed: &[u8], out_size: usize) -> Vec<u8> {
let mut memlimit = u64::MAX;
let mut in_pos = 0usize;
let mut out_pos = 0usize;
lzma_stream_buffer_decode(
let ret = lzma_stream_buffer_decode(
&mut memlimit,
0,
ptr::null(),
Expand All @@ -86,6 +88,17 @@ unsafe fn backend_decode(compressed: &[u8], out_size: usize) -> Vec<u8> {
&mut out_pos,
out.len(),
);
assert_eq!(ret, LZMA_OK, "{BACKEND_NAME} decode failed with {ret}");
assert_eq!(
in_pos,
compressed.len(),
"{BACKEND_NAME} decode left trailing input: consumed {in_pos} of {} bytes",
compressed.len()
);
assert_eq!(
out_pos, out_size,
"{BACKEND_NAME} decode produced {out_pos} bytes, expected {out_size}"
);
out.truncate(out_pos);
out
}
Expand Down
Loading