Add experimental uv resolver for Python lockfiles

[Liam-Deacon]

Summary

This PR adds an experimental uv-backed resolver option for Python user lockfile generation.

Pants still generates and consumes Pex lockfiles, but when opted in it uses uv pip compile to pre-resolve pins, then uses pex lock create --no-transitive to materialize the Pex lock.

Capability	Before	After (opt-in)
Lockfile generator	Pex (pex lock create)	still Pex
Resolver used for the solve step	pip	uv (PubGrub)
Extra flags	pip/pex flags only	uv flags via [uv].args

Motivation

Issue: #20679 — faster, more ergonomic lockfile generation by leveraging uv.

How it works

requirements (targets)
        |
        |  (opt-in) uv pip compile
        v
 pinned requirements.txt
        |
        |  pex lock create --no-transitive
        v
     Pex lockfile

Usage

1) Enable resolves and select the uv resolver

[python]
enable_resolves = true
lockfile_resolver = "uv"  # <-- opt in

# uv currently requires a single Python major/minor
interpreter_constraints = ["CPython==3.11.*"]

[python.resolves]
python-default = "3rdparty/python/default.lock"

# uv is currently supported for strict/sources, not universal
[python.resolves_to_lock_style]
python-default = "strict"

2) Pass arbitrary uv flags (passthrough)

Use the standard args pattern (same as other downloadable tools in Pants):

[uv]
# Example requested in the issue: prefer the first matching index.
args = ["--index-strategy", "unsafe-first-match"]

Equivalent CLI:

pants generate-lockfiles --resolve=python-default \
  --python-lockfile-resolver=uv \
  --uv-args='--index-strategy unsafe-first-match'

Current limitations (intentional safeguards)

Limitation	Reason
No lock_style = "universal"	uv pip compile resolves for a single interpreter environment today
No complete_platforms	not wired through in this initial integration
Interpreter constraints must select one Python major/minor	ensures uv resolves for exactly one target interpreter
Per-resolve sources / overrides / excludes not yet modeled in uv step	avoids silently changing semantics

Benchmark (real-world example)

Environment:

• Python: CPython==3.11.*
• Lock style: strict
• Resolver modes compared: pip vs uv
• Input: annotated + sorted concat of:
  • ~/repos/python-mono/3rdparty/python/requirements.txt
  • ~/repos/python-mono/3rdparty/python/requirements-dev.txt
• Excluded from the benchmark input to keep it public/reproducible:
  • any pip.antarcticaam.app/private references
  • blpapi, StressVaR, module-wrapper, pytest-localstack

Timing (/usr/bin/time -p, Pantsd disabled; repeated to show warm-cache behavior):

Method:

• Buildroot: /tmp/pants-uv-lockfile-benchmark (isolated), using python_requirements(...) pointing at the filtered requirements.in.
• Each run uses a fresh PANTS_WORKDIR to avoid Pants's local process cache reusing results.
• Tool caches persist across repeats:
  • PANTS_GLOBAL_PEX_ROOT=/tmp/pants-uv-lockfile-benchmark/cache/pex_root
  • PIP_CACHE_DIR=/tmp/pants-uv-lockfile-benchmark/cache/pip_cache
  • UV_CACHE_DIR=/tmp/pants-uv-lockfile-benchmark/cache/uv_cache
• “cold” = cache dirs removed before the run; “warm” = immediate repeat with caches retained.

Mode	Run	real (s)	user (s)	sys (s)
pip	cold	323.58	144.50	40.03
pip	warm (repeat)	117.52	136.15	29.27
uv	cold	90.49	108.48	29.05
uv	warm (repeat)	101.41	121.51	32.02

Speedup (lower is better):

• Cold: pip/uv = 3.58x
• Warm: pip/uv = 1.16x

Notes:

• Results vary with network / platform and the resolver's cache state. The intent is to show that the uv pre-solve materially reduces cold-cache solve time in this real-world-ish set.

Additional visibility (uv mode only):

Phase	Observed duration
uv pip compile pre-solve (previous run-set)	25.39s

In this run-set, uv reduced end-to-end wall time vs pip (especially cold-cache). Warm-cache repeats narrow the gap, which suggests resolver overhead is a larger fraction of first-time generation than subsequent runs.

Code changes

Area	Change
src/python/pants/backend/python/subsystems/setup.py	Adds [python].lockfile_resolver = {pip,uv}
src/python/pants/backend/python/subsystems/uv.py	Adds hermetic uv tool + [uv].args passthrough
src/python/pants/backend/python/goals/lockfile.py	Runs uv pip compile pre-step (opt-in) then pex lock create --no-transitive
src/python/pants/backend/python/goals/lockfile_test.py	Adds unit tests for uv-mode validation behavior
docs/docs/python/overview/lockfiles.mdx	Documents uv resolver configuration and flags

[jasonwbarnett]

Wow, this is a really creative design 🤩

[cburroughs]

Thanks for the contribution. We've just branched for 2.31.x, so merging this pull request now will come out in 2.32.x, please move the release notes updates to docs/notes/2.32.x.md if that's appropriate.

[Liam-Deacon]

Hi maintainers 👋 — just rebased this onto the latest main (resolving conflicts with the new sync feature in generate-lockfiles and the updated execute_process_or_raise API).

Summary of what this PR adds:

• A new [python].lockfile_resolver option (default: pex, opt-in: uv) that uses uv pip compile to pre-resolve fully-pinned requirements before calling pex lock create --no-transitive.
• Motivated by pants generate-lockfiles to use uv for dependency resolution #20679 — uv's --unsafe-first-match resolver behaviour lets large repos with pinned transitive deps generate lockfiles that pex's resolver struggles with.
• The uv subsystem integration (src/python/pants/backend/python/subsystems/uv.py) is already present in Pants, so no new subsystem plumbing is needed.
• Release note added to docs/notes/2.32.x.md.

Known limitations (documented in the option help):

• Requires a single Python major/minor interpreter constraint (e.g. CPython==3.11.*)
• Does not yet support lock_style="universal" or complete_platforms

Would love a review from anyone familiar with the Python backend / lockfile generation — @cburroughs, @stuhood, @benjy, or whoever is currently shepherding the lockfile work. Happy to iterate!

[benjyw]

Thanks for this! Excited to review. I will try and take a look in the next day or two. I somehow missed this back when it was first created in, looks like, December. Sorry about that!

Meanwhile, if you used LLMs in the making of this PR, can you elaborate on that? We are working on an official AI disclosure policy but haven't yet put one into practice.

[benjyw]

Before I review the meat of this PR: a couple of seemingly unrelated changes got pulled in here.

[benjyw]

Why these changes?

They aren't wrong, but AFAICT they are irrelevant to the purpose of this PR, and they don't seem to have any benefit. These strings are going to be consumed either here or immediately after, when they go out of scope.

[benjyw]

?

[Niccolum]

Hello, @Liam-Deacon @benjyw

Can I help you somehow with PR?

[Niccolum]

One question - why universal style is unsupported? I see no reason not to support him here.