Add relative rpath for runtime library discovery#3377
Closed
yulazariy wants to merge 3 commits intovalkey-io:unstablefrom
Closed
Add relative rpath for runtime library discovery#3377yulazariy wants to merge 3 commits intovalkey-io:unstablefrom
yulazariy wants to merge 3 commits intovalkey-io:unstablefrom
Conversation
Add $ORIGIN/../lib (Linux) and @loader_path/../lib (macOS) to the rpath configuration. This allows binaries to find shared libraries in a relative lib directory, enabling more flexible installation layouts without requiring LD_LIBRARY_PATH or DYLD_LIBRARY_PATH. This is particularly useful for relocatable installations where binaries and libraries are installed in a self-contained directory structure. Signed-off-by: yulazariy <yulazari@amazon.com>
yulazariy
force-pushed
the
add-relative-rpath
branch
from
5eb2989 to
4d33d40
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## unstable #3377 +/- ##
============================================
- Coverage 74.53% 74.49% -0.04%
============================================
Files 130 130
Lines 72731 72748 +17
============================================
- Hits 54208 54197 -11
- Misses 18523 18551 +28 🚀 New features to boost your workflow:
|
Contributor
sarthakaggarwal97
left a comment
sarthakaggarwal97
left a comment
There was a problem hiding this comment.
Seeing some errors in CI, might be worth looking into
lto1: fatal error: open debug.o failed: No such file or directory
compilation terminated.
lto-wrapper: fatal error: cc returned 1 exit status
compilation terminated.
/usr/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:692: valkey-server] Error 1
make[1]: *** Waiting for unfinished jobs....
Contributor
Author
Will take a look but not sure how it is related to the change as it only update the rpath. |
Replace .make-prerequisites with .make-deps-built to eliminate race condition where distclean runs in parallel with linking, causing LTO errors when object files are deleted during link-time optimization. Changes: - Remove automatic distclean trigger from .make-prerequisites - Add .make-deps-built marker file that only builds dependencies - Use order-only prerequisites (|) to avoid unnecessary rebuilds - Users must manually run 'make distclean' when build settings change This fixes build failures like: lto1: fatal error: open debug.o failed: No such file or directory /usr/bin/ld: cannot find zipmap.o: No such file or directory Signed-off-by: yulazariy <yulazari@amazon.com>
yulazariy
force-pushed
the
add-relative-rpath
branch
from
94a2c07 to
86c33df
Compare
Replace .make-prerequisites with .make-deps-built to eliminate race condition where distclean runs in parallel with linking, causing LTO errors when object files are deleted during link-time optimization. Also export MALLOC variable so unit tests can properly detect the allocator and avoid trying to use jemalloc headers when MALLOC=libc. Changes: - Remove automatic distclean trigger from .make-prerequisites - Add .make-deps-built marker file that only builds dependencies - Use order-only prerequisites (|) to avoid unnecessary rebuilds - Export MALLOC variable for sub-makes - Update unit test Makefile to use .make-deps-built This fixes build failures like: lto1: fatal error: open debug.o failed: No such file or directory /usr/bin/ld: cannot find zipmap.o: No such file or directory fatal error: 'jemalloc/jemalloc.h' file not found (on macOS) Signed-off-by: yulazariy <yulazari@amazon.com>
Contributor
Author
|
will be fixed in a different PR #3392 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add relative rpath for runtime library discovery
Summary
This PR adds relative rpath entries to the build configuration, allowing Valkey binaries to discover shared
libraries in a relative lib directory at runtime.
Changes
• Added $ORIGIN/../lib rpath for Linux builds
• Added @loader_path/../lib rpath for macOS builds
• Maintains existing absolute rpath entries for backward compatibility
Motivation
Currently, Valkey binaries use absolute rpaths, which limits installation flexibility. With relative rpaths,
binaries can find libraries in a predictable location relative to their own path, enabling:
• Relocatable installations without modifying environment variables
• Self-contained directory structures (e.g., bin/ and lib/ siblings)
• Easier packaging and distribution
Testing
Verified on macOS that the rpath is correctly set using
otool -l:path @loader_path/../lib (offset 12)