feat: integrate https://github.com/nilsnolde/osrm-bindings by nilsnolde · Pull Request #7485 · Project-OSRM/osrm-backend

nilsnolde · 2026-04-16T21:23:51Z

forgot to add here: @whytro did the real work during GSoC 2023. this PR is "just" integrating it into upstream.

getting to build linux was less of a hassle than I imagined. it's more of a hassle to review I guess.. my proposal: since it's mostly copy/paste from https://github.com/nilsnolde/osrm-bindings, it's not too bad to not review the code toooo much. one can somewhat trust the python packaging pipeline. I'll add more details later. IMO it's more important to understand how things work conceptually, so I'll add quite some more text to the PR description once this is a final PR. and of course feel free to ask any understanding question you have:) packaging is always awful, but honestly, python got really usable after they introduced (and widely supported) pyproject.toml.

General

nanobind for C++ binding (i.e. no setup.py, all CMake based), very close to libosrm API, only some plumbing python code
for PyPI distribution, we only build python 3.12 SABI (stable ABI) wheels, older version build from source

Repo Layout

everything is properly namespaced to python:

src/python/
├── CMakeLists.txt
├── README.md
├── include/
│   └── python/
│       ├── engineconfig_nb.hpp
│       ├── parameters/
│       ├── types/
│       └── utility/
├── osrm/
│   ├── __init__.py
│   ├── __main__.py
│   └── osrm_ext.pyi
└── src/
    ├── engineconfig_nb.cpp
    ├── osrm_nb.cpp
    ├── parameters/
    ├── types/
    └── utility/

test/python/
├── constants.py
├── test_index.py
├── test_match.py
├── test_nearest.py
├── test_route.py
├── test_table.py
├── test_tile.py
└── test_trip.py

Packaging Architecture

we distribute both sdist & wheels from release-monthly.yml
cibuildwheel to orchestrate the build, packaging, repairing (mostly vendor dynamic libs into the wheel), can be run locally too
cibuildwheel does the build directly on the system for macos & win; however, for linux it needs an old-ish glibc version which is the ABI compatibility barrier for python extensions. this project uses a manylinux_2_28 docker image based on almalinux 8. I forked https://github.com/pypa/manylinux a while ago to https://github.com/nilsnolde/manylinux, where I added the OSRM build dependencies for both aarch64 & x86: https://github.com/nilsnolde/manylinux/pkgs/container/manylinux.
we auth with PyPI over a "trusted publisher" so no secrets needed
versioning & publishing is integrated with release-monthly.yml

Open Questions

what should we do about the package name? osrm-bindings is ok-ish, osrm would of course be much nicer:)
we should probably open a osrm account on PyPI?
is it ok to transfer fix map matching test on windows nilsnolde/osrm-bindings#40 here and have someone from the community solve the win map-matching failure? I really don't feel like going back to win and figure out how to properly debug there;)

nilsnolde · 2026-04-16T21:27:04Z

+      { text: 'Node.js API', link: '/nodejs/api' },
+      { text: 'Python', items: [
+        { text: 'API', link: '/python/api' },
+        { text: 'Development', link: '/python/development' }


I tried to be really thorough, so I had claude add some python development and API docs. python doesn't work very well with vitepress with pulling out docstrings etc. it's mostly mkdocs these days in python land. not sure, but I could imagine mkdocs support JS docstrings as well. I guess you just switched to vitepress @DennisOSRM , not sure you'd be open to ditch it again potentially?

for reference, this is the mkdocs based documentation I configured for valhalla: https://valhalla.github.io/valhalla/

Sure, we can discuss the feasibility of switching. Perhaps independent of this PR, but whatever we choose should be visually on par with vitepress.

nilsnolde · 2026-04-16T21:32:55Z

@@ -0,0 +1,68 @@
+# osrm-bindings
+
+**Python bindings for [osrm-backend](https://github.com/Project-OSRM/osrm-backend) using [nanobind](https://github.com/wjakob/nanobind).**


this file is the project page on pypi.org

nilsnolde · 2026-04-16T21:34:58Z

+@ECHO OFF
+SETLOCAL EnableDelayedExpansion
+
+SET DATA_DIR=%CD%


could probably use the standard Makefile instead of duplicating the same logic here. I know there gmake for windows, could just work..

Perhaps after merging this change 😉

DennisOSRM · 2026-04-17T17:54:28Z

The PR probably needs a rebase onto latest master to pick up a fix for macOS ci.

nilsnolde · 2026-04-17T18:29:14Z

jeez, thanks, was just about to diagnose!

nilsnolde · 2026-04-29T15:22:28Z

@DennisOSRM this is ready for review again. I'll just have to revert the last commit before merging, where I'm testing if the wheels build properly, without uploading to pypi.

as for the "open questions": maybe it's best to continue releasing under my account. it's already on pypi and it's the most frictionless path IMO. pypi doesn't have the concept of "org" namespaces as e.g. npm does (otherwise we wouldn't have the mess with osrm pypi packages). but anyone wanting to be a co-admin, let me know, at least pypi lets me invite other admins for single packages.

nilsnolde · 2026-04-29T23:15:50Z

I also wanna say: in effect osrm is not buildable anymore with just system packages in many cases. since it removed all vendored packages and e.g. brew removed sol2 support, at least on macos vcpkg is a must. also on my manjaro linux sol2 is not in the official repos, only on the AUR (which is generally a bad habit to use). most others only have it on the very newest version like sid, rawhide & 26.06: https://pkgs.org/search/?q=sol2.

even if in theory vcpkg might be optional in cmake, in practice it's not on the majority of platforms. IMHO it was nicer before. what wasn't nice is that is was hard-vendored instead of submodule'd (in terms of maintenance). there's a way to have submodules (at least for header-only deps) to fall back to. I'm really more OSS than FOSS, but in such cases I'm all for "free". personally I really don't like being kinda locked in to smth like vcpkg (still, it's MS) without a hassle.

that's just my 2c from the packaging experience of the python bindings.

…to nn-py-bindings

nilsnolde · 2026-05-01T09:23:53Z

was mostly battling x64 macos in the past commits. now everything's working and all wheels are uploaded as artifacts: https://github.com/Project-OSRM/osrm-backend/actions/runs/25164784497.

next I wanna try linux aarch64, but after merging:)

DennisOSRM

Great work!

DennisOSRM · 2026-05-01T14:04:09Z

+CI wheel builds run inside a custom manylinux image
+([nilsnolde/manylinux](https://github.com/nilsnolde/manylinux), branch
+`osrm_python`) that ships vcpkg pre-bootstrapped at the SHA pinned in
+`vcpkg-configuration.json`, plus a pre-warmed vcpkg binary cache compiled


What happens when the vcpkg pin changes? Will that incur a slow build of the many linux image once?

mostly right. the barrier is basically whether any vcpkg update in upstream osrm-backend (only new version for whatever reason, new package etc) would break the OSRM compilation or python tests with the previous vcpkg still present in the previous manylinux image. if it does, the PR workflow will fail, I'll need to workflow_dispatch with the osrm-backend commit sha introducing that vcpkg change (i.e. build a new tagged image). if it doesn't, we don't have to care much about them being not in sync currently.

it's a bummer but kinda fair. the python builds failing won't be too obnoxious. they're the last steps to run, and only on a selection of jobs.

I do feel responsible for this now, but I invited at least you @DennisOSRM as collaborator in case I'm not available. eventually we could move the repo to the Project-OSRM org?

DennisOSRM · 2026-05-02T18:17:20Z

+@ECHO OFF
+SETLOCAL EnableDelayedExpansion
+
+SET DATA_DIR=%CD%


Perhaps after merging this change 😉

nilsnolde · 2026-05-03T12:26:15Z

thanks @DennisOSRM for the quick review. there was a master conflict, also changed a tiny thing in 4ed0a95.

nilsnolde · 2026-05-03T17:24:00Z

urgh, are you open to a force merge (if you have to) after I resolve this conflict and merge master @DennisOSRM ? lots of conflicts 😅

DennisOSRM · 2026-05-03T17:43:02Z

Thanks so much for the contribution. Glad, we could get it in this weekend.

linux works

7458ee1

nilsnolde commented Apr 16, 2026

View reviewed changes

add to agent.md

d4e62f9

nilsnolde commented Apr 16, 2026

View reviewed changes

nilsnolde changed the title ~~integrate https://github.com/nilsnolde/osrm-bindings~~ feat: integrate https://github.com/nilsnolde/osrm-bindings Apr 16, 2026

nilsnolde added 17 commits April 16, 2026 23:38

trigger CI

9fe055b

try CI

77de8bd

try windows

a752d92

trigger CI

e16b525

more ci

54ec941

suppress clang warnings

0dd6b0d

suppress more

91855bc

fix mac & win

cfa1065

fix path to conanrun.bat

3f3e452

add sdist so that version resolves properly for linux

3713f44

fix sdist to wheel

2493aeb

debug

d2f5e1a

try embedding in huge workflow

a3d2ef7

trigger CI

da8e068

minor fix

ab38ab8

huh, needs explicit vesion

3c5770f

more excluded for sdist

062c020

osx diagnostics

216bc0d

nilsnolde added 4 commits April 17, 2026 20:29

Merge remote-tracking branch 'origin/master' into nn-py-bindings

7dbde9a

trigger CI

427b420

more ci

a1209b0

try again macos

89b2322

nilsnolde requested a review from DennisOSRM April 29, 2026 15:22

nilsnolde added 3 commits April 29, 2026 17:25

fix linux

a6ade64

fix specifier

c7fcf89

fix both win & macos ci

246dddb

nilsnolde added 7 commits April 30, 2026 01:55

some renames and remove non-existent library from delvewheel command

38d3d78

make all macos-15 runners actually target macos 15.0

7294070

Merge branch 'master' into nn-py-bindings

5717e5e

oops

307b0a5

Merge branch 'nn-py-bindings' of github.com:nilsnolde/osrm-backend in…

708670d

…to nn-py-bindings

trigger CI

8a08909

that stupid x64 mac..

f9f54a6

revert ci testing stuff

bdaa43d

DennisOSRM approved these changes May 2, 2026

View reviewed changes

nilsnolde added 2 commits May 3, 2026 14:20

Merge remote-tracking branch 'origin/master' into nn-py-bindings

e926d2b

don't fetch full history, v6 removed that bug

4ed0a95

Merge remote-tracking branch 'origin/master' into nn-py-bindings

1920ce0

DennisOSRM approved these changes May 3, 2026

View reviewed changes

DennisOSRM enabled auto-merge May 3, 2026 17:22

Merge remote-tracking branch 'origin/master' into nn-py-bindings

b22002b

auto-merge was automatically disabled May 3, 2026 17:25
Head branch was pushed to by a user without write access

DennisOSRM added this pull request to the merge queue May 3, 2026

Merged via the queue into Project-OSRM:master with commit 4d04b38 May 3, 2026
1 check passed

nilsnolde deleted the nn-py-bindings branch May 3, 2026 17:46

		@@ -0,0 +1,68 @@
		# osrm-bindings

		Python bindings for [osrm-backend](https://github.com/Project-OSRM/osrm-backend) using [nanobind](https://github.com/wjakob/nanobind).

Uh oh!

Conversation

nilsnolde commented Apr 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

DennisOSRM commented Apr 17, 2026

Uh oh!

nilsnolde commented Apr 17, 2026

Uh oh!

nilsnolde commented Apr 29, 2026

Uh oh!

nilsnolde commented Apr 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nilsnolde commented May 1, 2026

Uh oh!

DennisOSRM left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nilsnolde May 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nilsnolde commented May 3, 2026

Uh oh!

nilsnolde commented May 3, 2026

Uh oh!

Uh oh!

DennisOSRM commented May 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

nilsnolde commented Apr 16, 2026 •

edited

Loading

nilsnolde commented Apr 29, 2026 •

edited

Loading

nilsnolde May 3, 2026 •

edited

Loading

DennisOSRM commented May 3, 2026 •

edited

Loading