Skip to content

ci: publish multi-arch docker images as part of monthly release#7476

Open
tete17 wants to merge 1 commit intoProject-OSRM:masterfrom
tete17:ci/release-docker-images
Open

ci: publish multi-arch docker images as part of monthly release#7476
tete17 wants to merge 1 commit intoProject-OSRM:masterfrom
tete17:ci/release-docker-images

Conversation

@tete17
Copy link
Copy Markdown
Contributor

@tete17 tete17 commented Apr 13, 2026

Summary

  • The new monthly release pipeline (release-monthly.yml) pushes the release tag using the default GITHUB_TOKEN, which GitHub intentionally prevents from triggering downstream workflows. As a result, osrm-backend-docker.yml — which is gated on push: tags: 'v*' — never runs on automated releases, and no images land on ghcr.io for monthly versions.
  • The release workflow already handles this gotcha for osrm-backend.yml by explicitly calling gh workflow run ... --ref "$TAG". This PR applies the same pattern to the docker workflow so multi-arch images are built and published on every release.
  • No changes to the Dockerfiles or build matrix — osrm-backend-docker.yml already produces linux/amd64 + linux/arm64 images for debian and alpine across normal/debug/assertions variants. The fix is purely trigger wiring.

Changes

  • .github/workflows/osrm-backend-docker.yml: add workflow_dispatch with a required ref input, and point actions/checkout at inputs.ref || github.ref so docker/metadata-action still emits tag-based image tags (e.g. ghcr.io/project-osrm/osrm-backend:v26.4.0) when invoked via dispatch.
  • .github/workflows/release-monthly.yml:
    • New "Trigger docker image build for tag" step mirroring the existing osrm-backend.yml dispatch.
    • Refactor the pre-publish wait loop into a wait_for_workflow helper that now blocks npm publish on both osrm-backend.yml and osrm-backend-docker.yml reaching completed/success. A release is only "done" when all advertised artifacts exist.

Why not push the tag with a PAT instead?

That would re-enable the existing push: tags: 'v*' trigger without any docker-workflow changes, but it requires maintaining a PAT secret, grants broader write access than GITHUB_TOKEN, and silently re-triggers every tag-gated workflow. Following the explicit-dispatch pattern already used for osrm-backend.yml has a smaller blast radius.

Test plan

  • actionlint .github/workflows/release-monthly.yml .github/workflows/osrm-backend-docker.yml
  • Dry run: invoke release-monthly.yml via workflow_dispatch with a sentinel version_override on a test branch (with the final npm publish step temporarily disabled), and confirm both osrm-backend.yml and osrm-backend-docker.yml are dispatched and waited on.
  • After the dry run, check https://github.com/Project-OSRM/osrm-backend/pkgs/container/osrm-backend for new tags (vX.Y.Z, vX.Y.Z-debug, vX.Y.Z-assertions, plus alpine equivalents).
  • docker buildx imagetools inspect ghcr.io/project-osrm/osrm-backend:vX.Y.Z — manifest list should contain both linux/amd64 and linux/arm64.
  • Smoke test: docker run --rm --platform linux/amd64 ghcr.io/project-osrm/osrm-backend:vX.Y.Z osrm-extract --help, repeat with --platform linux/arm64 under QEMU.
  • Clean up the throwaway tag, release, and package versions.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the monthly release workflow so that Docker multi-arch images are published to GHCR as part of automated monthly releases, despite GitHub’s GITHUB_TOKEN not triggering tag-based workflows.

Changes:

  • Dispatch osrm-backend-docker.yml explicitly from release-monthly.yml on the release tag.
  • Refactor the “wait for CI” logic to wait for both CI and Docker workflows before npm publish.
  • Add a workflow_dispatch ref input to the Docker workflow and check out inputs.ref (fallback to github.ref).

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
.github/workflows/release-monthly.yml Explicitly dispatches docker workflow and waits for both workflows to finish before publishing to npm.
.github/workflows/osrm-backend-docker.yml Adds dispatch input for tag ref and checks out the requested ref for builds.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/release-monthly.yml Outdated
Comment on lines +212 to +214
RUN_JSON="$(gh run list --workflow="$workflow" --limit 50 --json databaseId,status,conclusion,headSha,displayTitle 2>/dev/null || echo '[]')"
MATCHING_RUN="$(echo "$RUN_JSON" | jq -r --arg sha "$TAG_SHA" '.[] | select(.headSha == $sha) | @json' | head -1)"

Copy link

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The wait loop matches the first run whose headSha equals the tag’s commit. Since osrm-backend.yml also runs on push to master, there is typically already a completed master run with the same headSha (the commit you just tagged). In that case this step can return success before the newly dispatched tag build has even started, and npm publish may proceed without the release assets/images being produced. Consider filtering for the dispatched run (e.g., include event/headBranch/createdAt in gh run list --json ... and select the run that corresponds to the tag dispatch, rather than any run for the same SHA).

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is valuable feedback. We need to make sure that we wait for the assets to be available.

Comment thread .github/workflows/release-monthly.yml Outdated
WORKFLOWS="osrm-backend.yml osrm-backend-docker.yml"
echo "Waiting for workflows ($WORKFLOWS) to complete for tag $TAG (commit $TAG_SHA)"

MAX_WAIT=3600 # 1 hour; multi-arch arm64 under QEMU is slow
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure this suffices. The docker build take somewhere around five hours to complete.

What we probably need to do is to start the docker build at the end of this run, but not to wait for it to complete.

Copy link
Copy Markdown
Collaborator

@DennisOSRM DennisOSRM left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now, thinking about it a bit more, it could be solvable by a oneliner in realease-monthly.yml

gh workflow run osrm-backend-docker.yml --ref "${{ steps.version.outputs.tag }}" --field ref="{{ steps.version.output.tag }}"

That should dispatch the docker job without waiting for it, which is fine.

@tete17 tete17 force-pushed the ci/release-docker-images branch 3 times, most recently from c87d175 to d64d5c7 Compare April 20, 2026 16:39
The new release-monthly.yml workflow pushes the release tag with the
default GITHUB_TOKEN, which GitHub suppresses from triggering downstream
workflows. As a result, osrm-backend-docker.yml (gated on
`push: tags: v*`) never fired on monthly releases and no multi-arch
images were published to ghcr.io. The release workflow already handles
this for osrm-backend.yml via `gh workflow run`; apply the same pattern
to the docker workflow.

Multi-arch docker builds (amd64 + arm64 under QEMU across
debian/alpine × normal/debug/assertions) take several hours, so the
dispatch is fire-and-forget at the end of the release job — we don't
block npm publish on it.

Also fix the CI wait loop: osrm-backend.yml runs on both push-to-master
and the tag dispatch, sharing the same headSha. Filtering solely by
headSha could match the already-completed master run and return success
before the tag-dispatched build even started. Filter by
event=workflow_dispatch in addition to headSha so only the explicitly
dispatched tag run is considered.
@tete17 tete17 force-pushed the ci/release-docker-images branch from d64d5c7 to 4717cfd Compare April 27, 2026 15:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants