Live bootstrap to guix by vxtls · Pull Request #578 · fosslinux/live-bootstrap

vxtls · 2026-03-15T01:44:32Z

No description provided.

… callers

…flow - add --stage0-image in rootfs.py for qemu to boot an existing kernel-bootstrap image - when --stage0-image is combined with --build-guix-also, update BUILD_GUIX_ALSO in image config and sync /steps-guix into the image - require stage0 /init to contain guix handoff marker instead of patching /init implicitly - add run_steps_guix_if_requested() to make_bootable-generated /init so rebooted stage0 images can enter steps-guix directly - run /steps-guix/0.sh with bash in after.sh - make script-generator start mode convention-based: /steps stays kaem-first, alternate roots (e.g. /steps-guix) start in bash - remove redundant explicit /steps config-root argument from seed/preseeded/reconfigure script-generator calls

…the build

Drop the manual make -C opcodes i386-gen call from pass1.sh and rely on default_src_compile instead. This avoids failures when opcodes/ is not yet available in the current build context (make: *** opcodes: No such file or directory) and matches the stable top-level recursive build flow used elsewhere in the project.

- add gcc-15.2.0 to steps-guix manifest after binutils-2.41 - keep full gcc-15.2.0 pass1 src_prepare cleanup/regeneration flow - switch configure/build/install to kernel toolchain bootstrap mode: --without-headers, --enable-multilib, all-gcc, all-target-libgcc - install into /kernel-toolchain and prioritize /kernel-toolchain/bin in PATH - add missing gcc distfiles entry for SARIF spec - include decDPD helper files used during src_prepare

The stage1 gcc build runs with --without-headers, but all-target-libgcc was still pulling in gthr-default.h and failing on missing pthread.h. Adjust steps-guix/gcc-15.2.0/pass1.sh configure flags to match a headers-free bootstrap profile: - add --disable-threads - add --disable-shared - add --disable-libssp - add --disable-libsanitizer - add --disable-libquadmath - add --disable-libatomic - add --disable-libgomp Keep the existing multilib bootstrap flow: - make all-gcc - make all-target-libgcc - make install-gcc - make install-target-libgcc

- add steps-guix/argp-standalone-1.4.1/pass1.sh - run autoreconf before configure - build/install with standard flow into /kernel-toolchain - wire argp-standalone-1.4.1 into steps-guix manifest after gcc - prepare argp for later kernel-side elfutils dependency

When booting with --stage0-image, mirror ports can change between runs (e.g. file:// -> transient SimpleMirror port), but the reused image kept stale MIRRORS/MIRRORS_LEN values in /steps/bootstrap.cfg. Update stage0-work image preparation to patch bootstrap.cfg on each run: - rewrite MIRRORS and MIRRORS_LEN from current CLI mirrors - keep existing --build-guix-also handoff checks/sync behavior This fixes guest downloads trying old 10.0.2.2:<stale-port> endpoints during steps-guix builds.

…stsuite argp-standalone pass1 builds in a separate build directory. Its testsuite compiles sources that include <argp.h>, but without an explicit include path the header in the source root is not found and build fails. Set: - CPPFLAGS=-I/Users/luoyanpan/CLionProjects/guix/live-bootstrap/.. in src_configure so testsuite objects can resolve argp.h during the normal phase.

…olchain instead of relying on make install

…into manifest

…generation and fts/argp env wiring

…argp env wiring

…al LIBS and setting host/build + kernel-toolchain env

…ied sha256 and wire it before elfutils

…with static zlib

…d build stages

…hs, and disable unused-but-set-variable as error

…compress, and debuginfod client

…mpile via rpath-link and explicit LIBS

…ntamination

…t untouched

vxtls · 2026-03-19T00:07:57Z

@Googulator

EDIT: this is what Guix did originally: https://cgit.git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/patches/glibc-bootstrap-system.patch?h=v0.8 Doesn't seem easily adaptable to musl.

So, should we build a separate musl to achieve this?
Because when it comes to this issue (sh not in PATH), building a separate musl might help avoid further potential problems?
Or do you think it’s enough to just modify the source code of these Bootstrap components?

Googulator · 2026-03-19T00:17:22Z

Oops... I accidentally edited your comment instead of mine.

Anyway, a shell is indeed present at this stage - it just needs to be found via $PATH, rather than hardcoded.

I'm currently testing a possible fix: bootstrap-system.patch

Googulator · 2026-03-19T00:48:55Z

Seems like the fix worked, but bootar fails now with a new error:

starting phase `separate-from-pid1'
build process now running as PID 6
error: in phase 'separate-from-pid1': uncaught exception:
unbound-variable "module-lookup" "Unbound variable: ~S" (WAIT_ANY) #f
phase `separate-from-pid1' failed after 0.0 seconds
phase `separate-from-pid1' succeeded after 0.0 seconds
Backtrace:
starting phase `set-SOURCE-DATE-EPOCH'
phase `set-SOURCE-DATE-EPOCH' succeeded after 0.0 seconds
starting phase `set-paths'
environment variable `PATH' set to `/gnu/store/ksbb2yvmb55i965j2w1y7v4iv5kq90dk-guile-bootstrap-2.0/bin'
phase `set-paths' succeeded after 0.0 seconds
starting phase `install-locale'
error: in phase 'install-locale': uncaught exception:
unbound-variable "module-lookup" "Unbound variable: ~S" (LC_ADDRESS) #f
phase `install-locale' failed after 0.0 seconds
Backtrace:
           6 (primitive-load "/gnu/store/a0blpdn8d03mqc6lk80chicyd62…")
In ice-9/boot-9.scm:
    829:9  5 (catch srfi-34 #<procedure 5c34d0 at guix/build/gnu-bu…> …)
           8 (primitive-load "/gnu/store/a0blpdn8d03mqc6lk80chicyd62…")
In srfi/srfi-1.scm:
In ice-9/boot-9.scm:
    829:9  7 (catch srfi-34 #<procedure 5c34d0 at guix/build/gnu-bu…> …)
In srfi/srfi-1.scm:
    640:9  4 (for-each #<procedure 5c34b0 at guix/build/gnu-build-s…> …)
In ice-9/boot-9.scm:
    841:4  3 (with-throw-handler _ _ _)
In guix/build/gnu-build-system.scm:
   993:23  2 (_)
    88:17  1 (separate-from-pid1 #:separate-from-pid1? _)
In ice-9/boot-9.scm:
    640:9  6 (for-each #<procedure 5c34b0 at guix/build/gnu-build-s…> …)
   752:25  0 (dispatch-exception _ _ _)

In ice-9/boot-9.scm:
    841:4  5 (with-throw-handler _ _ _)
In guix/build/gnu-build-system.scm:
   993:23  4 (_)
ice-9/boot-9.scm:752:25: In procedure dispatch-exception:
In procedure module-lookup: Unbound variable: WAIT_ANY
In ice-9/boot-9.scm:

vxtls · 2026-03-19T00:58:45Z

I'm currently testing a possible fix: bootstrap-system.patch

use-execvp-sh-in-scm-system.patch
I'll add this patch; it should work the same way, but it will be more robust.

UPDATE:
use-execvp-sh-in-scm-system.patch

…ndependent bootstrap-guile

…-status semantics

vxtls · 2026-03-19T01:49:37Z

Seems like the fix worked, but bootar fails now with a new error:

starting phase `separate-from-pid1'
build process now running as PID 6
error: in phase 'separate-from-pid1': uncaught exception:
unbound-variable "module-lookup" "Unbound variable: ~S" (WAIT_ANY) #f
phase `separate-from-pid1' failed after 0.0 seconds
phase `separate-from-pid1' succeeded after 0.0 seconds
Backtrace:
starting phase `set-SOURCE-DATE-EPOCH'
phase `set-SOURCE-DATE-EPOCH' succeeded after 0.0 seconds
starting phase `set-paths'
environment variable `PATH' set to `/gnu/store/ksbb2yvmb55i965j2w1y7v4iv5kq90dk-guile-bootstrap-2.0/bin'
phase `set-paths' succeeded after 0.0 seconds
starting phase `install-locale'
error: in phase 'install-locale': uncaught exception:
unbound-variable "module-lookup" "Unbound variable: ~S" (LC_ADDRESS) #f
phase `install-locale' failed after 0.0 seconds
Backtrace:
           6 (primitive-load "/gnu/store/a0blpdn8d03mqc6lk80chicyd62…")
In ice-9/boot-9.scm:
    829:9  5 (catch srfi-34 #<procedure 5c34d0 at guix/build/gnu-bu…> …)
           8 (primitive-load "/gnu/store/a0blpdn8d03mqc6lk80chicyd62…")
In srfi/srfi-1.scm:
In ice-9/boot-9.scm:
    829:9  7 (catch srfi-34 #<procedure 5c34d0 at guix/build/gnu-bu…> …)
In srfi/srfi-1.scm:
    640:9  4 (for-each #<procedure 5c34b0 at guix/build/gnu-build-s…> …)
In ice-9/boot-9.scm:
    841:4  3 (with-throw-handler _ _ _)
In guix/build/gnu-build-system.scm:
   993:23  2 (_)
    88:17  1 (separate-from-pid1 #:separate-from-pid1? _)
In ice-9/boot-9.scm:
    640:9  6 (for-each #<procedure 5c34b0 at guix/build/gnu-build-s…> …)
   752:25  0 (dispatch-exception _ _ _)

In ice-9/boot-9.scm:
    841:4  5 (with-throw-handler _ _ _)
In guix/build/gnu-build-system.scm:
   993:23  4 (_)
ice-9/boot-9.scm:752:25: In procedure dispatch-exception:
In procedure module-lookup: Unbound variable: WAIT_ANY
In ice-9/boot-9.scm:

yes, i have this problem also, This may cause by WAIT_ANY is not defind, WAIT_ANY only exist in glibc but not musl
so we need to patch guix, set WAIT_ANY to -1 directly

Googulator · 2026-03-19T02:07:10Z

missing-defines.patch

This patch is needed to avoid errors due to WAIT_ANY and glibc-specific LC_* macros not being defined in musl.
With this + your 2nd execvp patch, the build got further, but somehow the /bin/sh issue reappeared, probably due to a different Guile environment getting called (maybe the host guile inside an isolated namespace?) - removing (system "export > $NIX_BUILD_TOP/environment-variables") fixed that, and now bootar builds successfully.

… for bootstrap compatibility

vxtls · 2026-03-19T02:41:28Z

removing (system "export > $NIX_BUILD_TOP/environment-variables") fixed that, and now bootar builds successfully.

i mean won't removing this cause any problems? Have you checked what this variable is for?
But, it’s actually quite an achievement to have gotten this far.
There are likely to be many more mistakes down the road.

Googulator · 2026-03-19T02:52:55Z

;; Dump the environment variables as a shell script,
;; for handy debugging.

This comment doesn't sound like it's actually used as part of the bootstrap, it's just for debugging purposes.

Googulator · 2026-03-19T03:52:24Z

fixnums-remove-new-asserts.patch

Yet another patch, this time to remove some new assertions that were added to the RNRS fixnums module between Guile 2.0 and 2.2. The version of bootar used in guix runs afoul of these with some BZ2 archives (notably tcc 0.9.27).

fosslinux

A lot of this looks quite good. Please don't be put off by the number of comments.

I must say, I am reasonably impressed that you managed to pull this off. This is a fairly significant set of work.

There are some things that I have intentionally not reviewed because I will be replacing what they do anyways before this is merged.

the location of KERNEL_SYSROOT should be changed. It shouldn't be defined in every build script, and shouldn't be at /kernel-toolchain (it should live somewhere else, like /usr/x86_64-guix-linux-musl, or possibly it should install everything to /usr/bin directory, and libs to /usr/lib/x86_64-guix-linux-musl and includes to /usr/include/i686-guix-linux-musl.
Any miscellaneous inputs that need to be packaged, like the bootstrap-seeds, could be installed to a more sensible place on the system, perhaps /opt/guix? Then we could have a subdirectory like /opt/guix/bootstrap-seeds.
KERNEL_TARGET also shouldn't be set in every script.
for the bootstrap seeds for guix, it would be good to identify that in the package name, for example instead of coreutils-8.30, coreutils-guix-seed-8.30.
in many places, you have something like

src_prepare() {
    default
}

this is totally redundant, default is, literally, the default :)

similarly, in many places also, you have a pattern like

local var
var=something
X=$var ...

this doesn't really help for clarity, unless it is a particularly complex expression, and in most of the places used is unnecessary

in many places PATH/ LD_LIBRARY_PATH /PKG_CONFIG is redefined for a configure script, this is almost certainly unnecessary. same with CC, AR, RANLIB, which I also doubt are necessary
PKG_CONFIG_LIBDIR appears here and there -- I've never heard of this variable before, what's up with that?
It is conventional for shared patches and files copied from other packages to be symlinked.
Every new file needs SPDX lines with author and license information. Please be careful with this, it is important to us that everything has proper copyright attribution.
Of course, eventually we will need to audit this for pregenerated files (but that can be later.)
(This is not something you need to fix.) There are a number of variables that are reused here that should be pulled up to global, like GUILE_LOAD_PATH, GUILE_SYSTEM_PATH, PKG_CONFIG_PATH, PKG_CONFIG_PATH
I'm a bit confused about this pattern --host="${host_triplet}" etc, is the default not sufficient?
I don't understand all the NETWORK_READY stuff. Could you briefly explain what it's doing?

seed/script-generator.c

steps-guix/argp-standalone-1.4.1/pass1.sh

fosslinux · 2026-03-18T05:37:03Z

steps-guix/avahi-0.8/pass1.sh

+src_configure() {
+    local host_triplet pkg_config_path
+    host_triplet="$(gcc -dumpmachine)"
+    pkg_config_path="${LIBDIR}/pkgconfig:${PREFIX}/lib/pkgconfig:${PREFIX}/share/pkgconfig"


Note to self: I should make PKG_CONFIG_PATH be set globally. (This is my fault.)

fosslinux · 2026-03-19T11:12:08Z

steps-guix/mescc-tools-static-stripped-0.5.2-i686-linux/pass1.sh

+src_unpack() {
+    mkdir -p mescc-tools-static-stripped-0.5.2-i686-linux
+}
+src_prepare() { :; }


this should almost certainly be unnecessary

same as above, if don't have this stub, it will fail because it missing sources and distfiles

fosslinux · 2026-03-19T11:13:49Z

steps-guix/sqlite-3510200/pass1.sh

+src_unpack() {
+    local url fname
+
+    url="$(awk 'NR==1 { print $2 }' ../sources)"
+    fname="$(awk 'NR==1 { print $4 }' ../sources)"
+
+    if [ -z "${fname}" ]; then
+        fname="$(basename "${url}")"
+    fi
+
+    cp "${DISTFILES}/${fname}" .
+    unzip -q "${fname}"
+
+    dirname="${fname%.zip}"
+}


Hmm, let's add proper zip processing support in helpers.sh instead.

fosslinux · 2026-03-19T11:14:57Z

steps-guix/unzip-6.0/pass1.sh

I think we can get by without unzip. We have bsdtar from libarchive, which is able to deal with DEFLATE (.zip) files.

fosslinux · 2026-03-19T11:16:50Z

steps/jump/linux.sh

    if [ "${BARE_METAL}" = True ]; then
        kexec -l "/boot/vmlinuz" \
-            --append="root=/dev/sda1 rootfstype=ext3 init=/init rw rootwait"
+            --append="root=/dev/sda1 init=/init rw rootwait"


Why this change?

let kernel to decide the filesystem itself

AFAIK this was needed so the kernel doesn't try to look for ext4 extensions, because ext4 created by a newer kernel tends to crash older kernels.

but, why we need to switch to old kernel?
btw, Has guix pull --bootstrap run successfully yet?

steps/jump/break.sh

fosslinux · 2026-03-19T11:25:00Z

I'm going to make a list of things that I'd like to see in separate PRs, whether made by you, me or someone else;

the changes to external.img
Wrt this,

no, it just a change from ext4 to a raw byte stream(Of course, as Googulator suggests, I can also use tar, the implementation is actually quite simple, I don't know what you think, i mean the custom format is way more simple and easy to implement) As soon as I have some free time, I'll submit a new PR.

I am quite pro this change away from ext4, although I do have some questions about how exactly this operates, but they should be a bit clearer once it is separated from this guix work :)
As to tar vs this custom format, I don't really mind, tar is also somewhat simple, but this format seems to work too and is even simpler... and it would be simple to implement even in hardware or on a very simple microprocessor....
zip support in helpers.sh
PKG_CONFIG_PATH as a global variable in helpers.sh, same with various guile variables

… arithmetic module

…cks in pass1.sh

…ass1.sh

…r and make rootfs resume dispatch direct

… helpers.sh

…nd move build script to pass2

vxtls added 30 commits February 18, 2026 10:14

script-generator: support explicit manifest/config roots and wire all…

21ddab3

… callers

script-generator: remove grammar that M2-Planet didn't support

71efe6b

script-generator: fix env SRCDIR to jump to steps-guix folder and do …

6e19398

…the build

mirror.sh: add steps-guix dir

c54da75

steps-guix/argp-standalone-1.4.1/sources: newline at end of file

2f22444

steps-guix: build argp-standalone natively into /kernel-toolchain

6022208

fix(argp-standalone): install argp artifacts manually into /kernel-to…

e727bcc

…olchain instead of relying on make install

feat(musl-fts): add steps-guix musl-fts-1.2.7 build step and wire it …

92afa6f

…into manifest

musl-fts-1.2.7: update checksum

afd79c2

feat(elfutils): add steps-guix elfutils-0.194 pass1 with autotools re…

9515d30

…generation and fts/argp env wiring

fix(elfutils): switch to native compiler for pass1 while keeping fts/…

cb1a8a6

…argp env wiring

fix(elfutils): avoid configure runtime probe failure by dropping glob…

bcd3be0

…al LIBS and setting host/build + kernel-toolchain env

feat(musl-obstack): add steps-guix musl-obstack-1.2.3 step with verif…

51d4773

…ied sha256 and wire it before elfutils

fix(elfutils): build static-only to avoid libelf.so TEXTREL failures …

fee7d7d

…with static zlib

fix(elfutils): enforce static linking via LDFLAGS env in configure an…

280eaba

…d build stages

fix(elfutils): drop static LDFLAGS, keep configure-time toolchain pat…

d0f7f6f

…hs, and disable unused-but-set-variable as error

fix(elfutils): add --disable-textrelcheck

0066bbd

fix(elfutils): add FNM_EXTMATCH compatibility patch across strip, elf…

8f29302

…compress, and debuginfod client

fix(elfutils): resolve libdw transitive fts/obstack linkage in src_co…

8703b08

…mpile via rpath-link and explicit LIBS

feat(steps-guix): build linux64 kernel and switch via jump with backup

6b10401

fix(linux-6.12.74): keep host tools on /usr/bin and avoid cross ld co…

e821a06

…ntamination

fix(steps-guix): remove forced ext3 rootfstype in linux64 handover

156cd66

refactor(steps-guix): use improve-based linux64 handover to keep /ini…

55be03a

…t untouched

vxtls added 2 commits March 18, 2026 21:03

fix(guile-2.2.4): make scm_system use execvp(sh,-c,cmd) for /bin/sh-i…

8bed6f4

…ndependent bootstrap-guile

fix(guile-2.2.4): harden scm_system execvp(sh) path and preserve wait…

6a5ba94

…-status semantics

fix(guile-2.2.4): add fallback WAIT_ANY and LC_* constants in posix.c…

a20351b

… for bootstrap compatibility

fosslinux reviewed Mar 19, 2026

View reviewed changes

vxtls added 16 commits March 19, 2026 08:24

fix(guile-2.2.4): remove fixnums range-guard assertion blocks in rnrs…

2816e89

… arithmetic module

fix(argp): remove debugging code

00e3f91

fix(bash): remove debugging code

4f66a58

refactor(guix-1.5.0): drop redundant post-sed and post-patch grep che…

8c41939

…cks in pass1.sh

refactor(guile-2.2.4): remove redundant post-sed grep assertions in p…

ef6abd5

…ass1.sh

fix(guix-hash): chenge build sequence

ea9c3d6

fix(failure-flow): keep interactive shell handling in script-generato…

d709331

…r and make rootfs resume dispatch direct

refactor(script-generator): move resume init mount/network logic into…

4af0359

… helpers.sh

fix(guix-daemon-and-pull): remove unneccessary dryrun

3dcc4ec

refactor(guix-1.5.0): [not tested]split seed prep into improve step a…

8bb3da9

…nd move build script to pass2

fix(steps-guix/jump): remove check

4674fc5

fix(steps/break): remove debug code

c55fd54

fix(guix-daemon-and-pull): remove extra check

a170296

fix(linux-6.12.74): Pass env vars to make as args instead of exporting

eb44754

fix(steps-guix-jump): harden linux64 kexec handoff

3817181

fix: install bash back

c0e18b0

Conversation

vxtls commented Mar 15, 2026

Uh oh!

vxtls commented Mar 19, 2026

Uh oh!

Googulator commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Googulator commented Mar 19, 2026

Uh oh!

vxtls commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vxtls commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Googulator commented Mar 19, 2026

Uh oh!

vxtls commented Mar 19, 2026

Uh oh!

Googulator commented Mar 19, 2026

Uh oh!

Googulator commented Mar 19, 2026

Uh oh!

fosslinux left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

fosslinux commented Mar 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Googulator commented Mar 19, 2026 •

edited

Loading

vxtls commented Mar 19, 2026 •

edited

Loading

vxtls commented Mar 19, 2026 •

edited

Loading