-
Notifications
You must be signed in to change notification settings - Fork 1
NixOS rootfs configuration for Firecracker dev VMs #1167
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: devel
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,108 @@ | ||||||||||||||||||
| # fc-dev — NixOS configuration for Firecracker dev VMs. | ||||||||||||||||||
| # Not a real host — produces an ext4 rootfs image for Firecracker microVMs. | ||||||||||||||||||
| # | ||||||||||||||||||
| # Build rootfs: nix build .#fc-dev-rootfs | ||||||||||||||||||
| # Build kernel: nix build .#fc-dev-kernel | ||||||||||||||||||
| # | ||||||||||||||||||
| # The VM runs systemd as init, starts sshd on boot, and includes the full | ||||||||||||||||||
| # Bazel development toolchain from bazel-dev.nix. | ||||||||||||||||||
| { | ||||||||||||||||||
| modulesPath, | ||||||||||||||||||
| pkgs, | ||||||||||||||||||
| lib, | ||||||||||||||||||
| ... | ||||||||||||||||||
| }: | ||||||||||||||||||
| { | ||||||||||||||||||
| imports = [ | ||||||||||||||||||
| ../../modules/bazel-dev.nix | ||||||||||||||||||
| (modulesPath + "/profiles/minimal.nix") | ||||||||||||||||||
| ./make-rootfs.nix | ||||||||||||||||||
| ]; | ||||||||||||||||||
|
|
||||||||||||||||||
| # Firecracker boots the kernel directly — no bootloader. | ||||||||||||||||||
| boot.loader.grub.enable = false; | ||||||||||||||||||
| boot.initrd.enable = false; | ||||||||||||||||||
| boot.isContainer = false; | ||||||||||||||||||
| boot.kernelParams = [ | ||||||||||||||||||
| "console=ttyS0" | ||||||||||||||||||
| "reboot=k" | ||||||||||||||||||
| "panic=1" | ||||||||||||||||||
| ]; | ||||||||||||||||||
|
|
||||||||||||||||||
| # Minimal kernel modules for virtio (Firecracker's device model). | ||||||||||||||||||
| boot.initrd.availableKernelModules = lib.mkForce [ ]; | ||||||||||||||||||
| boot.kernelModules = [ | ||||||||||||||||||
| "virtio_blk" | ||||||||||||||||||
| "virtio_net" | ||||||||||||||||||
| "virtio_pci" | ||||||||||||||||||
| "virtio_mmio" | ||||||||||||||||||
| ]; | ||||||||||||||||||
|
|
||||||||||||||||||
| networking.hostName = "fc-dev"; | ||||||||||||||||||
| # Static network config — set by the VM pod entrypoint via kernel cmdline | ||||||||||||||||||
| # or DHCP. For simplicity, use a static config matching the TAP subnet. | ||||||||||||||||||
| networking.useDHCP = false; | ||||||||||||||||||
| networking.interfaces.eth0 = { | ||||||||||||||||||
| ipv4.addresses = [ | ||||||||||||||||||
| { | ||||||||||||||||||
| address = "10.0.0.2"; | ||||||||||||||||||
| prefixLength = 30; | ||||||||||||||||||
| } | ||||||||||||||||||
| ]; | ||||||||||||||||||
| }; | ||||||||||||||||||
| networking.defaultGateway = { | ||||||||||||||||||
| address = "10.0.0.1"; | ||||||||||||||||||
| interface = "eth0"; | ||||||||||||||||||
| }; | ||||||||||||||||||
| networking.nameservers = [ | ||||||||||||||||||
| "8.8.8.8" | ||||||||||||||||||
| "1.1.1.1" | ||||||||||||||||||
| ]; | ||||||||||||||||||
|
|
||||||||||||||||||
| # SSH access — the only way into the VM. | ||||||||||||||||||
| services.openssh = { | ||||||||||||||||||
| enable = true; | ||||||||||||||||||
| settings = { | ||||||||||||||||||
| PermitRootLogin = "prohibit-password"; | ||||||||||||||||||
| PasswordAuthentication = false; | ||||||||||||||||||
| }; | ||||||||||||||||||
| }; | ||||||||||||||||||
| # Placeholder key — real key injected via process_api CreateProcess or | ||||||||||||||||||
| # written to ~/.ssh/authorized_keys via WS command after boot. | ||||||||||||||||||
| users.users.root.openssh.authorizedKeys.keys = [ | ||||||||||||||||||
| "ssh-ed25519 AAAA-placeholder-replaced-at-runtime" | ||||||||||||||||||
| ]; | ||||||||||||||||||
|
Comment on lines
+70
to
+74
|
||||||||||||||||||
| # Placeholder key — real key injected via process_api CreateProcess or | |
| # written to ~/.ssh/authorized_keys via WS command after boot. | |
| users.users.root.openssh.authorizedKeys.keys = [ | |
| "ssh-ed25519 AAAA-placeholder-replaced-at-runtime" | |
| ]; | |
| # No baked-in SSH key — real key is injected via process_api CreateProcess | |
| # or written to ~/.ssh/authorized_keys via WS command after boot. | |
| users.users.root.openssh.authorizedKeys.keys = [ ]; |
Copilot
AI
Apr 3, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
bazel-dev.nix already adds python3 to environment.systemPackages; adding python313 here will pull two Python versions into the rootfs and can make python3 resolution ambiguous (and increases image size). Consider choosing a single Python (e.g., override python3 in bazel-dev.nix via an option/overlay, or remove one of the entries here).
| python313 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| # Produces system.build.ext4 — an ext4 filesystem image containing the | ||
| # full NixOS system closure, suitable for Firecracker's virtio-blk rootfs. | ||
| { | ||
| config, | ||
| pkgs, | ||
| lib, | ||
| modulesPath, | ||
| ... | ||
| }: | ||
| let | ||
| make-ext4-fs = import (modulesPath + "/../lib/make-ext4-fs.nix"); | ||
| in | ||
| { | ||
| system.build.ext4 = make-ext4-fs { | ||
| inherit pkgs lib; | ||
| inherit (pkgs) | ||
| e2fsprogs | ||
| libfaketime | ||
| perl | ||
| fakeroot | ||
| zstd | ||
| ; | ||
| storePaths = [ config.system.build.toplevel ]; | ||
| compressImage = false; | ||
| volumeLabel = "fc-dev-rootfs"; | ||
| populateImageCommands = '' | ||
| # System profile symlink — the only way to find the system closure. | ||
| # process_api spawns /nix/var/nix/profiles/system/init to start systemd, | ||
| # which then runs NixOS activation (creates /etc, /tmp, etc.). | ||
| mkdir -p ./files/nix/var/nix/profiles | ||
| ln -s ${config.system.build.toplevel} ./files/nix/var/nix/profiles/system | ||
|
|
||
| # NixOS activation checks for this marker. | ||
| mkdir -p ./files/etc | ||
| touch ./files/etc/NIXOS | ||
| ''; | ||
| }; | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fc-dev-kernelis set toconfig.boot.kernelPackages.kernel, which builds a kernel package output directory (containing multiple artifacts). The comment and PR description imply this output is a singlevmlinuxfile; consider either adjusting the documentation to reflect the directory output, or exposing a flake output that points directly at the intended kernel artifact path (e.g., thevmlinux/bzImagefile) to avoid ambiguity for provisioning scripts.