Skip to content

Add QEMU q35 VM backend and Azure deployment#66

Draft
motatoes wants to merge 3 commits intomainfrom
feat/qemu-backend-azure
Draft

Add QEMU q35 VM backend and Azure deployment#66
motatoes wants to merge 3 commits intomainfrom
feat/qemu-backend-azure

Conversation

@motatoes
Copy link
Contributor

Summary

  • Adds internal/qemu/ package as an alternative VM backend to Firecracker, using QEMU q35 machine type with KVM acceleration
  • Worker selects backend via OPENSANDBOX_VM_BACKEND env var ("qemu" or "firecracker", default: "qemu")
  • Adds Azure deployment scripts (deploy/azure/) for quick dev testing on D-series VMs
  • Decouples WorkspaceAutosaver from Firecracker via SyncFSer interface

What's in the QEMU package

File Purpose
qmp.go QMP (QEMU Machine Protocol) client for VM lifecycle control
agent_client.go gRPC agent client using AF_VSOCK (kernel vhost-vsock)
manager.go sandbox.Manager implementation — create, kill, list, hibernate/wake
snapshot.go Hibernate via QMP migrate, wake via -incoming, cold boot fallback
network.go TAP device + iptables DNAT/masquerade (shared with Firecracker approach)
rootfs.go ext4 rootfs/workspace preparation (identical to Firecracker)

Key differences from Firecracker

  • vsock: AF_VSOCK via kernel's vhost-vsock (no UDS+CONNECT handshake)
  • Snapshots: Single migration file (not separate mem + vmstate)
  • Boot args: Uses root=/dev/vda rw and no pci=off (virtio-pci needs PCI bus)
  • Process model: QEMU spawned as child process (vs Firecracker REST API)

Tested on Azure Standard_D48as_v6 (AMD EPYC 9V74)

  • test-exec.ts: 23/23 passed — exec.run, streaming, kill, file write/read, network, apt-get
  • test-commands.ts: 28/29 passed — stderr, exit codes, 10K line output, env vars, pipes, 10 concurrent commands
  • test-file-ops.ts: 24/24 passed — 1MB files, special chars, nested dirs, delete, listing

Test plan

  • GOOS=linux go vet ./internal/qemu/... passes
  • Sandbox creation via API
  • Synchronous exec (/exec/run)
  • Streaming exec (exec.start with WebSocket)
  • File read/write via SDK
  • Network connectivity (ping, DNS, apt-get)
  • Concurrent commands on same sandbox
  • Hibernate/wake cycle
  • Concurrent sandbox creation (needs timeout tuning)

🤖 Generated with Claude Code

Replace Firecracker microVMs with QEMU q35+KVM as an alternative VM backend.
The worker selects the backend via OPENSANDBOX_VM_BACKEND env var (default: "qemu").

New internal/qemu/ package:
- QMP client for VM lifecycle control (stop/cont/quit/migrate)
- AF_VSOCK agent communication (replaces Firecracker's UDS+CONNECT protocol)
- Hibernate/wake via QMP migrate (single state file vs Firecracker's mem+vmstate)
- TAP networking with iptables DNAT/masquerade (same as Firecracker)
- Manager implementing sandbox.Manager interface

Key changes:
- cmd/worker/main.go: backend selection switch (qemu vs firecracker)
- internal/worker/autosave.go: SyncFSer interface decouples from Firecracker
- internal/config/config.go: VMBackend and QEMUBin config fields
- deploy/azure/: deployment and host setup scripts for Azure VMs

Tested on Azure Standard_D48as_v6 (AMD EPYC):
- test-exec.ts: 23/23 passed
- test-commands.ts: 28/29 passed
- test-file-ops.ts: 24/24 passed

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@vercel
Copy link

vercel bot commented Mar 13, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
opensandbox Ready Ready Preview, Comment Mar 13, 2026 6:27pm

Request Review

Fixes 8 issues encountered during initial QEMU deployment:
- Use host's generic Ubuntu kernel instead of dead S3 URL
- Extract guest kernel modules (vsock, overlay) for rootfs
- Fix rootfs build args and module patching in deploy script
- Fix DB seed (correct table name, UUID IDs, key_prefix column)
- Stop services before binary copy to avoid "text file busy"
- Fix systemd LimitNPROC=unlimited to LimitNPROC=infinity
- Use X-API-Key header instead of Authorization: Bearer for tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant