Skip to content

architecture.md

Latest commit

 

History

History
356 lines (224 loc) · 7.64 KB

architecture.md

File metadata and controls

356 lines (224 loc) · 7.64 KB

LocalAIStack Architecture

1. Purpose and Scope

LocalAIStack is an infrastructure-oriented software stack designed to manage local AI workstations as long-lived, reproducible computing environments.

This document describes:

  • The architectural layers of LocalAIStack
  • Core responsibilities of each layer
  • Key design decisions and trade-offs
  • Extension and evolution principles

It does not describe installation steps or end-user workflows.

2. Architectural Principles

LocalAIStack follows these core architectural principles:

2.1 Local-First Infrastructure

All core functionality must work without external network access.

External services (e.g. model registries, translation APIs) are optional and replaceable.

2.2 Hardware-Aware Control

Software capabilities are constrained by detected hardware.

LocalAIStack never assumes uniform hardware and never exposes functionality that cannot be reliably supported by the current machine.

2.3 Layered and Decoupled Design

Each layer has a clearly defined responsibility:

  • Control logic does not execute workloads
  • Runtime does not decide policy
  • Modules do not manage global state

2.4 Deterministic and Reproducible State

Given the same hardware and configuration:

  • Installation results must be deterministic
  • Runtime behavior must be reproducible
  • Version drift must be observable and reversible

2.5 Vendor and Model Neutrality

LocalAIStack does not encode assumptions about:

  • GPU vendors
  • Model providers
  • Framework ecosystems
  • Cloud platforms

3. High-Level Architecture

┌──────────────────────────────┐
│          Interfaces          │
│  Web UI / CLI / API          │
└──────────────▲───────────────┘
               │
┌──────────────┴───────────────┐
│        Control Layer          │
│  Policy, State, Resolution   │
└──────────────▲───────────────┘
               │
┌──────────────┴───────────────┐
│        Runtime Layer          │
│  Containers / Native Exec    │
└──────────────▲───────────────┘
               │
┌──────────────┴───────────────┐
│        Software Modules       │
│  Languages / AI / Services   │
└──────────────────────────────┘

4. Layer Responsibilities

4.1 Interfaces Layer

Responsibilities

  • User interaction
  • Status visualization
  • Operation triggering

Components

  • Web-based UI
  • Command-line interface (CLI)
  • Internal API (REST or gRPC)

Non-Responsibilities

  • No direct package installation
  • No hardware probing
  • No policy decisions

Interfaces only issue intent, never perform actions directly.

4.2 Control Layer (Core)

The Control Layer is the core of LocalAIStack.

Responsibilities

  • Hardware detection and classification
  • Capability policy evaluation
  • Software resolution and dependency planning
  • State tracking and reconciliation
  • Upgrade and rollback orchestration

Subcomponents

4.2.1 Hardware Detector

Detects and normalizes hardware attributes:

  • CPU cores and topology
  • System memory
  • GPU model, memory, and interconnects
  • Storage characteristics

Outputs a hardware profile consumed by the policy engine.

4.2.2 Capability Policy Engine

Maps hardware profiles to allowed capabilities.

Example policies:

  • Maximum supported model size
  • Allowed inference runtimes
  • Parallelism constraints
  • Memory and VRAM thresholds

Policies are declarative and versioned.

4.2.3 Software Resolver

Determines:

  • Which software modules are installable
  • Compatible versions and combinations
  • Required runtime backends
  • Conflicting dependencies

The resolver produces an execution plan, not actions.

4.2.4 State Manager

Maintains system state:

  • Installed modules
  • Versions and hashes
  • Runtime status
  • Configuration overrides

State is persistent and auditable.

4.3 Runtime Layer

The Runtime Layer is responsible for executing software, not deciding what should exist.

Execution Modes

  • Container-based (default)
  • Native execution (performance-critical paths)

Responsibilities

  • Process lifecycle management
  • Resource isolation
  • Log collection
  • Health reporting

Non-Responsibilities

  • No dependency resolution
  • No policy enforcement
  • No user-facing decisions

4.4 Software Modules Layer

Software modules represent installable units.

Module Categories

  • Programming language environments
  • AI inference engines
  • AI frameworks
  • Infrastructure services
  • AI applications
  • Developer tools

Module Definition

Each module is described by a manifest containing:

  • Metadata
  • Hardware requirements
  • Dependencies
  • Runtime constraints
  • Exposed interfaces
  • Optional integrity metadata (checksum/signature)

Modules are self-describing and independently versioned.

The module registry loads manifests, validates schema/integrity, and resolves dependency graphs to produce install plans with explicit version selection.

5. Model Management Architecture

Model management is treated as a first-class concern.

Responsibilities

  • Model metadata tracking
  • Storage layout management
  • Integrity verification
  • Hardware compatibility checks

Non-Responsibilities

  • No automatic model execution
  • No preference for specific providers

Models are resources, not services.

6. Hardware Capability Tiers

LocalAIStack classifies machines into capability tiers.

Example:

  • Tier 1: Entry-level inference
  • Tier 2: Mid-range local LLM workloads
  • Tier 3: Multi-GPU and large-model systems

Tier definitions are policy-driven, not hardcoded.

7. Internationalization Architecture

Design Approach

  • All UI text is key-based
  • Language resolution occurs at the interface layer
  • AI-assisted translation is optional and cacheable

Constraints

  • No runtime dependency on external translation services
  • Translations must not affect system behavior

8. Extension Model

LocalAIStack is designed to be extended without modifying core logic.

Extension Points

  • New software modules
  • Additional runtime backends
  • Alternative interfaces
  • Custom policy sets

Extensions are loaded via manifests and registered with the Control Layer.

9. Failure Handling and Recovery

LocalAIStack treats failure as a first-class condition.

Strategies

  • Atomic operations
  • Explicit error states
  • Partial installation detection
  • Version pinning and rollback

Silent failure is considered a bug.

10. Non-Goals

LocalAIStack explicitly does not aim to be:

  • A cloud orchestration system
  • A training cluster manager
  • A hosted SaaS platform
  • A proprietary appliance OS

11. Evolution Strategy

LocalAIStack is expected to evolve in phases:

  1. Stable mid-range local inference workflows
  2. Broader application ecosystem support
  3. Multi-node and collaborative scenarios (optional)

Backward compatibility and migration paths are mandatory concerns.

12. Summary

LocalAIStack is designed as infrastructure, not as an application bundle.

Its architecture prioritizes:

  • Predictability over convenience
  • Explicit policy over implicit behavior
  • Long-term maintainability over short-term optimization