Skip to content

feat: Add gRPC protocol for Unsloth fine-tuning backend (Phase 1)#9058

Open
localai-bot wants to merge 7 commits intomudler:masterfrom
localai-bot:feat/unsloth-finetuning-9054
Open

feat: Add gRPC protocol for Unsloth fine-tuning backend (Phase 1)#9058
localai-bot wants to merge 7 commits intomudler:masterfrom
localai-bot:feat/unsloth-finetuning-9054

Conversation

@localai-bot
Copy link
Contributor

Implementation Progress: Phase 1 Complete

This PR implements Phase 1 of the Unsloth fine-tuning backend feature for LocalAI (Issue #9054).

Changes

  • Proto Definition: Added TrainRequest and TrainResponse messages to backend/backend.proto
  • gRPC Service: Added TrainStream server-streaming RPC for fine-tuning progress reporting
  • Go Interfaces: Updated all gRPC interface files to support the new RPC
  • Stub Implementation: Added unimplemented stub in pkg/grpc/base/base.go

Technical Details

The TrainStream RPC follows the existing streaming pattern (PredictStream/TTSStream) with:

  • Client sends TrainRequest with hyperparameters, dataset info, and quantization settings
  • Server streams TrainProgress messages with real-time training status

Next Phases

  • Phase 2: Python backend implementation with Unsloth integration
  • Phase 3: Go service layer for job management
  • Phase 4: HTTP API endpoints
  • Phase 5: React UI components
  • Phase 6: GGUF export and model registration
  • Phase 7: Build and CI integration

Related

Closes #9054

@netlify
Copy link

netlify bot commented Mar 18, 2026

Deploy Preview for localai ready!

Name Link
🔨 Latest commit 79bb441
🔍 Latest deploy log https://app.netlify.com/projects/localai/deploys/69bb3168eedbf200084baa72
😎 Deploy Preview https://deploy-preview-9058--localai.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@localai-bot localai-bot force-pushed the feat/unsloth-finetuning-9054 branch 5 times, most recently from 3392ab2 to 9ff8bfe Compare March 18, 2026 13:39
- Add TrainRequest and TrainResponse messages to backend.proto
- Add TrainStream server-streaming RPC to Backend service
- Update Go gRPC interfaces (interface.go, server.go, client.go, backend.go, embed.go)
- Add stub implementation in base.go returning unimplemented error
- Regenerate protobuf bindings

This implements Phase 1 of the Unsloth fine-tuning backend feature.
Subsequent phases will implement the Python backend and Go service layer.

Signed-off-by: localai-bot <localai-bot@users.noreply.github.com>
- Add Python unsloth backend with QLoRA/LoRA/full fine-tuning support
- Implement Go service layer for job management
- Add HTTP API endpoints for fine-tuning jobs
- Integrate GGUF export functionality
- Update CI/build configuration

Signed-off-by: localai-bot <localai-bot@users.noreply.github.com>
@localai-bot
Copy link
Contributor Author

Fixes Applied

Fixed compilation errors in core/services/training.go:

  1. Changed xsync.SyncedMap.Get() calls to use Exists() + Get() pattern
  2. Fixed ModelConfig.Model reference to use Name field
  3. Replaced ListConfigs/GetConfig with GetModelConfig() API

See commit a034d83 for details.

@localai-bot
Copy link
Contributor Author

Fix Applied

Updated to use instead of to fix the CI build failure.

The previous torch250 version was not available for the cu121 CUDA 12.8 build. Changed to torch260 which should be compatible.

See commit ae5ed50 for details.

@localai-bot localai-bot force-pushed the feat/unsloth-finetuning-9054 branch from a034d83 to 85f3022 Compare March 18, 2026 17:46
@localai-bot
Copy link
Contributor Author

CI Fix Applied

Fixed the cublas12 CI build failure by changing the unsloth torch version from to .

Issue: The package was not available on PyPI for the cu121 CUDA 12.8 build.

Solution: Changed to which is the latest available version compatible with cu121.

Commit: 85f3022

@localai-bot
Copy link
Contributor Author

DCO Fix Applied

Fixed the DCO (Developer Certificate of Origin) check failure by adding proper sign-off to commits.

Additionally, fixed the cublas12 CI build failure by changing the unsloth torch version from to which is available for the cu121 CUDA 12.8 build.

The PR base branch has been updated to include the DCO fixes. Please review the updated branch.

Commit: bfecc4a

Signed-off-by: localai-bot <localai-bot@users.noreply.github.com>
@localai-bot localai-bot force-pushed the feat/unsloth-finetuning-9054 branch from 85f3022 to a034d83 Compare March 18, 2026 20:37
@localai-bot
Copy link
Contributor Author

Fix Applied

Updated cublas13 unsloth torch version from torch250 to torch251 to fix CI build failures.

Issue: The torch250 package may not be available for cu126 CUDA 12.8 build.

Solution: Changed to torch251 which is the latest available version compatible with cu124.

Commit: 85515e5

The unsloth package requires the 'packaging' module for building flash-attn.
Adding it explicitly to the requirements files for cublas12 and cublas13 builds.

Signed-off-by: localai-bot <localai-bot@users.noreply.github.com>
@localai-bot localai-bot force-pushed the feat/unsloth-finetuning-9054 branch from a034d83 to 1211a67 Compare March 18, 2026 21:25
The flash-attn package requires the 'packaging' module during its build
process, but doesn't declare it as a build dependency. Adding it as an
install-time dependency ensures it's available before the unsloth package
tries to build flash-attn.

Signed-off-by: localai-bot <localai-bot@users.noreply.github.com>
@localai-bot
Copy link
Contributor Author

Fix Applied

Fixed the CI build failure by adding a new file with the dependency.

Issue: The package requires the module during its build process, but doesn't declare it as a build dependency. This caused the build to fail with .

Solution: Added to which is installed before the unsloth package requirements, ensuring the module is available during the flash-attn build.

Commit: 79bb441

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature Request: Add Model Fine-tuning Support via Unsloth Backend

1 participant