feat: Add gRPC protocol for Unsloth fine-tuning backend (Phase 1)#9058
feat: Add gRPC protocol for Unsloth fine-tuning backend (Phase 1)#9058localai-bot wants to merge 7 commits intomudler:masterfrom
Conversation
✅ Deploy Preview for localai ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
3392ab2 to
9ff8bfe
Compare
- Add TrainRequest and TrainResponse messages to backend.proto - Add TrainStream server-streaming RPC to Backend service - Update Go gRPC interfaces (interface.go, server.go, client.go, backend.go, embed.go) - Add stub implementation in base.go returning unimplemented error - Regenerate protobuf bindings This implements Phase 1 of the Unsloth fine-tuning backend feature. Subsequent phases will implement the Python backend and Go service layer. Signed-off-by: localai-bot <localai-bot@users.noreply.github.com>
- Add Python unsloth backend with QLoRA/LoRA/full fine-tuning support - Implement Go service layer for job management - Add HTTP API endpoints for fine-tuning jobs - Integrate GGUF export functionality - Update CI/build configuration Signed-off-by: localai-bot <localai-bot@users.noreply.github.com>
Fixes AppliedFixed compilation errors in core/services/training.go:
See commit a034d83 for details. |
Fix AppliedUpdated to use instead of to fix the CI build failure. The previous torch250 version was not available for the cu121 CUDA 12.8 build. Changed to torch260 which should be compatible. See commit ae5ed50 for details. |
a034d83 to
85f3022
Compare
CI Fix AppliedFixed the cublas12 CI build failure by changing the unsloth torch version from to . Issue: The package was not available on PyPI for the cu121 CUDA 12.8 build. Solution: Changed to which is the latest available version compatible with cu121. Commit: 85f3022 |
DCO Fix AppliedFixed the DCO (Developer Certificate of Origin) check failure by adding proper sign-off to commits. Additionally, fixed the cublas12 CI build failure by changing the unsloth torch version from to which is available for the cu121 CUDA 12.8 build. The PR base branch has been updated to include the DCO fixes. Please review the updated branch. Commit: bfecc4a |
Signed-off-by: localai-bot <localai-bot@users.noreply.github.com>
85f3022 to
a034d83
Compare
Fix AppliedUpdated cublas13 unsloth torch version from Issue: The Solution: Changed to Commit: 85515e5 |
The unsloth package requires the 'packaging' module for building flash-attn. Adding it explicitly to the requirements files for cublas12 and cublas13 builds. Signed-off-by: localai-bot <localai-bot@users.noreply.github.com>
a034d83 to
1211a67
Compare
The flash-attn package requires the 'packaging' module during its build process, but doesn't declare it as a build dependency. Adding it as an install-time dependency ensures it's available before the unsloth package tries to build flash-attn. Signed-off-by: localai-bot <localai-bot@users.noreply.github.com>
Fix AppliedFixed the CI build failure by adding a new file with the dependency. Issue: The package requires the module during its build process, but doesn't declare it as a build dependency. This caused the build to fail with . Solution: Added to which is installed before the unsloth package requirements, ensuring the module is available during the flash-attn build. Commit: 79bb441 |
Implementation Progress: Phase 1 Complete
This PR implements Phase 1 of the Unsloth fine-tuning backend feature for LocalAI (Issue #9054).
Changes
Technical Details
The TrainStream RPC follows the existing streaming pattern (PredictStream/TTSStream) with:
Next Phases
Related
Closes #9054