feat(vllm): add grammar and structured output support#8806

Open

eureka928 wants to merge 8 commits intomudler:masterfrom

eureka928:feat/vllm-structured-output

Commits on Mar 6, 2026

feat(proto): add JSONSchema and ResponseFormat fields to PredictOptions
eureka928
committed
feat(backend): pass JSONSchema and ResponseFormat through gRPC
eureka928
committed
feat(endpoints): extract raw JSON schema for structured output
eureka928
committed
feat(vllm): add structured output support via guided decoding
eureka928
committed
fix: refine vLLM structured output implementation
eureka928
committed
fix(vllm): support both vLLM API versions and add grammar passthrough
eureka928
committed
docs: update constrained grammars with vLLM structured output support
eureka928
committed

Commits on Mar 13, 2026

refactor: use Metadata map instead of dedicated proto fields for structured output
eureka928
committed