feat: OpenAI Responses API with structured tool calling and multi-turn support

[eloe]

Summary

Implements the OpenAI Responses API (/v1/responses), enabling agent frameworks that target this newer API surface to work with mlx-vlm. This is a growing requirement as frameworks like OpenAI Agents SDK and others migrate from Chat Completions to the Responses API.

Features

Request handling:

• Accepts input as a string or structured message list (text, image, function_call_output)
• instructions field mapped to system message
• previous_response_id for multi-turn conversation replay via in-memory LRU store (256 entries)
• developer role mapped to system (OpenAI convention)
• Tool definitions in flat Responses format automatically normalized to nested Chat Completions format for Jinja template compatibility

Response format:

• Full ResponseObject with output[] containing message and function_call items
• usage with input_tokens, output_tokens, total_tokens, and input_tokens_details.cached_tokens
• status: completed or incomplete (with incomplete_details.reason)

Streaming (SSE):
Complete event sequence per the spec:

1. response.created
2. response.in_progress
3. response.output_item.added
4. response.content_part.added
5. response.output_text.delta (per token)
6. response.output_text.done
7. response.content_part.done
8. response.output_item.done
9. response.function_call_arguments.delta / .done (when tools detected)
10. response.completed

Tool calling:

• Reuses the existing tool parser infrastructure (_infer_tool_parser, process_tool_calls)
• Streaming suppresses tool-call markup tokens from text deltas
• Function call results emitted as ResponseFunctionCallItem output items

Architecture

File	Purpose
responses_models.py	Pydantic models for request/response/streaming events
responses_store.py	LRU store for previous_response_id replay
prompt_utils.py	_normalize_tools() for Responses-to-Chat tool format bridging
server.py	Endpoint handler, streaming generator

Tests

40 tests across 8 categories:

• Models (A): Request/response schema validation, computed properties
• Store (B): LRU eviction, save/get/replay, conversation reconstruction
• Endpoints (C): Basic text, message lists, instructions, tools, developer role, function_call_output
• Streaming (D): SSE event sequence, DONE sentinel
• Prompt Cache (E): Cache state wiring for both endpoints
• Concurrency (F): Semaphore acquisition in responses path
• Finish Reason (G): Correct status for tool calls vs plain text
• JSON Mode (H): response_format acceptance

python -m pytest mlx_vlm/tests/test_responses_api.py -v