fix: preserve Anthropic thinking blocks and signatures in LiteLLM round-trip by giulio-leone · Pull Request #4811 · google/adk-python

giulio-leone · 2026-03-13T03:50:56Z

Summary

Fixes #4801 — Adaptive thinking is broken when using Claude models through LiteLLM.

Root Cause

When Claude produces extended thinking with thinking_blocks (each containing a type, thinking text, and signature), the round-trip through ADK's LiteLLM integration silently loses them:

_extract_reasoning_value() only read reasoning_content (a flattened string without signatures), ignoring the richer thinking_blocks field
_content_to_message_param() set reasoning_content on the outgoing ChatCompletionAssistantMessage, but LiteLLM's anthropic_messages_pt() prompt template silently drops the reasoning_content field entirely
Result: thinking blocks vanish from conversation history after turn 1; Claude stops producing them

Fix

Three coordinated changes in lite_llm.py:

Change	What it does
`_is_anthropic_provider()` helper	Detects `anthropic`, `bedrock`, `vertex_ai` providers
`_extract_reasoning_value()`	Now prefers `thinking_blocks` (with per-block signatures) over `reasoning_content`
`_convert_reasoning_value_to_parts()`	Handles `ChatCompletionThinkingBlock` dicts, preserving `thought_signature`
`_content_to_message_param()`	For Anthropic providers, embeds thinking blocks directly in the message `content` list as `{"type": "thinking", ...}` dicts — this format passes through LiteLLM's `anthropic_messages_pt()` correctly

For non-Anthropic providers (OpenAI, etc.), behavior is unchanged — reasoning_content is still used.

Verification

LiteLLM's anthropic_messages_pt() was tested to confirm:
- reasoning_content field → DROPPED (existing LiteLLM bug)
- content as list with {"type": "thinking", ...} → PRESERVED ✅
- Signatures in thinking blocks → PRESERVED when in content list ✅

Tests

Added 7 targeted tests covering:

_is_anthropic_provider() — provider detection
_extract_reasoning_value() — prefers thinking_blocks over reasoning_content
_convert_reasoning_value_to_parts() — signature preservation from block dicts
_convert_reasoning_value_to_parts() — plain string fallback (no signature)
_content_to_message_param() — Anthropic: thinking blocks embedded in content list
_content_to_message_param() — OpenAI: reasoning_content field used (unchanged)
_content_to_message_param() — Anthropic thinking + tool calls combined

Full test suite: 4732 passed, 0 failures

gemini-code-assist · 2026-03-13T03:51:00Z

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

google-cla · 2026-03-13T03:51:07Z

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

adk-bot · 2026-03-13T03:51:52Z

Response from ADK Triaging Agent

Hello @giulio-leone, thank you for your contribution!

Before we can merge this PR, we need you to sign our Contributor License Agreement (CLA). You can find more information and sign the CLA at https://cla.developers.google.com/.

Thanks!

rohityan · 2026-03-13T18:41:43Z

Hi @giulio-leone , Thank you for your contribution! It appears you haven't yet signed the Contributor License Agreement (CLA). Please visit https://cla.developers.google.com/ to complete the signing process. Once the CLA is signed, we'll be able to proceed with the review of your PR. Thank you!

giulio-leone · 2026-03-14T21:59:15Z

@rohityan Thanks for the heads-up! I'll get the Google CLA signed. Will follow up once it's done.

giulio-leone · 2026-03-15T15:53:58Z

I have signed the Google CLA. Could you please re-run the CLA check? Thank you!

giulio-leone · 2026-03-15T17:18:18Z

Hi @rohityan — the CLA is now signed and passing ✅ (it was a Co-authored-by trailer issue that has been resolved). Ready for review!

rohityan · 2026-03-17T21:44:56Z

Hi @giulio-leone, This PR has merge conflicts that require changes from your end. Could you please rebase your branch with the latest main branch to address these? Once this is complete, please let us know so we can proceed with the review.

giulio-leone · 2026-03-17T23:15:47Z

Rebased this branch onto the latest main and resolved the merge conflicts.

Local validation after the rebase is clean:

./.venv/bin/pyink --check src/google/adk/models/lite_llm.py tests/unittests/models/test_litellm.py
./.venv/bin/pytest -q tests/unittests/models/test_litellm.py -> 247 passed
./.venv/bin/python /Users/giulioleone/.copilot/session-state/8fa0e071-2c29-4ffb-8819-1b478429c9f0/files/evidence/adk_pr_4811_runtime_proof.py -> PASS
second clean pass on the Anthropic thinking-block subset -> 28 passed + runtime proof PASS

Updated head: f5f7cc805ad62850912220c8e5f953b6731edfb6.

giulio-leone · 2026-03-19T12:22:36Z

Rebased onto current main and added one follow-up fix + regression test for the model-driven Anthropic round-trip.

Local validation on the rebased branch:

pyink --check src/google/adk/models/lite_llm.py tests/unittests/models/test_litellm.py
python -m pytest tests/unittests/models/test_litellm.py -q → 248 passed
repeated the same validation suite twice consecutively with no code changes between passes

Runtime proof beyond unit tests (real source + installed LiteLLM prompt templates, Bedrock Claude path):

main provider-only path serializes reasoning_content, and LiteLLM drops the thinking block before the Bedrock wire payload
this branch provider-only path serializes an inline thinking block, and the Bedrock wire payload keeps reasoningContent.signature == "c2lnX2E="
model-driven path now preserves the base64 signature on both ADK message reconstruction and final Bedrock wire payload (c2lnX2E=), covered by a new regression test

This should leave the PR both fresh and with concrete local proof for the Claude/LiteLLM thinking-block path.

giulio-leone · 2026-03-19T12:28:11Z

Small follow-up: I amended the top commit message only to restore a green CLA check after the force-push. No code changes from the validation comment above. Current head: 1dc20fe37dccb16f6d629acb7cc89924c9aa76bb.

giulio-leone · 2026-03-19T15:35:16Z

Refreshed onto the latest main (fbe76595) to clear the behind state.

Fresh local validation on the new head 6193fadadca6ac640acf863e4e134a892ce7ba24:

pyink --check src/google/adk/models/lite_llm.py tests/unittests/models/test_litellm.py
python -m pytest tests/unittests/models/test_litellm.py -k 'thinking or anthropic' -q → 25 passed
real provider-only Bedrock/Claude proof still holds: main drops the thinking block before the wire payload, while this branch preserves reasoningContent.reasoningText.signature == "c2lnX2E="

…nd-trip When using Claude models through LiteLLM, extended thinking blocks (with signatures) were lost after the first turn because: 1. _extract_reasoning_value() only read reasoning_content (flattened string without signatures), ignoring thinking_blocks 2. _content_to_message_param() set reasoning_content on the outgoing message, which LiteLLM's anthropic_messages_pt() template silently drops This fix: - Adds _is_anthropic_provider() helper to detect anthropic/bedrock/ vertex_ai providers - Updates _extract_reasoning_value() to prefer thinking_blocks (with per-block signatures) over reasoning_content - Updates _convert_reasoning_value_to_parts() to handle ChatCompletionThinkingBlock dicts, preserving thought_signature - Updates _content_to_message_param() to embed thinking blocks directly in the message content list for Anthropic providers, bypassing the broken reasoning_content path Fixes google#4801

Cover the model-driven LiteLLM Anthropic round-trip with a regression test.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

giulio-leone · 2026-03-21T01:51:19Z

Refreshed this branch onto current origin/main, resolved the post-rebase import-order drift in the touched test file, and re-ran validation from the refreshed head 2015c6f1.

Validation:

two consecutive clean passes of pyink --check + isort --check-only on src/google/adk/models/lite_llm.py and tests/unittests/models/test_litellm.py
pytest tests/unittests/models/test_litellm.py -q -> 248 passed on both passes
repo-aligned mypy diff gate against current origin/main: main errors 1342, branch errors 1342, new errors introduced 0
final sanity check rerun from the committed, clean head also passed

Direct behavioral proof against current origin/main for the Anthropic-family provider paths this PR is fixing:

input: reasoning part with thought_signature=b"sigx", rendered through _content_to_message_param(...)
rebased branch:
- bedrock/anthropic.claude-3-5-sonnet -> thinking_signature == "c2lneA=="
- vertex_ai/claude-3-5-sonnet@20241022 -> thinking_signature == "c2lneA=="
current origin/main:
- both provider paths still emit raw thinking_signature == "sigx"

So the refreshed branch is preserving the Anthropic-family signature in the provider-compatible encoded form, while current main still leaks the raw unencoded value on those paths.

adk-bot added the models [Component] Issues related to model support label Mar 13, 2026

rohityan self-assigned this Mar 13, 2026

rohityan added the request clarification [Status] The maintainer need clarification or more information from the author label Mar 13, 2026

giulio-leone force-pushed the fix/litellm-anthropic-thinking-roundtrip branch from 9b92452 to 4b10202 Compare March 14, 2026 20:08

giulio-leone force-pushed the fix/litellm-anthropic-thinking-roundtrip branch 2 times, most recently from b3c35fc to 59e6e04 Compare March 15, 2026 16:03

giulio-leone force-pushed the fix/litellm-anthropic-thinking-roundtrip branch from 1d6844c to f5f7cc8 Compare March 17, 2026 23:15

giulio-leone force-pushed the fix/litellm-anthropic-thinking-roundtrip branch from f5f7cc8 to 1ed2e51 Compare March 19, 2026 12:22

giulio-leone force-pushed the fix/litellm-anthropic-thinking-roundtrip branch from 1ed2e51 to 1dc20fe Compare March 19, 2026 12:26

giulio-leone force-pushed the fix/litellm-anthropic-thinking-roundtrip branch from 1dc20fe to 6193fad Compare March 19, 2026 15:35

giulio-leone and others added 4 commits March 21, 2026 02:45

chore: finalize LiteLLM anthropic rebase resolution

793880f

fix(models): re-encode anthropic thinking signatures

e4ed1fe

Cover the model-driven LiteLLM Anthropic round-trip with a regression test.

test(litellm): restore import order after rebase

2015c6f

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

giulio-leone force-pushed the fix/litellm-anthropic-thinking-roundtrip branch from 6193fad to 2015c6f Compare March 21, 2026 01:51

Conversation

giulio-leone commented Mar 13, 2026

Summary

Root Cause

Fix

Verification

Tests

Uh oh!

gemini-code-assist bot commented Mar 13, 2026

Uh oh!

google-cla bot commented Mar 13, 2026

Uh oh!

adk-bot commented Mar 13, 2026

Uh oh!

rohityan commented Mar 13, 2026

Uh oh!

giulio-leone commented Mar 14, 2026

Uh oh!

giulio-leone commented Mar 15, 2026

Uh oh!

giulio-leone commented Mar 15, 2026

Uh oh!

rohityan commented Mar 17, 2026

Uh oh!

giulio-leone commented Mar 17, 2026

Uh oh!

giulio-leone commented Mar 19, 2026

Uh oh!

giulio-leone commented Mar 19, 2026

Uh oh!

giulio-leone commented Mar 19, 2026

Uh oh!

giulio-leone commented Mar 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants