Skip to content
Merged
Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
cdee972
feat: add get_llm() and get_secrets() to OpenHandsCloudWorkspace
openhands-agent Mar 13, 2026
5173810
feat: LLM api_key accepts SecretSource; workspace returns LookupSecre…
openhands-agent Mar 13, 2026
bba405a
feat: add env_headers to LookupSecret for sandbox-only secret resolution
openhands-agent Mar 13, 2026
1615d98
simplify: revert LLM SecretSource widening, get_llm returns real LLM
openhands-agent Mar 13, 2026
0f3c159
refactor: get_llm uses /users/me?expose_secrets=true instead of /sett…
openhands-agent Mar 13, 2026
4f1756f
security: send X-Session-API-Key with get_llm() request
openhands-agent Mar 13, 2026
5982d77
feat: add SaaS credential inheritance example for cloud workspace
openhands-agent Mar 16, 2026
c2a24ba
fix: sandbox DELETE uses correct path + query param; add .pr/ test ar…
openhands-agent Mar 16, 2026
c9fcada
fix: SecretSource serialization in update_secrets + sandbox DELETE path
openhands-agent Mar 16, 2026
fe40666
improve: test prompt now exercises secret values (print last 50%)
openhands-agent Mar 16, 2026
f934fa9
docs: update integration test report with complete findings
openhands-agent Mar 16, 2026
13dd4bc
refactor: drop env_headers, use expose_secrets context instead
openhands-agent Mar 16, 2026
4972ae9
docs: finalize test artifact with all results
openhands-agent Mar 16, 2026
3deba1e
docs: update test artifact with unambiguous verification output
openhands-agent Mar 16, 2026
6552831
docs: replace stale .pr/logs with final passing test output
openhands-agent Mar 16, 2026
6f3c8de
fix: clarify SaaS server is custom build from PR #13383
openhands-agent Mar 16, 2026
84770cd
chore: Remove PR-only artifacts [automated]
Mar 16, 2026
7fd6d48
refactor: narrow LookupSecret.get_value() return type to str
openhands-agent Mar 16, 2026
10cbd94
fix: resolve CI failures — renumber example 09→10, fix E501 line-too-…
openhands-agent Mar 16, 2026
d5367f5
docs: add provider tokens e2e test results to .pr/
openhands-agent Mar 17, 2026
d8b4ac4
docs: add full stdout of example 10 (SaaS credentials e2e)
openhands-agent Mar 17, 2026
8755c4d
rename: 10_cloud_workspace_saas_credentials → 10_cloud_workspace_shar…
openhands-agent Mar 17, 2026
066dde6
We should not override LLM / simplify prompt
xingyaoww Mar 17, 2026
f9e0cd3
Simplify prompt
xingyaoww Mar 17, 2026
15104c4
Merge branch 'main' into feat/cloud-workspace-get-llm-secrets
xingyaoww Mar 17, 2026
e69170a
chore: Remove PR-only artifacts [automated]
Mar 17, 2026
50f88ba
Add retry (up to 3 attempts) for get_llm and get_secrets API calls
openhands-agent Mar 17, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 72 additions & 0 deletions .pr/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# PR #2409 — Integration Test Report

## Test: `workspace.get_llm()` and `workspace.get_secrets()` against staging

**Date:** 2026-03-16 (initial), **2026-03-17** (provider tokens e2e)
**Target:** `https://ohpr-13383-240.staging.all-hands.dev` (deploy PR [#3436](https://github.com/OpenHands/deploy/pull/3436))
**Server PR:** [OpenHands/OpenHands#13383](https://github.com/OpenHands/OpenHands/pull/13383) (companion)
**Server commit:** `c0ed13f374ec647bf5349984961823650c8928c8` (includes `get_provider_tokens(as_env_vars=True)` refactor)
**SaaS server:** Custom build from [OpenHands/OpenHands#13383](https://github.com/OpenHands/OpenHands/pull/13383) (provides `/sandboxes/{id}/settings/secrets` endpoints)
**Sandbox agent-server:** `ghcr.io/openhands/agent-server:1.13.0-python` (stock — no SDK PR changes needed in sandbox)

## Provider Tokens E2E (2026-03-17) ✅

Full e2e run against staging with updated server commit `c0ed13f` that exposes provider tokens (e.g. `github_token`) via the sandbox secrets endpoint.

| Phase | Status | Details |
|---|---|---|
| `workspace.get_llm()` | ✅ | `model=litellm_proxy/minimax-m2.5`, api_key + base_url inherited from SaaS |
| `workspace.get_secrets()` | ✅ | Returns **3 secrets**: `['secret_1', 'secret_2', 'github_token']` |
| Provider token discovery | ✅ | `github_token` included — **provider tokens now flow to SDK** |
| `LookupSecret` resolution | ✅ | All 3 secrets resolved: `secret_1` (len=5), `secret_2` (len=5), `github_token` (len=40) |
| Sandbox cleanup | ✅ | Sandbox `5vwSsTFHIIjxie4pvjN3fQ` created + deleted cleanly |

> Log: [`.pr/logs/e2e_provider_tokens.log`](logs/e2e_provider_tokens.log)

## Initial Results (2026-03-16) ✅

| Component | Status | Details |
|---|---|---|
| Sandbox provisioning | ✅ | Created, RUNNING in ~50s, cleaned up on exit |
| `workspace.get_llm()` | ✅ | Retrieves `litellm_proxy/minimax-m2.5` + api_key + base_url from SaaS |
| `workspace.get_secrets()` | ✅ | Discovers `['DUMMY_1', 'DUMMY_2']` via `GET /sandboxes/{id}/settings/secrets` |
| `update_secrets(LookupSecret)` | ✅ | LookupSecret with session key in `headers` survives serialization |
| Env vars exported inside sandbox | ✅ | `_export_envs` resolves LookupSecret → secrets appear as real env vars |
| Sandbox cleanup | ✅ | `DELETE /api/v1/sandboxes/{id}` succeeds |

> Log: [`.pr/logs/run_combined.log`](logs/run_combined.log)

### Agent verification output (from inside the sandbox)

The agent ran the exact Python command we gave it. The output proves the secrets
were resolved by the `_export_envs` pipeline and exported as real env vars:

```
$ python3 -c "import os; v=os.environ.get('DUMMY_1',''); print(f'DUMMY_1: len={len(v)}, last_half={v[len(v)//2:]}')" && \
python3 -c "import os; v=os.environ.get('DUMMY_2',''); print(f'DUMMY_2: len={len(v)}, last_half={v[len(v)//2:]}')"

DUMMY_1: len=14, last_half=ecret 1
DUMMY_2: len=14, last_half=ecret 2
```

Both secrets are 14 characters long, non-empty, and the second half matches
the expected values (`"Dummy secret 1"` → `"ecret 1"`, `"dummy secret 2"` → `"ecret 2"`).

## Issues found and fixed during testing

### 1. Sandbox DELETE 405
`cleanup()` called `DELETE /api/v1/sandboxes?sandbox_id=X` (query param on collection route) → 405.
**Fix:** Changed to `DELETE /api/v1/sandboxes/{sandbox_id}?sandbox_id={sandbox_id}`.

### 2. SecretStr serialization redacts headers
`SecretSource.model_dump(mode="json")` redacts `SecretStr` fields (returns `**********`).
`LookupSecret.headers` containing `X-Session-API-Key` was lost during `update_secrets()` serialization.
**Fix:** Added `context={"expose_secrets": True}` to `model_dump()` in `RemoteConversation.update_secrets()`.

### 3. Removed `env_headers` (was unnecessary complexity)
Original design added `env_headers` field to `LookupSecret` so session key VALUE never appeared
in serialized JSON (only the env var NAME). This created a deployment dependency — the agent-server
image needed the new field, but the stock image didn't have it.
**Fix:** Dropped `env_headers` entirely. Using the existing `headers` field with `expose_secrets`
context is simpler and works with the stock agent-server image. No custom build or redeploy needed.
Net: **-35 lines, +10 lines**.
26 changes: 26 additions & 0 deletions .pr/logs/e2e_provider_tokens.log
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
=== E2E Test: SDK credential inheritance ===
Cloud URL: https://ohpr-13383-240.staging.all-hands.dev
Deploy commit: c0ed13f374ec647bf5349984961823650c8928c8 (server PR #13383)
Deploy PR: https://github.com/OpenHands/deploy/pull/3436 (branch ohpr-13383-240)
Date: 2026-03-17

--- Phase 1: get_llm() ---
Sandbox ID: 5vwSsTFHIIjxie4pvjN3fQ
Session key: set
✅ get_llm() succeeded: model=litellm_proxy/minimax-m2.5
api_key: ***w86g
base_url: https://llm-proxy.staging.all-hands.dev

--- Phase 2: get_secrets() ---
✅ get_secrets() returned 3 secrets: ['secret_1', 'secret_2', 'github_token']
🎉 Provider tokens found: ['github_token']
Custom secrets: ['secret_1', 'secret_2']

--- Phase 3: LookupSecret resolution ---
✅ secret_1: resolved, len=5, last4=...est1
✅ secret_2: resolved, len=5, last4=...est2
✅ github_token: resolved, len=40, last4=...4eur

Sandbox 5vwSsTFHIIjxie4pvjN3fQ deleted

=== E2E Test Complete ===
Loading
Loading