Add shared long-term memory server (experimental)

[reyortiz3]

Summary

ToolHive manages MCPs (tools) and Skills (procedural knowledge as OCI artifacts). The missing primitive is shared long-term memory — a knowledge store that agents can query and contribute to across sessions. Without it every agent session starts cold, and facts learned in one session are invisible to others.

This PR introduces the memory server core (Plan 1 of 3):

• pkg/memory/ — domain types (Entry, Revision, ListFilter), three pluggable interfaces (Store, VectorStore, Embedder), a Service orchestration layer with conflict detection and score-weighted search ranking, trust/staleness scoring formulas, and gomock mocks
• pkg/memory/sqlite/ — SQLite-backed Store and VectorStore (Go-native cosine similarity, no CGo dependency); goose migrations including a TypeEpisodic type for time-indexed event records
• pkg/memory/embedder/ollama/ — Ollama HTTP embedder that probes vector dimensions on startup
• cmd/thv-memory/ — standalone MCP server binary serving 9 tools over streamable HTTP (/mcp), with a /health liveness probe, YAML config with sensible defaults, and a background lifecycle job (TTL expiry, score recomputation every 24h)
• docs/proposals/2026-04-22-shared-memory-server.md — full design doc covering architecture, tool surface, scoring, conflict detection, Skills relationship, comparison with LinkedIn's Cognitive Memory Agent, and the recommended three-tier memory activation strategy

Key design decisions:

• Conflict detection is write-time (cosine similarity > 0.85 blocks the write and returns candidates for the agent to resolve — no LLM inference needed)
• Search results are ranked by similarity × trust_score × (1 − 0.3 × staleness_score) so flagged/stale entries don't rank above fresh, trusted ones
• The agent IS the retrieval orchestrator — tools are explicit MCP calls, not auto-triggered pipelines
• Three memory types: semantic (aggregated facts), procedural (how-to), episodic (time-indexed events with CreatedAfter/CreatedBefore list filters)

Plans 2 (CLI thv memory subcommand + system workload integration) and 3 (Kubernetes MCPMemoryServer CRD) are follow-up work.

Type of change

• New feature

Test plan

• Unit tests (task test)
• Linting (task lint-fix)
• Manual testing (integration test in cmd/thv-memory/integration_test.go wires real SQLite store + vector store + fake embedder end-to-end: remember → search → access count increment → delete → ErrNotFound; conflict detection test verifies force-write path)

Changes

File	Change
pkg/memory/types.go	Domain types: Entry, Revision, ListFilter (with time-range fields), VectorFilter, Type (semantic/procedural/episodic), scoring types
pkg/memory/interfaces.go	Store, VectorStore, Embedder interfaces + mockgen directives
pkg/memory/service.go	Service: conflict detection, Remember, Search with composite ranking
pkg/memory/scoring.go	ComputeTrustScore, ComputeStalenessScore
pkg/memory/sqlite/	SQLite Store, VectorStore, goose migrations (001 initial + 002 adds episodic type)
pkg/memory/embedder/ollama/	Ollama HTTP embedder
pkg/memory/mocks/	Generated gomock mocks for all three interfaces
cmd/thv-memory/main.go	Entry point: HTTP server lifecycle, graceful shutdown
cmd/thv-memory/server.go	MCP server construction, tool registration, streamable HTTP handler + /health
cmd/thv-memory/config.go	YAML config with defaults (SQLite, Ollama, 0.0.0.0:8080)
cmd/thv-memory/lifecycle/job.go	Background job: TTL expiry + score recomputation
cmd/thv-memory/tools/	9 MCP tool handlers
cmd/thv-memory/integration_test.go	End-to-end integration test
docs/proposals/2026-04-22-shared-memory-server.md	Design doc

Does this introduce a user-facing change?

No — this adds a new standalone binary (cmd/thv-memory) and supporting packages. Nothing in the existing CLI or operator is modified. The binary is not yet wired into thv commands (that is Plan 2).

Implementation plan

Approved implementation plan

This PR was planned and implemented with Claude Code. The design spec is at docs/proposals/2026-04-22-shared-memory-server.md. The implementation follows the spec with the following notable adaptations:

• Type names use Go stutter-avoidance convention (Entry not MemoryEntry, Store not MemoryStore) to satisfy the revive linter
• goose.NewProvider (scoped) used instead of global goose.SetBaseFS/SetDialect to avoid concurrent-open races
• server.NewStreamableHTTPServer + server.WithStdioContextFunc used to match actual mcp-go v0.48.0 API
• SQLite VectorStore uses Go-native cosine similarity (no CGo/sqlite-vec dependency) with a load-and-score approach; external VectorStore providers are pluggable via the VectorStore interface for datasets > 100K entries

Special notes for reviewers

This is experimental — do not merge until Plans 2 and 3 are ready. Specific areas to scrutinise:

• pkg/memory/sqlite/vector.go: the load-all-and-score approach works for small datasets but will not scale past ~100K entries. The VectorStore interface is designed to be swapped for Qdrant/pgvector when needed.
• pkg/memory/service.go: the conflict threshold (0.85) and staleness penalty weight (0.3) are initial values — they will need tuning against real usage data.
• cmd/thv-memory/server.go: no auth middleware on the MCP endpoint yet. Auth will be enforced at the ToolHive proxy layer when the system workload integration lands in Plan 2.

Generated with Claude Code

[github-actions]

Large PR Detected

This PR exceeds 1000 lines of changes and requires justification before it can be reviewed.

How to unblock this PR:

Add a section to your PR description with the following format:

## Large PR Justification

[Explain why this PR must be large, such as:]
- Generated code that cannot be split
- Large refactoring that must be atomic
- Multiple related changes that would break if separated
- Migration or data transformation

Alternative:

Consider splitting this PR into smaller, focused changes (< 1000 lines each) for easier review and reduced risk.

See our Contributing Guidelines for more details.

---

This review will be automatically dismissed once you add the justification section.

[codecov]

Codecov Report

❌ Patch coverage is 25.90874% with 958 lines in your changes missing coverage. Please review.
✅ Project coverage is 66.51%. Comparing base (cffe934) to head (9bb592f).
⚠️ Report is 68 commits behind head on main.

Files with missing lines	Patch %	Lines
cmd/thv-memory/resources/api.go	0.00%	168 Missing ⚠️
demo/recruiter/cmd/demo/main.go	0.00%	162 Missing ⚠️
pkg/memory/sqlite/store.go	59.07%	80 Missing and 17 partials ⚠️
cmd/thv-memory/server.go	0.00%	78 Missing ⚠️
cmd/thv-memory/main.go	0.00%	69 Missing ⚠️
cmd/thv-memory/tools/crystallize.go	0.00%	39 Missing ⚠️
cmd/thv-memory/tools/remember.go	0.00%	38 Missing ⚠️
cmd/thv-memory/tools/consolidate.go	0.00%	37 Missing ⚠️
cmd/thv-memory/config.go	0.00%	36 Missing ⚠️
pkg/memory/service.go	63.91%	20 Missing and 15 partials ⚠️
... and 11 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5015      +/-   ##
==========================================
- Coverage   69.02%   66.51%   -2.52%     
==========================================
  Files         554      621      +67     
  Lines       73075    61912   -11163     
==========================================
- Hits        50443    41180    -9263     
+ Misses      19620    17571    -2049     
- Partials     3012     3161     +149     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.