+ Why agentic memory is hard, and what the research tells us to do about it.
+
+
+
+
+
+
The Problem
+
Every agent session starts with no memory of what came before.
+
+
+
+
+ A recruiter learns a visa policy the hard way. The next session repeats the mistake.
+ The hiring manager has no idea where the pipeline stands.
+
+
+
+
+
+
What the Research Tells Us
+
Three bodies of work converge on the same failure modes.
+
+
+
+
A Survey on the Memory Mechanism of LLM-based Agents
+
arXiv 2603.07670 · 2025
+
Most systems neglect the manage phase — consolidation, conflict resolution, staleness. Silent contradictions are the most common failure mode.
+
+
+
LinkedIn Cognitive Memory Agent (CMA)
+
LinkedIn Engineering Blog · 2024
+
Hierarchical episodic + semantic + procedural memory at LinkedIn scale. Trust-weighted retrieval and per-application tenant isolation in production.
+
+
+
Practical Guide to LLM Memory Systems
+
Towards Data Science · 2024
+
Trust differentiation between human and agent memory is widely neglected. Staleness is the second leading cause of retrieval degradation.
+
+
+
+
+
+ Key insight #1 — Memory isn't just storage. Write, retrieval, and management (consolidation, conflict, staleness) all need first-class design.
+
+
+ Key insight #2 — Human-authored memories must outrank agent-authored ones. Trust differentiation is the most neglected dimension in practice.
+
+
+
+
+
+
+
Five Design Tensions
+
+ Every memory system is pulled along five axes that tug in opposite directions.
+ The right balance shifts with the application — a medical triage agent operates under a very different
+ faithfulness–efficiency frontier than a recipe recommender.
+ (arXiv 2603.07670)
+
+
+
+
+
+
+
+
+
⚡ Utility
+
vs Efficiency
+
+
+
Maximising utility tempts you to store everything — bloating storage and retrieval cost. Aggressive compression silently discards the one rare fact that matters three weeks later.
+
+
+
Store semantically compressed memories, not raw transcripts. Trust scores surface high-value entries first so retrieval cost stays bounded.
+
+
+
+
+
+
+
🎯 Faithfulness
+
vs Adaptivity
+
+
+
Stale or hallucinated recall can be worse than no recall at all. But locking down memory prevents it from reflecting a world that changes.
+
+
+
Conflict detection on write. Staleness scoring updated every 24h. Human-authored entries always outrank agent entries. Correction history preserved.
+
+
+
+
+
+
+
🔄 Adaptivity
+
vs Stability
+
+
+
Memory that updates freely drifts — an agent can overwrite good knowledge with a bad observation. Full retrains are expensive and disrupt continuity.
+
+
+
Incremental writes via memory_remember. Updates create a revision history — nothing is ever silently overwritten. Flag + review before trust score recovers.
+
+
+
+
+
+
+
📊 Efficiency
+
latency · tokens · storage
+
+
+
Every retrieved memory costs tokens in the context window. Large retrieval sets slow inference and dilute relevance. Embedding calls add latency on every write.
+
+
+
Embed once on write, never on read. Single-pass vector search with configurable top_k. TTL-based expiry keeps the store bounded. Score-weighted ranking cuts noise.
+
+
+
+
+
+
+
🏛️ Governance
+
privacy · deletion · policy
+
+
+
Memory systems accumulate sensitive data. Without explicit deletion and access controls, memory becomes a liability — agents can recall things they shouldn't.
+
+
+
Explicit memory_forget. Agents can only delete their own entries; humans can delete any. TTL for automatic expiry. Read-only Resources protected by ErrReadOnly.
+
+
+
+
+
+
+
Tension
+
Our position
+
+
+
+
+
+
Our Approach vs. the Literature
+
+
+
+
+
+
Dimension
+
LinkedIn CMA
+
ToolHive Memory
+
+
+
+
+
Conflict detection
+
On roadmap
+
✅ Built — cosine sim > 0.85 on write
+
+
+
Trust / staleness
+
Time-based, on roadmap
+
✅ Built — formula + 24 h background job
+
+
+
User control
+
Planned
+
✅ list · update · flag · forget
+
+
+
Memory types
+
Episodic + Semantic + Procedural
+
+ Resource (read-only reference docs)
+
+
+
Retrieval
+
LLM-orchestrated multi-step
+
Single-pass vector — agent is the orchestrator
+
+
+
Crystallization
+
Not described
+
✅ Procedural → versioned Skill (OCI)
+
+
+
Tenant isolation
+
Per-application storage isolation
+
Auth at proxy level — storage isolation on roadmap
+
+
+
+
+
+
+ We are ahead on conflict detection, trust scoring, and user control.
+ LinkedIn is ahead on hierarchical aggregation and tenant isolation — both planned for a later phase.
+
+
+
+
+
+
Part 2
+
Architecture
+
+ How it's built — components, memory types, and the full lifecycle.
+
+
+
+
+
+
Architecture
+
+ Memory is a system workload inside ToolHive —
+ auto-provisioned, singleton per scope, excluded from thv stop --all.
+ Fully pluggable storage, vector, and embedder backends.
+
+
+
+
+
+
+
+
💻 Local (thv CLI)
+
+ thv memory init
+ Personal memory, local container.
+ SQLite + sqlite-vec defaults.
+ State in ~/.local/state/toolhive/
+
+
+
+
👥 Team (thv serve)
+
+ Shared instance, all agents connect
+ through the ToolHive API proxy.
+ Auth via existing OIDC middleware.
+ Postgres + Qdrant recommended.
+
+
+
+
☸️ Kubernetes (thv-operator)
+
+ MCPMemoryServer CRD.
+ Operator reconciles → Deployment
+ + Service + PVC automatically.
+ Registered in MCPRegistry.
+
+
+
+
+
+
+
+
Hierarchical Memory roadmap
+
+ Today memory is a flat namespace. The path to layered, scope-aware memory is well defined.
+
+
+
+
+
+
TODAY — tags as a workaround
+
+ Tag memories with a project label:
+ tags: ["project:payments"]
+ Filter in search:
+ memory_search("auth", tags=["project:payments"])
+ ⚠ Convention only — no enforcement.
+ An unfiltered search still returns everything.
+
+
+
+
+
+
ROADMAP — namespace isolation
+
+ Add Namespace to every entry.
+ Proxy stamps it from OIDC token or project context — agents never set it.
+ Search walks up the hierarchy:
+ project → team → global
+ One schema migration. No API surface change.
+
+
+
+
+
+
+
+
+
+
+
Memory Types
+
Four types cover reference docs, facts, events, and learned processes.
+
+
+
+
📄
+
+
Resource
+
Read-only reference documents uploaded via REST API. Agents discover via resources/list or memory_search — never writable by agents. source: resource
+
+
+
+
🧠
+
+
Semantic
+
Aggregated facts and domain knowledge — things that are durably true. Conflict detection (cosine sim > 0.85) prevents silent contradictions on write. human | agent
+
+
+
+
📅
+
+
Episodic
+
Time-indexed event records — things that happened. Phone screens, decisions, observations. Queryable by tags and time range. e.g. "Alice Chen screened 2024-03-10"
+
+
+
+
🔧
+
+
Procedural
+
Learned behaviors and processes. Emerges from episodic patterns. Can be crystallized into a versioned Skill (OCI artifact) the whole team can reuse. crystallizable → Skill
+ Conflict detection on write — cosine similarity > 0.85 returns matching entries. The agent (with context) decides: force-write, update, or abort.
+
+
+
+
+
+
+
From Memory to Skill
+
Fluid procedural knowledge crystallizes into versioned, distributable runbooks.
+
+
+
+
+ Procedural memories and Skills are the same knowledge at different stages of maturity —
+ fluid and evolving in memory, crystallized and versioned as a Skill.
+
+
+
+
+
+
Part 3
+
Live Demo
+
+ The Recruiter — hiring a Senior Go Engineer at Stacklok
+
+
+
+
+
+
The Recruiter Scenario
+
A one-week hiring process — every session shares a single memory server.
+
+
+
+
Cast
+
+ 👩 Recruiter — runs phone screens
+ 🧑💼 Hiring Manager — checks pipeline cold
+ 🖥️ Memory Server — one server, all sessions
+
+
+
+
What we'll see
+
+ ✅ Policy recalled without being told
+ ✅ Cross-session knowledge sharing
+ ✅ Process learned from repeated failure
+ ✅ Runbook born from lived experience
+
+
+
+
+
+ claude --mcp-config .demo.mcp.json
+ — the only integration needed
+
+
+
+
+
+
Scenario — All Phases
+
+
+
1
+
+ Resource uploadresource
+ Job description registered as a read-only MCP Resource — discoverable, never modifiable
+
+
+
+
2
+
+ Semantic memorysemantic
+ 3 company-wide facts: no visa sponsorship · salary $100–150K · remote US async culture
+
+ Hiring Manager — cold searchsemanticepisodic
+ Joins with zero context. Memory gives full pipeline status, comp band, and JD
+
+
+
+
5
+
+ Bob Martinezepisodicprocedural
+ Archived. Recruiter spots a pattern → agent writes a procedural memory unprompted
+
+
+
+
6
+
+ Charlie Kimproceduralepisodic
+ Retrieves the checklist (written by a different session). Applies it. Charlie advances.
+
+
+
+
7
+
+ Crystallize → Skillprocedural
+ One week of screens → phone-screen runbook the whole recruiting team can reuse
+
+
+
+
+
+
+
+
+
Phases 1 – 2
+
Setup — prime the memory server
+
+
+
+
+
Phase 1 · Resource
+
+ POST /api/resources
+ Job description uploaded as a read-only MCP Resource.
+ Agents discover it via memory_search or resources/list.
+ Protected by ErrReadOnly — no agent can modify it.
+
+
+
+
Phase 2 · Semantic memory
+
+ 🧠 Company does not sponsor US work visas for any engineering role
+
+
+ 🧠 Senior Go Engineer base: $100K–$150K + equity
+
+
+ 🧠 Engineering team fully remote, US timezone, async-first
+
+
+
+
Written once by the setup script — recalled by any agent session at any time.
+
+
+
+
+
+
Phase 3
+
Recruiter — Alice Chen phone screen
+
+
+
+
+
The recruiter says
+
+ "She mentioned she's on OPT and would need an H1-B transfer. Before I move her forward I want to make sure that's not a blocker. Can you check if we have any policy on that?"
+
+
+
+
What the agent does
+
+
→ memory_search("visa sponsorship policy")
+
Finds: "Company does not sponsor US work visas"
+
→ memory_remember (episodic)
+
Alice Chen — OPT / H1-B needed → archived
+
+
+
+
+
+ The agent found a policy it was never told about in this session —
+ retrieved from shared memory written in Phase 2.
+
+
+
+
+
+
+
Phase 4
+
Hiring Manager — cold pipeline review
+
+
+
+ "I haven't been in the loop on recruiting — can you catch me up? Pipeline status, approved comp range, and a reminder of what we're hiring for."
+
+
+
+
memory_search("candidates screened") Alice Chen → archived (visa). 1 of 1.
+
memory_search("visa sponsorship") No sponsorship policy. Explains Alice.
+
memory_search("salary compensation") $100K–$150K base + equity. Approved band.
+
memory_search("job description") Retrieves the JD Resource. Full requirements.
+
+
+
The hiring manager never spoke to the recruiter. The memory server was the handoff.
+
+
+
+
+
+
Phase 5
+
Recruiter — Bob Martinez + a lesson learned
+
+
+
+
+
+ "…this is the second screen in a row where something obvious knocked the candidate out early. I feel like we could save everyone time if we checked those things right at the start of each call. Worth noting that pattern somewhere."
+
+
+
+
+ 🔧 Procedural memory written
+ "Phone screen gate: (1) confirm work-auth in first 5 min, (2) ask candidate to explain Raft — weak answers correlate with underperformance on distributed systems work."
+
+
+
+
+
+ The recruiter said "worth noting." The agent recognised it as a reusable process and chose the right memory type without being asked.
+
+
+
+
+
+
+
Phase 6
+
Recruiter — Charlie Kim (HIRE)
+
+
+
+
+
Before the screen
+
+ "About to jump on a screen with Charlie Kim. Do we have anything on how to run these calls?"
+
+
+ → memory_search("phone screen process")
+ Retrieves the gate checklist from Phase 5
+
+ The checklist was written by a different agent session. This session retrieved and applied it cold.
+
+
+
+
+
+
+
Phase 7
+
Crystallize — one week of screens → a reusable Skill
+
+
+
+ "We've wrapped the first week of screens. I want to turn what we learned into something the whole recruiting team can reuse — a proper runbook for future phone screens."
+
+ Human reviews → thv skills push → OCI artifact.
+ Originals archived with crystallized_into pointer.
+
+
+
+
+
One week of lived experience → a versioned runbook any recruiter can follow from day one.
+
+
+
+
+
What We Just Saw
+
+
+
+
📄
+
Resource — reference docs agents discover through MCP; protected from modification
+
+
+
🧠
+
Semantic — company-wide facts written once, recalled by any session with no explicit handoff
+
+
+
📅
+
Episodic — time-indexed events building a shared pipeline log across recruiter sessions
+
+
+
🔧
+
Procedural — process knowledge that emerged from failure; retrieved by a session that never wrote it
+
+
+
✨
+
Crystallization — lived team experience promoted into a versioned Skill the whole org can distribute
+
+
+
+
+ Any MCP-compatible agent. One config file. Shared memory across every session.
+
+
+
+
+
+
References
+
+
+
A Survey on the Memory Mechanism of LLM-based Agents
+
arXiv:2603.07670 · 2025
+
Taxonomy of memory types and operations lifecycle (acquire, manage, utilize). The "manage" phase — consolidation, conflict resolution, staleness — is the most neglected in practice. Informed our lifecycle design and the 24 h background job.
+
+
+
The LinkedIn Generative AI Application Tech Stack: Personalization with Cognitive Memory Agent
+
LinkedIn Engineering Blog · 2024
+
Production deployment of hierarchical episodic + aggregated semantic + procedural memory at LinkedIn scale. Trust-weighted retrieval and per-application isolation. Informed our comparison table and the prioritisation of conflict detection and user control.
+
+
+
A Practical Guide to Implementing Memory in LLM Applications
+
Towards Data Science · 2024
+
Practitioner analysis of memory degradation: staleness and trust neglect are the top two causes. Recommends human-authored memory outranking agent-authored, and explicit staleness scoring — both implemented here.
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/proposals/2026-04-22-shared-memory-server.md b/docs/proposals/2026-04-22-shared-memory-server.md
new file mode 100644
index 0000000000..899d7b15e1
--- /dev/null
+++ b/docs/proposals/2026-04-22-shared-memory-server.md
@@ -0,0 +1,312 @@
+# Shared Long-Term Memory Server
+
+**Date:** 2026-04-22
+**Status:** Implementation in progress (Plan 1 of 3 complete)
+
+---
+
+## Problem
+
+ToolHive manages MCPs (tools) and Skills (procedural knowledge as OCI artifacts). The missing
+primitive is **shared long-term memory**: a team-wide knowledge store that agents can query and
+contribute to across sessions.
+
+Without it, every agent session starts cold. Facts learned by one agent are invisible to others.
+Patterns that emerge from repeated interactions are lost when the session ends.
+
+---
+
+## Memory Types
+
+Two long-term memory namespaces are in scope:
+
+| Type | Purpose | Example |
+|---|---|---|
+| `semantic` | Aggregated facts and world-state knowledge | "Company does not sponsor visas" |
+| `procedural` | How-to knowledge, heuristics, SOPs | "Always run `task lint-fix` before committing" |
+| `episodic` | Time-indexed event records | "Recruiter archived candidate on 2024-03-15 — visa required" |
+
+**Out of scope:** working memory and conversational memory — agents handle those internally via
+their context window.
+
+---
+
+## Architecture
+
+### System Workload
+
+The memory server is ToolHive's first **system workload** — a managed MCP server auto-provisioned
+by ToolHive rather than explicitly started by users. Key properties:
+
+- Auto-provisioned on first use (`thv memory init`)
+- Persistent — excluded from `thv stop --all`
+- Singleton per scope (one per team in `thv serve` mode)
+- Registered in the registry under the reserved name `toolhive.memory`
+
+### Transport
+
+The memory server uses **MCP streamable HTTP** transport (not stdio). Agents connect via
+`http://:8080/mcp`. A `/health` liveness probe is available at the same host.
+
+### Pluggable Backends
+
+Three independent interfaces, configured via `memory-server.yaml`:
+
+```yaml
+storage:
+ provider: sqlite # sqlite (default) | postgres | mongodb
+ dsn: /data/memory.db
+
+vector:
+ provider: sqlite-vec # sqlite-vec (default) | qdrant | pgvector
+ url: ""
+
+embedder:
+ provider: ollama # ollama (default) | openai | cohere
+ model: nomic-embed-text
+ url: http://localhost:11434
+
+server:
+ host: 0.0.0.0
+ port: 8080
+ lifecycle_interval_hours: 24
+```
+
+Zero-infra teams use SQLite defaults with no external dependencies. Teams with Postgres can
+collapse both storage and vector into pgvector.
+
+### Deployment Modes
+
+**Local (`thv` CLI):** Personal memory, local container, SQLite defaults.
+
+**Team (`thv serve`):** Shared instance; all team agents connect via the API server proxy.
+Auth enforced via existing OIDC middleware.
+
+**Kubernetes (`thv-operator`):** New `MCPMemoryServer` CRD (Plan 3). Operator reconciles to
+`Deployment + Service + PVC`.
+
+---
+
+## MCP Tool Surface
+
+Agents consume memory exactly like any other MCP — no special integration.
+
+| Tool | Description |
+|---|---|
+| `memory_remember` | Write a memory. Runs conflict detection; returns conflicts if similarity > 0.85 |
+| `memory_search` | Semantic vector search, results ranked by composite trust+staleness score |
+| `memory_recall` | Fetch a specific entry by ID, including full revision history |
+| `memory_forget` | Delete a memory |
+| `memory_update` | Correct content; previous version saved to revision history |
+| `memory_flag` | Mark as potentially stale without deleting |
+| `memory_list` | Structured listing with filters: type, author, tags, time-range |
+| `memory_consolidate` | Merge related entries; originals archived with pointer |
+| `memory_crystallize` | Promote procedural memories to a Skill scaffold for human authoring |
+
+### Conflict Detection
+
+On `memory_remember`, the server embeds the new content and searches for similar active entries.
+If any entry has cosine similarity > 0.85, the write is blocked and the agent receives a
+`conflict_detected` response with the conflicting entries. The agent decides: force-write,
+update the existing entry, or abort. No LLM inference — the agent (which has context) is better
+placed to judge whether two similar entries actually conflict.
+
+### Search Ranking
+
+`memory_search` returns results ranked by a composite score that combines vector similarity with
+the entry's trust and staleness signals:
+
+```
+composite = similarity × trust_score × (1 - 0.3 × staleness_score)
+```
+
+This prevents a high-similarity but flagged or stale entry from ranking above a fresher,
+more trusted one.
+
+---
+
+## Trust and Staleness Scoring
+
+### Trust Score
+
+```
+trust_score = author_weight
+ × age_decay(created_at, half_life=180d)
+ × (1 - min(corrections × 0.05, 0.30))
+ × (0.5 if flagged else 1.0)
+
+author_weight: human=1.0, agent=0.7
+```
+
+### Staleness Score
+
+```
+staleness_score = normalize(days_since_last_access, max=90d)
+ + (0.3 if flagged)
+ + min(corrections × 0.1, 0.3)
+```
+
+Entries with `staleness_score > 0.8` surface in the lifecycle audit log every 24 hours.
+
+---
+
+## Skills Relationship
+
+Skills (existing) and procedural memory are the same kind of knowledge at different stages of
+maturity:
+
+```
+Agent/human observes something
+ │
+ ▼
+ Procedural Memory ← fluid, emergent, evolving
+ (memory server)
+ │
+ (patterns emerge,
+ human crystallizes)
+ │
+ ▼
+ Skill (OCI) ← crystallized, versioned, distributed
+ (existing skills system)
+```
+
+`memory_crystallize` bridges the gap: it takes stable procedural memory entries and produces a
+`SKILL.md` scaffold for a human to author and push via `thv skills push`. The source entries are
+archived with a `crystallized_into` pointer so search returns the canonical Skill instead.
+
+---
+
+## Recommended Memory Activation Strategy
+
+Not every agent interaction should touch the memory server. The recommended approach is a
+three-tier strategy:
+
+### Tier 1 — Session-boundary injection (always)
+
+At the **start** of every task-bearing session, the system prompt instructs the agent to run one
+`memory_search` call with the task description before doing anything else. This is silent,
+cheap (one vector search), and covers the most valuable case: cross-session continuity.
+
+```
+Before starting work, call memory_search with the task description to load
+relevant team knowledge. Do this once, silently — do not explain it to the user.
+```
+
+At the **end** of a session, the agent writes what was discovered or decided that would be
+useful to a different agent in a future session.
+
+### Tier 2 — Signal-based mid-session reads (agent-decided)
+
+The system prompt instructs the agent to call `memory_search` when it encounters:
+
+1. **Uncertainty** — "I don't have enough context to answer this confidently"
+2. **Cross-session references** — phrases like "last time", "previously", "we decided",
+ "our policy", "do you remember"
+3. **Team-specific facts** — questions about preferences, conventions, or domain knowledge
+ not in the codebase or current context
+
+### Tier 3 — Write on observation, not speculation
+
+The agent calls `memory_remember` only for facts that:
+- Were not already in the search results from Tier 1
+- Would be useful to a **different** agent in a **future** session
+
+The system prompt guidance:
+
+```
+Write a memory when you learn something that:
+- corrects or refines an existing fact (use memory_update instead)
+- is a team decision, constraint, or policy that will apply again
+- is a recurring pattern observed more than once
+
+Do NOT write memories for facts already in the codebase, documentation,
+or the current conversation context.
+```
+
+### Why not automatic ingestion?
+
+Auto-ingestion (LinkedIn's streaming pipeline approach) requires:
+- An LLM call in the ingestion path to extract facts from raw transcripts
+- Quality control to decide what is worth persisting
+- Evaluation tooling to measure ingestion accuracy
+
+These are deferred to a later plan. The explicit tool-use model is more predictable and
+debuggable for a v1, and the agent (which has full context) makes better judgments about
+what is worth writing than a pipeline operating on raw text.
+
+---
+
+## Comparison with LinkedIn's Cognitive Memory Agent
+
+LinkedIn's CMA (described in their [engineering blog](https://www.linkedin.com/blog/engineering/ai/the-linkedin-generative-ai-application-tech-stack-personalization-with-cognitive-memory-agent))
+is the closest public reference. Key differences:
+
+| Dimension | LinkedIn CMA | ToolHive Memory |
+|---|---|---|
+| Conflict detection | On roadmap | Implemented (cosine > 0.85) |
+| Trust/staleness scoring | Time-based prioritization planned | Implemented (full formula + background job) |
+| User control (list/update/delete/flag) | On roadmap | Implemented |
+| Search ranking | Implicit | Composite score: similarity × trust × (1 − staleness penalty) |
+| Episodic memory type | Distinct tier | `TypeEpisodic` with time-range `ListFilter` |
+| Retrieval orchestration | LLM-powered multi-step planner | Agent calls tools directly (agent IS the orchestrator) |
+| Hierarchical aggregation | Auto tree: events → summaries → facets | Explicit: `memory_consolidate` + `memory_crystallize` |
+| Tenant isolation | Per-application isolated stores | Auth at proxy layer; storage-level namespace deferred |
+| Auto ingestion pipeline | Streaming + batch LLM extraction | Deferred; `memory_distill` returns candidates for agent review |
+
+---
+
+## Implementation Status
+
+### Plan 1 — Memory server core (this branch)
+
+- [x] `pkg/memory/` — domain types, interfaces (`Store`, `VectorStore`, `Embedder`), scoring, service
+- [x] `pkg/memory/sqlite/` — SQLite Store + VectorStore (Go cosine similarity, no CGo)
+- [x] `pkg/memory/embedder/ollama/` — Ollama HTTP embedder
+- [x] `pkg/memory/mocks/` — gomock mocks for all three interfaces
+- [x] `cmd/thv-memory/` — MCP server binary (streamable HTTP on `/mcp`)
+- [x] `cmd/thv-memory/lifecycle/` — 24h background job (TTL expiry, score recomputation)
+- [x] `cmd/thv-memory/tools/` — 9 MCP tool handlers
+- [x] Integration test (SQLite + fake embedder, end-to-end remember → search → delete)
+
+### Plan 2 — CLI + system workload integration (not started)
+
+- `thv memory` subcommand tree
+- System workload auto-provisioning (`thv memory init`)
+- Registry integration under `toolhive.memory`
+
+### Plan 3 — Kubernetes operator (not started)
+
+- `MCPMemoryServer` CRD
+- Operator controller: reconciles to `Deployment + Service + PVC`
+- `MCPRegistry` integration
+
+---
+
+## Package Layout
+
+```
+pkg/memory/
+├── types.go — Entry, Revision, ListFilter, VectorFilter, scoring types
+├── interfaces.go — Store, VectorStore, Embedder interfaces + mockgen directives
+├── service.go — Orchestration: conflict detection, remember, search
+├── scoring.go — ComputeTrustScore, ComputeStalenessScore
+├── errors.go — ErrNotFound
+├── mocks/ — Generated gomock mocks
+├── sqlite/
+│ ├── db.go — DB wrapper, WAL pragmas, goose migrations
+│ ├── store.go — Store implementation
+│ ├── vector.go — VectorStore implementation (Go cosine similarity)
+│ └── migrations/ — goose SQL migrations
+└── embedder/
+ └── ollama/ — Ollama HTTP embedder
+
+cmd/thv-memory/
+├── main.go — Entry point, HTTP server lifecycle
+├── server.go — MCP server construction, tool registration, HTTP handler
+├── config.go — YAML config with defaults
+├── lifecycle/
+│ └── job.go — Background maintenance job
+└── tools/ — One file per MCP tool
+ ├── remember.go, search.go, recall.go, forget.go, update.go
+ ├── flag.go, list.go, consolidate.go, crystallize.go
+```
diff --git a/pkg/memory/embedder/ollama/embedder.go b/pkg/memory/embedder/ollama/embedder.go
new file mode 100644
index 0000000000..00ca3817ee
--- /dev/null
+++ b/pkg/memory/embedder/ollama/embedder.go
@@ -0,0 +1,88 @@
+// SPDX-FileCopyrightText: Copyright 2025 Stacklok, Inc.
+// SPDX-License-Identifier: Apache-2.0
+
+// Package ollama provides a memory.Embedder backed by a local Ollama server.
+package ollama
+
+import (
+ "bytes"
+ "context"
+ "encoding/json"
+ "fmt"
+ "io"
+ "net/http"
+ "net/url"
+
+ "github.com/stacklok/toolhive/pkg/memory"
+)
+
+// Embedder calls the Ollama /api/embeddings endpoint.
+type Embedder struct {
+ baseURL string
+ model string
+ dimensions int
+ client *http.Client
+}
+
+// New creates an Ollama embedder. It probes the server once to discover the
+// embedding dimension. Returns an error if the server is unreachable or the
+// model returns an empty vector.
+func New(baseURL, model string) (*Embedder, error) {
+ if _, err := url.ParseRequestURI(baseURL); err != nil {
+ return nil, fmt.Errorf("invalid Ollama URL %q: %w", baseURL, err)
+ }
+ if model == "" {
+ return nil, fmt.Errorf("model name is required")
+ }
+ e := &Embedder{baseURL: baseURL, model: model, client: &http.Client{}}
+
+ emb, err := e.Embed(context.Background(), "probe")
+ if err != nil {
+ return nil, fmt.Errorf("probing Ollama embedder: %w", err)
+ }
+ e.dimensions = len(emb)
+ return e, nil
+}
+
+// Embed calls the Ollama /api/embeddings endpoint and returns the vector.
+func (e *Embedder) Embed(ctx context.Context, text string) ([]float32, error) {
+ body, err := json.Marshal(map[string]string{"model": e.model, "prompt": text})
+ if err != nil {
+ return nil, err
+ }
+
+ req, err := http.NewRequestWithContext(ctx, http.MethodPost, e.baseURL+"/api/embeddings", bytes.NewReader(body))
+ if err != nil {
+ return nil, err
+ }
+ req.Header.Set("Content-Type", "application/json")
+
+ resp, err := e.client.Do(req)
+ if err != nil {
+ return nil, fmt.Errorf("calling Ollama: %w", err)
+ }
+ defer func() {
+ _, _ = io.Copy(io.Discard, resp.Body)
+ _ = resp.Body.Close()
+ }()
+
+ if resp.StatusCode != http.StatusOK {
+ return nil, fmt.Errorf("ollama returned status %d", resp.StatusCode)
+ }
+
+ var result struct {
+ Embedding []float32 `json:"embedding"`
+ }
+ if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
+ return nil, fmt.Errorf("decoding Ollama response: %w", err)
+ }
+ if len(result.Embedding) == 0 {
+ return nil, fmt.Errorf("ollama returned empty embedding")
+ }
+ return result.Embedding, nil
+}
+
+// Dimensions returns the fixed vector length produced by this embedder.
+func (e *Embedder) Dimensions() int { return e.dimensions }
+
+var _ memory.Embedder = (*Embedder)(nil)
diff --git a/pkg/memory/embedder/ollama/embedder_test.go b/pkg/memory/embedder/ollama/embedder_test.go
new file mode 100644
index 0000000000..bb601fae1b
--- /dev/null
+++ b/pkg/memory/embedder/ollama/embedder_test.go
@@ -0,0 +1,35 @@
+// SPDX-FileCopyrightText: Copyright 2025 Stacklok, Inc.
+// SPDX-License-Identifier: Apache-2.0
+
+package ollama_test
+
+import (
+ "context"
+ "encoding/json"
+ "net/http"
+ "net/http/httptest"
+ "testing"
+
+ "github.com/stretchr/testify/require"
+
+ "github.com/stacklok/toolhive/pkg/memory/embedder/ollama"
+)
+
+func TestEmbed(t *testing.T) {
+ t.Parallel()
+
+ want := []float32{0.1, 0.2, 0.3}
+ srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+ require.Equal(t, "/api/embeddings", r.URL.Path)
+ _ = json.NewEncoder(w).Encode(map[string]any{"embedding": want})
+ }))
+ t.Cleanup(srv.Close)
+
+ e, err := ollama.New(srv.URL, "nomic-embed-text")
+ require.NoError(t, err)
+ require.Equal(t, 3, e.Dimensions())
+
+ got, err := e.Embed(context.Background(), "hello world")
+ require.NoError(t, err)
+ require.InDeltaSlice(t, want, got, 0.001)
+}
diff --git a/pkg/memory/errors.go b/pkg/memory/errors.go
new file mode 100644
index 0000000000..77aa21a1da
--- /dev/null
+++ b/pkg/memory/errors.go
@@ -0,0 +1,14 @@
+// SPDX-FileCopyrightText: Copyright 2025 Stacklok, Inc.
+// SPDX-License-Identifier: Apache-2.0
+
+package memory
+
+import "errors"
+
+// ErrNotFound is returned when a memory entry does not exist.
+var ErrNotFound = errors.New("memory entry not found")
+
+// ErrReadOnly is returned when an agent attempts to mutate an entry whose
+// source type is read-only (SourceSkill or SourceResource). Use the
+// management REST API to modify resource entries.
+var ErrReadOnly = errors.New("memory entry is read-only")
diff --git a/pkg/memory/interfaces.go b/pkg/memory/interfaces.go
new file mode 100644
index 0000000000..5f26d39076
--- /dev/null
+++ b/pkg/memory/interfaces.go
@@ -0,0 +1,51 @@
+// SPDX-FileCopyrightText: Copyright 2025 Stacklok, Inc.
+// SPDX-License-Identifier: Apache-2.0
+
+package memory
+
+import "context"
+
+//go:generate mockgen -destination mocks/mock_store.go -package mocks github.com/stacklok/toolhive/pkg/memory Store
+//go:generate mockgen -destination mocks/mock_vector.go -package mocks github.com/stacklok/toolhive/pkg/memory VectorStore
+//go:generate mockgen -destination mocks/mock_embedder.go -package mocks github.com/stacklok/toolhive/pkg/memory Embedder
+
+// Store is the structured persistence layer for memory entries.
+// It handles CRUD, lifecycle transitions, and score updates.
+// Implementations must be safe for concurrent use.
+type Store interface {
+ Create(ctx context.Context, entry Entry) error
+ Get(ctx context.Context, id string) (Entry, error)
+ // Update replaces the content of an existing entry and appends the
+ // previous content to History. The embedding must be recomputed by
+ // the caller (Service) after this call succeeds.
+ Update(ctx context.Context, id string, content string, author AuthorType, correctionNote string) error
+ Flag(ctx context.Context, id string, reason string) error
+ Unflag(ctx context.Context, id string) error
+ Delete(ctx context.Context, id string) error
+ List(ctx context.Context, filter ListFilter) ([]Entry, error)
+ Archive(ctx context.Context, id string, reason ArchiveReason, ref string) error
+ IncrementAccess(ctx context.Context, id string) error
+ UpdateScores(ctx context.Context, id string, trustScore, stalenessScore float32) error
+ // ListExpired returns all active entries whose ExpiresAt is in the past.
+ ListExpired(ctx context.Context) ([]Entry, error)
+ // ListActive returns all non-archived entries for score recomputation.
+ ListActive(ctx context.Context) ([]Entry, error)
+}
+
+// VectorStore stores and queries embedding vectors for memory entries.
+// Implementations must be safe for concurrent use.
+type VectorStore interface {
+ // Upsert stores or replaces the embedding for the given entry ID.
+ Upsert(ctx context.Context, id string, embedding []float32) error
+ // Search returns the topK entries most similar to query, restricted by filter.
+ Search(ctx context.Context, query []float32, topK int, filter VectorFilter) ([]ScoredID, error)
+ Delete(ctx context.Context, id string) error
+}
+
+// Embedder converts text to a fixed-dimension float32 vector.
+// Implementations must be safe for concurrent use.
+type Embedder interface {
+ Embed(ctx context.Context, text string) ([]float32, error)
+ // Dimensions returns the fixed vector length produced by this embedder.
+ Dimensions() int
+}
diff --git a/pkg/memory/mocks/mock_embedder.go b/pkg/memory/mocks/mock_embedder.go
new file mode 100644
index 0000000000..ee6fbc7b24
--- /dev/null
+++ b/pkg/memory/mocks/mock_embedder.go
@@ -0,0 +1,70 @@
+// Code generated by MockGen. DO NOT EDIT.
+// Source: github.com/stacklok/toolhive/pkg/memory (interfaces: Embedder)
+//
+// Generated by this command:
+//
+// mockgen -destination mocks/mock_embedder.go -package mocks github.com/stacklok/toolhive/pkg/memory Embedder
+//
+
+// Package mocks is a generated GoMock package.
+package mocks
+
+import (
+ context "context"
+ reflect "reflect"
+
+ gomock "go.uber.org/mock/gomock"
+)
+
+// MockEmbedder is a mock of Embedder interface.
+type MockEmbedder struct {
+ ctrl *gomock.Controller
+ recorder *MockEmbedderMockRecorder
+ isgomock struct{}
+}
+
+// MockEmbedderMockRecorder is the mock recorder for MockEmbedder.
+type MockEmbedderMockRecorder struct {
+ mock *MockEmbedder
+}
+
+// NewMockEmbedder creates a new mock instance.
+func NewMockEmbedder(ctrl *gomock.Controller) *MockEmbedder {
+ mock := &MockEmbedder{ctrl: ctrl}
+ mock.recorder = &MockEmbedderMockRecorder{mock}
+ return mock
+}
+
+// EXPECT returns an object that allows the caller to indicate expected use.
+func (m *MockEmbedder) EXPECT() *MockEmbedderMockRecorder {
+ return m.recorder
+}
+
+// Dimensions mocks base method.
+func (m *MockEmbedder) Dimensions() int {
+ m.ctrl.T.Helper()
+ ret := m.ctrl.Call(m, "Dimensions")
+ ret0, _ := ret[0].(int)
+ return ret0
+}
+
+// Dimensions indicates an expected call of Dimensions.
+func (mr *MockEmbedderMockRecorder) Dimensions() *gomock.Call {
+ mr.mock.ctrl.T.Helper()
+ return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Dimensions", reflect.TypeOf((*MockEmbedder)(nil).Dimensions))
+}
+
+// Embed mocks base method.
+func (m *MockEmbedder) Embed(ctx context.Context, text string) ([]float32, error) {
+ m.ctrl.T.Helper()
+ ret := m.ctrl.Call(m, "Embed", ctx, text)
+ ret0, _ := ret[0].([]float32)
+ ret1, _ := ret[1].(error)
+ return ret0, ret1
+}
+
+// Embed indicates an expected call of Embed.
+func (mr *MockEmbedderMockRecorder) Embed(ctx, text any) *gomock.Call {
+ mr.mock.ctrl.T.Helper()
+ return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Embed", reflect.TypeOf((*MockEmbedder)(nil).Embed), ctx, text)
+}
diff --git a/pkg/memory/mocks/mock_store.go b/pkg/memory/mocks/mock_store.go
new file mode 100644
index 0000000000..29e3d1f860
--- /dev/null
+++ b/pkg/memory/mocks/mock_store.go
@@ -0,0 +1,214 @@
+// Code generated by MockGen. DO NOT EDIT.
+// Source: github.com/stacklok/toolhive/pkg/memory (interfaces: Store)
+//
+// Generated by this command:
+//
+// mockgen -destination mocks/mock_store.go -package mocks github.com/stacklok/toolhive/pkg/memory Store
+//
+
+// Package mocks is a generated GoMock package.
+package mocks
+
+import (
+ context "context"
+ reflect "reflect"
+
+ memory "github.com/stacklok/toolhive/pkg/memory"
+ gomock "go.uber.org/mock/gomock"
+)
+
+// MockStore is a mock of Store interface.
+type MockStore struct {
+ ctrl *gomock.Controller
+ recorder *MockStoreMockRecorder
+ isgomock struct{}
+}
+
+// MockStoreMockRecorder is the mock recorder for MockStore.
+type MockStoreMockRecorder struct {
+ mock *MockStore
+}
+
+// NewMockStore creates a new mock instance.
+func NewMockStore(ctrl *gomock.Controller) *MockStore {
+ mock := &MockStore{ctrl: ctrl}
+ mock.recorder = &MockStoreMockRecorder{mock}
+ return mock
+}
+
+// EXPECT returns an object that allows the caller to indicate expected use.
+func (m *MockStore) EXPECT() *MockStoreMockRecorder {
+ return m.recorder
+}
+
+// Archive mocks base method.
+func (m *MockStore) Archive(ctx context.Context, id string, reason memory.ArchiveReason, ref string) error {
+ m.ctrl.T.Helper()
+ ret := m.ctrl.Call(m, "Archive", ctx, id, reason, ref)
+ ret0, _ := ret[0].(error)
+ return ret0
+}
+
+// Archive indicates an expected call of Archive.
+func (mr *MockStoreMockRecorder) Archive(ctx, id, reason, ref any) *gomock.Call {
+ mr.mock.ctrl.T.Helper()
+ return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Archive", reflect.TypeOf((*MockStore)(nil).Archive), ctx, id, reason, ref)
+}
+
+// Create mocks base method.
+func (m *MockStore) Create(ctx context.Context, entry memory.Entry) error {
+ m.ctrl.T.Helper()
+ ret := m.ctrl.Call(m, "Create", ctx, entry)
+ ret0, _ := ret[0].(error)
+ return ret0
+}
+
+// Create indicates an expected call of Create.
+func (mr *MockStoreMockRecorder) Create(ctx, entry any) *gomock.Call {
+ mr.mock.ctrl.T.Helper()
+ return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Create", reflect.TypeOf((*MockStore)(nil).Create), ctx, entry)
+}
+
+// Delete mocks base method.
+func (m *MockStore) Delete(ctx context.Context, id string) error {
+ m.ctrl.T.Helper()
+ ret := m.ctrl.Call(m, "Delete", ctx, id)
+ ret0, _ := ret[0].(error)
+ return ret0
+}
+
+// Delete indicates an expected call of Delete.
+func (mr *MockStoreMockRecorder) Delete(ctx, id any) *gomock.Call {
+ mr.mock.ctrl.T.Helper()
+ return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Delete", reflect.TypeOf((*MockStore)(nil).Delete), ctx, id)
+}
+
+// Flag mocks base method.
+func (m *MockStore) Flag(ctx context.Context, id, reason string) error {
+ m.ctrl.T.Helper()
+ ret := m.ctrl.Call(m, "Flag", ctx, id, reason)
+ ret0, _ := ret[0].(error)
+ return ret0
+}
+
+// Flag indicates an expected call of Flag.
+func (mr *MockStoreMockRecorder) Flag(ctx, id, reason any) *gomock.Call {
+ mr.mock.ctrl.T.Helper()
+ return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Flag", reflect.TypeOf((*MockStore)(nil).Flag), ctx, id, reason)
+}
+
+// Get mocks base method.
+func (m *MockStore) Get(ctx context.Context, id string) (memory.Entry, error) {
+ m.ctrl.T.Helper()
+ ret := m.ctrl.Call(m, "Get", ctx, id)
+ ret0, _ := ret[0].(memory.Entry)
+ ret1, _ := ret[1].(error)
+ return ret0, ret1
+}
+
+// Get indicates an expected call of Get.
+func (mr *MockStoreMockRecorder) Get(ctx, id any) *gomock.Call {
+ mr.mock.ctrl.T.Helper()
+ return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Get", reflect.TypeOf((*MockStore)(nil).Get), ctx, id)
+}
+
+// IncrementAccess mocks base method.
+func (m *MockStore) IncrementAccess(ctx context.Context, id string) error {
+ m.ctrl.T.Helper()
+ ret := m.ctrl.Call(m, "IncrementAccess", ctx, id)
+ ret0, _ := ret[0].(error)
+ return ret0
+}
+
+// IncrementAccess indicates an expected call of IncrementAccess.
+func (mr *MockStoreMockRecorder) IncrementAccess(ctx, id any) *gomock.Call {
+ mr.mock.ctrl.T.Helper()
+ return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "IncrementAccess", reflect.TypeOf((*MockStore)(nil).IncrementAccess), ctx, id)
+}
+
+// List mocks base method.
+func (m *MockStore) List(ctx context.Context, filter memory.ListFilter) ([]memory.Entry, error) {
+ m.ctrl.T.Helper()
+ ret := m.ctrl.Call(m, "List", ctx, filter)
+ ret0, _ := ret[0].([]memory.Entry)
+ ret1, _ := ret[1].(error)
+ return ret0, ret1
+}
+
+// List indicates an expected call of List.
+func (mr *MockStoreMockRecorder) List(ctx, filter any) *gomock.Call {
+ mr.mock.ctrl.T.Helper()
+ return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "List", reflect.TypeOf((*MockStore)(nil).List), ctx, filter)
+}
+
+// ListActive mocks base method.
+func (m *MockStore) ListActive(ctx context.Context) ([]memory.Entry, error) {
+ m.ctrl.T.Helper()
+ ret := m.ctrl.Call(m, "ListActive", ctx)
+ ret0, _ := ret[0].([]memory.Entry)
+ ret1, _ := ret[1].(error)
+ return ret0, ret1
+}
+
+// ListActive indicates an expected call of ListActive.
+func (mr *MockStoreMockRecorder) ListActive(ctx any) *gomock.Call {
+ mr.mock.ctrl.T.Helper()
+ return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ListActive", reflect.TypeOf((*MockStore)(nil).ListActive), ctx)
+}
+
+// ListExpired mocks base method.
+func (m *MockStore) ListExpired(ctx context.Context) ([]memory.Entry, error) {
+ m.ctrl.T.Helper()
+ ret := m.ctrl.Call(m, "ListExpired", ctx)
+ ret0, _ := ret[0].([]memory.Entry)
+ ret1, _ := ret[1].(error)
+ return ret0, ret1
+}
+
+// ListExpired indicates an expected call of ListExpired.
+func (mr *MockStoreMockRecorder) ListExpired(ctx any) *gomock.Call {
+ mr.mock.ctrl.T.Helper()
+ return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ListExpired", reflect.TypeOf((*MockStore)(nil).ListExpired), ctx)
+}
+
+// Unflag mocks base method.
+func (m *MockStore) Unflag(ctx context.Context, id string) error {
+ m.ctrl.T.Helper()
+ ret := m.ctrl.Call(m, "Unflag", ctx, id)
+ ret0, _ := ret[0].(error)
+ return ret0
+}
+
+// Unflag indicates an expected call of Unflag.
+func (mr *MockStoreMockRecorder) Unflag(ctx, id any) *gomock.Call {
+ mr.mock.ctrl.T.Helper()
+ return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Unflag", reflect.TypeOf((*MockStore)(nil).Unflag), ctx, id)
+}
+
+// Update mocks base method.
+func (m *MockStore) Update(ctx context.Context, id, content string, author memory.AuthorType, correctionNote string) error {
+ m.ctrl.T.Helper()
+ ret := m.ctrl.Call(m, "Update", ctx, id, content, author, correctionNote)
+ ret0, _ := ret[0].(error)
+ return ret0
+}
+
+// Update indicates an expected call of Update.
+func (mr *MockStoreMockRecorder) Update(ctx, id, content, author, correctionNote any) *gomock.Call {
+ mr.mock.ctrl.T.Helper()
+ return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Update", reflect.TypeOf((*MockStore)(nil).Update), ctx, id, content, author, correctionNote)
+}
+
+// UpdateScores mocks base method.
+func (m *MockStore) UpdateScores(ctx context.Context, id string, trustScore, stalenessScore float32) error {
+ m.ctrl.T.Helper()
+ ret := m.ctrl.Call(m, "UpdateScores", ctx, id, trustScore, stalenessScore)
+ ret0, _ := ret[0].(error)
+ return ret0
+}
+
+// UpdateScores indicates an expected call of UpdateScores.
+func (mr *MockStoreMockRecorder) UpdateScores(ctx, id, trustScore, stalenessScore any) *gomock.Call {
+ mr.mock.ctrl.T.Helper()
+ return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "UpdateScores", reflect.TypeOf((*MockStore)(nil).UpdateScores), ctx, id, trustScore, stalenessScore)
+}
diff --git a/pkg/memory/mocks/mock_vector.go b/pkg/memory/mocks/mock_vector.go
new file mode 100644
index 0000000000..0df2f3d80c
--- /dev/null
+++ b/pkg/memory/mocks/mock_vector.go
@@ -0,0 +1,85 @@
+// Code generated by MockGen. DO NOT EDIT.
+// Source: github.com/stacklok/toolhive/pkg/memory (interfaces: VectorStore)
+//
+// Generated by this command:
+//
+// mockgen -destination mocks/mock_vector.go -package mocks github.com/stacklok/toolhive/pkg/memory VectorStore
+//
+
+// Package mocks is a generated GoMock package.
+package mocks
+
+import (
+ context "context"
+ reflect "reflect"
+
+ memory "github.com/stacklok/toolhive/pkg/memory"
+ gomock "go.uber.org/mock/gomock"
+)
+
+// MockVectorStore is a mock of VectorStore interface.
+type MockVectorStore struct {
+ ctrl *gomock.Controller
+ recorder *MockVectorStoreMockRecorder
+ isgomock struct{}
+}
+
+// MockVectorStoreMockRecorder is the mock recorder for MockVectorStore.
+type MockVectorStoreMockRecorder struct {
+ mock *MockVectorStore
+}
+
+// NewMockVectorStore creates a new mock instance.
+func NewMockVectorStore(ctrl *gomock.Controller) *MockVectorStore {
+ mock := &MockVectorStore{ctrl: ctrl}
+ mock.recorder = &MockVectorStoreMockRecorder{mock}
+ return mock
+}
+
+// EXPECT returns an object that allows the caller to indicate expected use.
+func (m *MockVectorStore) EXPECT() *MockVectorStoreMockRecorder {
+ return m.recorder
+}
+
+// Delete mocks base method.
+func (m *MockVectorStore) Delete(ctx context.Context, id string) error {
+ m.ctrl.T.Helper()
+ ret := m.ctrl.Call(m, "Delete", ctx, id)
+ ret0, _ := ret[0].(error)
+ return ret0
+}
+
+// Delete indicates an expected call of Delete.
+func (mr *MockVectorStoreMockRecorder) Delete(ctx, id any) *gomock.Call {
+ mr.mock.ctrl.T.Helper()
+ return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Delete", reflect.TypeOf((*MockVectorStore)(nil).Delete), ctx, id)
+}
+
+// Search mocks base method.
+func (m *MockVectorStore) Search(ctx context.Context, query []float32, topK int, filter memory.VectorFilter) ([]memory.ScoredID, error) {
+ m.ctrl.T.Helper()
+ ret := m.ctrl.Call(m, "Search", ctx, query, topK, filter)
+ ret0, _ := ret[0].([]memory.ScoredID)
+ ret1, _ := ret[1].(error)
+ return ret0, ret1
+}
+
+// Search indicates an expected call of Search.
+func (mr *MockVectorStoreMockRecorder) Search(ctx, query, topK, filter any) *gomock.Call {
+ mr.mock.ctrl.T.Helper()
+ return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Search", reflect.TypeOf((*MockVectorStore)(nil).Search), ctx, query, topK, filter)
+}
+
+// Upsert mocks base method.
+func (m *MockVectorStore) Upsert(ctx context.Context, id string, embedding []float32) error {
+ m.ctrl.T.Helper()
+ ret := m.ctrl.Call(m, "Upsert", ctx, id, embedding)
+ ret0, _ := ret[0].(error)
+ return ret0
+}
+
+// Upsert indicates an expected call of Upsert.
+func (mr *MockVectorStoreMockRecorder) Upsert(ctx, id, embedding any) *gomock.Call {
+ mr.mock.ctrl.T.Helper()
+ return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Upsert", reflect.TypeOf((*MockVectorStore)(nil).Upsert), ctx, id, embedding)
+}
diff --git a/pkg/memory/scoring.go b/pkg/memory/scoring.go
new file mode 100644
index 0000000000..93997aff87
--- /dev/null
+++ b/pkg/memory/scoring.go
@@ -0,0 +1,70 @@
+// SPDX-FileCopyrightText: Copyright 2025 Stacklok, Inc.
+// SPDX-License-Identifier: Apache-2.0
+
+package memory
+
+import (
+ "math"
+ "time"
+)
+
+const (
+ authorWeightHuman = 1.0
+ authorWeightAgent = 0.7
+ halfLifeDays = 180.0
+ maxCorrectionPenalty = 0.30
+ correctionPenaltyPerCorrection = 0.05
+ flagTrustMultiplier = 0.5
+ maxStalenessAccessDays = 90.0
+ flagStalenessBonus = 0.3
+ correctionStalenessPerItem = 0.1
+ maxCorrectionStaleness = 0.3
+)
+
+// ComputeTrustScore returns a value in [0,1] representing how trustworthy
+// this memory entry is. Higher = more trustworthy.
+//
+// Formula: author_weight × age_decay × (1 - correction_penalty) × flag_multiplier
+func ComputeTrustScore(entry Entry) float32 {
+ weight := authorWeightAgent
+ if entry.Author == AuthorHuman {
+ weight = authorWeightHuman
+ }
+
+ ageInDays := time.Since(entry.CreatedAt).Hours() / 24
+ decay := math.Exp(-ageInDays * math.Log(2) / halfLifeDays)
+
+ corrections := len(entry.History)
+ correctionPenalty := math.Min(float64(corrections)*correctionPenaltyPerCorrection, maxCorrectionPenalty)
+
+ flagMultiplier := 1.0
+ if entry.FlaggedAt != nil {
+ flagMultiplier = flagTrustMultiplier
+ }
+
+ score := weight * decay * (1 - correctionPenalty) * flagMultiplier
+ return float32(math.Max(0, math.Min(1, score)))
+}
+
+// ComputeStalenessScore returns a value in [0,1] representing how stale
+// this memory entry is. Higher = more stale (more likely to need review).
+//
+// Formula: access_age_normalized + flag_bonus + correction_bonus
+func ComputeStalenessScore(entry Entry) float32 {
+ lastAccess := entry.LastAccessedAt
+ if lastAccess.IsZero() {
+ lastAccess = entry.CreatedAt
+ }
+ daysSinceAccess := time.Since(lastAccess).Hours() / 24
+ base := math.Min(daysSinceAccess/maxStalenessAccessDays, 1.0)
+
+ flagBonus := 0.0
+ if entry.FlaggedAt != nil {
+ flagBonus = flagStalenessBonus
+ }
+
+ corrections := len(entry.History)
+ correctionBonus := math.Min(float64(corrections)*correctionStalenessPerItem, maxCorrectionStaleness)
+
+ return float32(math.Min(1.0, base+flagBonus+correctionBonus))
+}
diff --git a/pkg/memory/scoring_test.go b/pkg/memory/scoring_test.go
new file mode 100644
index 0000000000..e8a5da982e
--- /dev/null
+++ b/pkg/memory/scoring_test.go
@@ -0,0 +1,138 @@
+// SPDX-FileCopyrightText: Copyright 2025 Stacklok, Inc.
+// SPDX-License-Identifier: Apache-2.0
+
+package memory_test
+
+import (
+ "testing"
+ "time"
+
+ "github.com/stretchr/testify/require"
+
+ "github.com/stacklok/toolhive/pkg/memory"
+)
+
+func TestComputeTrustScore(t *testing.T) {
+ t.Parallel()
+ now := time.Now()
+
+ tests := []struct {
+ name string
+ entry memory.Entry
+ wantMin float32
+ wantMax float32
+ }{
+ {
+ name: "fresh human entry has high trust",
+ entry: memory.Entry{
+ Author: memory.AuthorHuman,
+ CreatedAt: now,
+ },
+ wantMin: 0.95,
+ wantMax: 1.0,
+ },
+ {
+ name: "fresh agent entry has lower trust than human",
+ entry: memory.Entry{
+ Author: memory.AuthorAgent,
+ CreatedAt: now,
+ },
+ wantMin: 0.65,
+ wantMax: 0.75,
+ },
+ {
+ name: "flagged entry has halved trust",
+ entry: func() memory.Entry {
+ ft := now
+ return memory.Entry{
+ Author: memory.AuthorHuman,
+ CreatedAt: now,
+ FlaggedAt: &ft,
+ }
+ }(),
+ wantMin: 0.45,
+ wantMax: 0.55,
+ },
+ {
+ name: "two corrections reduce trust",
+ entry: memory.Entry{
+ Author: memory.AuthorHuman,
+ CreatedAt: now,
+ History: []memory.Revision{{}, {}},
+ },
+ wantMin: 0.85,
+ wantMax: 0.95,
+ },
+ {
+ name: "old entry has decayed trust",
+ entry: memory.Entry{
+ Author: memory.AuthorHuman,
+ CreatedAt: now.AddDate(0, 0, -180), // half-life
+ },
+ wantMin: 0.45,
+ wantMax: 0.55,
+ },
+ }
+
+ for _, tc := range tests {
+ t.Run(tc.name, func(t *testing.T) {
+ t.Parallel()
+ score := memory.ComputeTrustScore(tc.entry)
+ require.GreaterOrEqual(t, score, tc.wantMin, "trust score too low")
+ require.LessOrEqual(t, score, tc.wantMax, "trust score too high")
+ })
+ }
+}
+
+func TestComputeStalenessScore(t *testing.T) {
+ t.Parallel()
+ now := time.Now()
+
+ tests := []struct {
+ name string
+ entry memory.Entry
+ wantMin float32
+ wantMax float32
+ }{
+ {
+ name: "recently accessed entry is fresh",
+ entry: memory.Entry{
+ CreatedAt: now,
+ LastAccessedAt: now,
+ },
+ wantMin: 0.0,
+ wantMax: 0.05,
+ },
+ {
+ name: "entry not accessed for 90 days is stale",
+ entry: memory.Entry{
+ CreatedAt: now.AddDate(0, 0, -90),
+ LastAccessedAt: now.AddDate(0, 0, -90),
+ },
+ wantMin: 0.95,
+ wantMax: 1.0,
+ },
+ {
+ name: "flagged entry adds staleness bonus",
+ entry: func() memory.Entry {
+ ft := now
+ return memory.Entry{
+ CreatedAt: now,
+ LastAccessedAt: now,
+ FlaggedAt: &ft,
+ }
+ }(),
+ wantMin: 0.28,
+ wantMax: 0.32,
+ },
+ }
+
+ for _, tc := range tests {
+ t.Run(tc.name, func(t *testing.T) {
+ t.Parallel()
+ score := memory.ComputeStalenessScore(tc.entry)
+ require.GreaterOrEqual(t, score, tc.wantMin)
+ require.LessOrEqual(t, score, tc.wantMax)
+ })
+ }
+}
diff --git a/pkg/memory/service.go b/pkg/memory/service.go
new file mode 100644
index 0000000000..7db0e45f82
--- /dev/null
+++ b/pkg/memory/service.go
@@ -0,0 +1,205 @@
+// SPDX-FileCopyrightText: Copyright 2025 Stacklok, Inc.
+// SPDX-License-Identifier: Apache-2.0
+
+package memory
+
+import (
+ "context"
+ "fmt"
+ "sort"
+ "time"
+
+ "github.com/google/uuid"
+ "go.uber.org/zap"
+)
+
+const (
+ conflictSimilarityThreshold = float32(0.85)
+ defaultConflictTopK = 5
+ // stalenessSearchPenaltyWeight controls how much staleness reduces ranking score.
+ stalenessSearchPenaltyWeight = float32(0.3)
+)
+
+// Service orchestrates Store, VectorStore, and Embedder to provide
+// the full memory lifecycle including conflict detection and scoring.
+type Service struct {
+ store Store
+ vectors VectorStore
+ embedder Embedder
+ log *zap.Logger
+}
+
+// NewService constructs a Service. All dependencies are required.
+//
+// The Store provides durable persistence for memory entries.
+// The VectorStore enables semantic similarity search over entry embeddings.
+// The Embedder converts text to vectors; the caller is responsible for
+// ensuring the same Embedder is used consistently — switching embedders
+// will invalidate stored vectors.
+func NewService(store Store, vectors VectorStore, embedder Embedder, log *zap.Logger) (*Service, error) {
+ if store == nil {
+ return nil, fmt.Errorf("store is required")
+ }
+ if vectors == nil {
+ return nil, fmt.Errorf("vector store is required")
+ }
+ if embedder == nil {
+ return nil, fmt.Errorf("embedder is required")
+ }
+ if log == nil {
+ return nil, fmt.Errorf("logger is required")
+ }
+ return &Service{store: store, vectors: vectors, embedder: embedder, log: log}, nil
+}
+
+// RememberInput is the input to Service.Remember.
+type RememberInput struct {
+ Content string
+ Type Type
+ Tags []string
+ Author AuthorType
+ AgentID string
+ SessionID string
+ Source SourceType
+ SkillRef string
+ TTLDays *int
+ // Force bypasses conflict detection and writes unconditionally.
+ Force bool
+}
+
+// RememberResult is returned by Service.Remember.
+// If Conflicts is non-empty, MemoryID is empty and the write was not performed.
+type RememberResult struct {
+ MemoryID string
+ Conflicts []ConflictResult
+}
+
+// Remember embeds content, checks for conflicts, and writes the entry if none found.
+// When Force is true the conflict check is skipped entirely.
+func (s *Service) Remember(ctx context.Context, in RememberInput) (*RememberResult, error) {
+ embedding, err := s.embedder.Embed(ctx, in.Content)
+ if err != nil {
+ return nil, fmt.Errorf("embedding content: %w", err)
+ }
+
+ if !in.Force {
+ conflicts, err := s.detectConflicts(ctx, embedding, in.Type)
+ if err != nil {
+ return nil, fmt.Errorf("detecting conflicts: %w", err)
+ }
+ if len(conflicts) > 0 {
+ return &RememberResult{Conflicts: conflicts}, nil
+ }
+ }
+
+ id := "mem_" + uuid.New().String()
+ now := time.Now().UTC()
+ entry := Entry{
+ ID: id,
+ Type: in.Type,
+ Content: in.Content,
+ Tags: in.Tags,
+ Author: in.Author,
+ AgentID: in.AgentID,
+ SessionID: in.SessionID,
+ Source: sourceOrDefault(in.Source),
+ SkillRef: in.SkillRef,
+ Status: EntryStatusActive,
+ TTLDays: in.TTLDays,
+ CreatedAt: now,
+ UpdatedAt: now,
+ }
+ if in.TTLDays != nil {
+ t := now.AddDate(0, 0, *in.TTLDays)
+ entry.ExpiresAt = &t
+ }
+ entry.TrustScore = ComputeTrustScore(entry)
+ entry.StalenessScore = ComputeStalenessScore(entry)
+
+ if err := s.store.Create(ctx, entry); err != nil {
+ return nil, fmt.Errorf("creating entry: %w", err)
+ }
+ if err := s.vectors.Upsert(ctx, id, embedding); err != nil {
+ // Best-effort rollback: remove the orphaned store entry.
+ _ = s.store.Delete(ctx, id)
+ return nil, fmt.Errorf("upserting vector: %w", err)
+ }
+
+ return &RememberResult{MemoryID: id}, nil
+}
+
+// Search embeds the query, searches the vector store, fetches entries, and
+// increments access counts.
+func (s *Service) Search(ctx context.Context, query string, memType *Type, topK int) ([]ScoredEntry, error) {
+ if topK <= 0 {
+ topK = 10
+ }
+ embedding, err := s.embedder.Embed(ctx, query)
+ if err != nil {
+ return nil, fmt.Errorf("embedding query: %w", err)
+ }
+
+ active := EntryStatusActive
+ ids, err := s.vectors.Search(ctx, embedding, topK, VectorFilter{Type: memType, Status: &active})
+ if err != nil {
+ return nil, fmt.Errorf("vector search: %w", err)
+ }
+
+ var results []ScoredEntry
+ for _, scored := range ids {
+ entry, err := s.store.Get(ctx, scored.ID)
+ if err != nil {
+ s.log.Warn("skipping missing entry", zap.String("id", scored.ID), zap.Error(err))
+ continue
+ }
+ // Increment access count; failure is non-fatal.
+ _ = s.store.IncrementAccess(ctx, scored.ID)
+ // Composite score: boost by trust, penalise by staleness.
+ composite := scored.Similarity * entry.TrustScore * (1 - stalenessSearchPenaltyWeight*entry.StalenessScore)
+ results = append(results, ScoredEntry{Entry: entry, Similarity: composite})
+ }
+ sort.Slice(results, func(i, j int) bool {
+ return results[i].Similarity > results[j].Similarity
+ })
+ return results, nil
+}
+
+// detectConflicts returns any existing entries whose embedding similarity to
+// the candidate exceeds conflictSimilarityThreshold.
+func (s *Service) detectConflicts(ctx context.Context, embedding []float32, memType Type) ([]ConflictResult, error) {
+ active := EntryStatusActive
+ candidates, err := s.vectors.Search(ctx, embedding, defaultConflictTopK, VectorFilter{
+ Type: &memType,
+ Status: &active,
+ })
+ if err != nil {
+ return nil, err
+ }
+
+ var conflicts []ConflictResult
+ for _, c := range candidates {
+ if c.Similarity < conflictSimilarityThreshold {
+ continue
+ }
+ entry, err := s.store.Get(ctx, c.ID)
+ if err != nil {
+ // Skip entries that can't be fetched; they may have been deleted concurrently.
+ s.log.Warn("skipping conflict candidate", zap.String("id", c.ID), zap.Error(err))
+ continue
+ }
+ conflicts = append(conflicts, ConflictResult{
+ ID: entry.ID,
+ Content: entry.Content,
+ Similarity: c.Similarity,
+ TrustScore: entry.TrustScore,
+ })
+ }
+ return conflicts, nil
+}
+
+func sourceOrDefault(s SourceType) SourceType {
+ if s == "" {
+ return SourceMemory
+ }
+ return s
+}
diff --git a/pkg/memory/service_test.go b/pkg/memory/service_test.go
new file mode 100644
index 0000000000..8e0d721acc
--- /dev/null
+++ b/pkg/memory/service_test.go
@@ -0,0 +1,153 @@
+// SPDX-FileCopyrightText: Copyright 2025 Stacklok, Inc.
+// SPDX-License-Identifier: Apache-2.0
+
+package memory_test
+
+import (
+ "context"
+ "testing"
+ "time"
+
+ "github.com/stretchr/testify/require"
+ "go.uber.org/mock/gomock"
+ "go.uber.org/zap/zaptest"
+
+ "github.com/stacklok/toolhive/pkg/memory"
+ "github.com/stacklok/toolhive/pkg/memory/mocks"
+)
+
+func TestService_Remember_NoConflict(t *testing.T) {
+ t.Parallel()
+ ctrl := gomock.NewController(t)
+ store := mocks.NewMockStore(ctrl)
+ vectors := mocks.NewMockVectorStore(ctrl)
+ embedder := mocks.NewMockEmbedder(ctrl)
+
+ emb := []float32{1, 0, 0}
+ embedder.EXPECT().Embed(gomock.Any(), "test fact").Return(emb, nil)
+ active := memory.EntryStatusActive
+ vectors.EXPECT().Search(gomock.Any(), emb, 5, memory.VectorFilter{
+ Type: ptrOf(memory.TypeSemantic),
+ Status: &active,
+ }).Return(nil, nil)
+ store.EXPECT().Create(gomock.Any(), gomock.Any()).Return(nil)
+ vectors.EXPECT().Upsert(gomock.Any(), gomock.Any(), emb).Return(nil)
+
+ svc, err := memory.NewService(store, vectors, embedder, zaptest.NewLogger(t))
+ require.NoError(t, err)
+
+ result, err := svc.Remember(context.Background(), memory.RememberInput{
+ Content: "test fact",
+ Type: memory.TypeSemantic,
+ Author: memory.AuthorHuman,
+ })
+ require.NoError(t, err)
+ require.NotEmpty(t, result.MemoryID)
+ require.Empty(t, result.Conflicts)
+}
+
+func TestService_Remember_ConflictDetected(t *testing.T) {
+ t.Parallel()
+ ctrl := gomock.NewController(t)
+ store := mocks.NewMockStore(ctrl)
+ vectors := mocks.NewMockVectorStore(ctrl)
+ embedder := mocks.NewMockEmbedder(ctrl)
+
+ emb := []float32{1, 0, 0}
+ embedder.EXPECT().Embed(gomock.Any(), "conflicting fact").Return(emb, nil)
+ active := memory.EntryStatusActive
+ vectors.EXPECT().Search(gomock.Any(), emb, 5, memory.VectorFilter{
+ Type: ptrOf(memory.TypeSemantic),
+ Status: &active,
+ }).Return([]memory.ScoredID{{ID: "mem_existing", Similarity: 0.92}}, nil)
+ store.EXPECT().Get(gomock.Any(), "mem_existing").Return(memory.Entry{
+ ID: "mem_existing",
+ Content: "existing fact",
+ }, nil)
+
+ svc, err := memory.NewService(store, vectors, embedder, zaptest.NewLogger(t))
+ require.NoError(t, err)
+
+ result, err := svc.Remember(context.Background(), memory.RememberInput{
+ Content: "conflicting fact",
+ Type: memory.TypeSemantic,
+ Author: memory.AuthorAgent,
+ })
+ require.NoError(t, err)
+ require.Empty(t, result.MemoryID)
+ require.Len(t, result.Conflicts, 1)
+ require.Equal(t, "mem_existing", result.Conflicts[0].ID)
+}
+
+func TestService_Remember_Force(t *testing.T) {
+ t.Parallel()
+ ctrl := gomock.NewController(t)
+ store := mocks.NewMockStore(ctrl)
+ vectors := mocks.NewMockVectorStore(ctrl)
+ embedder := mocks.NewMockEmbedder(ctrl)
+
+ emb := []float32{1, 0, 0}
+ embedder.EXPECT().Embed(gomock.Any(), "forced fact").Return(emb, nil)
+ store.EXPECT().Create(gomock.Any(), gomock.Any()).Return(nil)
+ vectors.EXPECT().Upsert(gomock.Any(), gomock.Any(), emb).Return(nil)
+
+ svc, err := memory.NewService(store, vectors, embedder, zaptest.NewLogger(t))
+ require.NoError(t, err)
+
+ result, err := svc.Remember(context.Background(), memory.RememberInput{
+ Content: "forced fact",
+ Type: memory.TypeSemantic,
+ Author: memory.AuthorHuman,
+ Force: true,
+ })
+ require.NoError(t, err)
+ require.NotEmpty(t, result.MemoryID)
+}
+
+func TestService_Search_CompositeScoring(t *testing.T) {
+ t.Parallel()
+ ctrl := gomock.NewController(t)
+ store := mocks.NewMockStore(ctrl)
+ vectors := mocks.NewMockVectorStore(ctrl)
+ embedder := mocks.NewMockEmbedder(ctrl)
+
+ emb := []float32{1, 0, 0}
+ embedder.EXPECT().Embed(gomock.Any(), "auth endpoint").Return(emb, nil)
+
+ active := memory.EntryStatusActive
+ // Two results: high raw similarity but stale/flagged vs lower similarity but fresh+trusted.
+ vectors.EXPECT().Search(gomock.Any(), emb, 10, memory.VectorFilter{Status: &active}).
+ Return([]memory.ScoredID{
+ {ID: "stale_high", Similarity: 0.95},
+ {ID: "fresh_low", Similarity: 0.80},
+ }, nil)
+
+ now := time.Now()
+ flagTime := now.Add(-24 * time.Hour)
+
+ store.EXPECT().Get(gomock.Any(), "stale_high").Return(memory.Entry{
+ ID: "stale_high", Author: memory.AuthorAgent,
+ TrustScore: 0.5, StalenessScore: 0.8, CreatedAt: now, FlaggedAt: &flagTime,
+ }, nil)
+ store.EXPECT().IncrementAccess(gomock.Any(), "stale_high").Return(nil)
+
+ store.EXPECT().Get(gomock.Any(), "fresh_low").Return(memory.Entry{
+ ID: "fresh_low", Author: memory.AuthorHuman,
+ TrustScore: 1.0, StalenessScore: 0.0, CreatedAt: now,
+ }, nil)
+ store.EXPECT().IncrementAccess(gomock.Any(), "fresh_low").Return(nil)
+
+ svc, err := memory.NewService(store, vectors, embedder, zaptest.NewLogger(t))
+ require.NoError(t, err)
+
+ results, err := svc.Search(context.Background(), "auth endpoint", nil, 0)
+ require.NoError(t, err)
+ require.Len(t, results, 2)
+
+ // fresh_low (composite ≈ 0.80) should rank above stale_high (0.95 × 0.5 × (1-0.3×0.8) ≈ 0.361)
+ require.Equal(t, "fresh_low", results[0].Entry.ID)
+ require.Equal(t, "stale_high", results[1].Entry.ID)
+ require.Greater(t, results[0].Similarity, results[1].Similarity)
+}
+
+func ptrOf[T any](v T) *T { return &v }
diff --git a/pkg/memory/sqlite/db.go b/pkg/memory/sqlite/db.go
new file mode 100644
index 0000000000..50a56d0ece
--- /dev/null
+++ b/pkg/memory/sqlite/db.go
@@ -0,0 +1,106 @@
+// SPDX-FileCopyrightText: Copyright 2025 Stacklok, Inc.
+// SPDX-License-Identifier: Apache-2.0
+
+// Package sqlite provides SQLite-backed implementations of the memory.Store
+// and memory.VectorStore interfaces.
+package sqlite
+
+import (
+ "context"
+ "database/sql"
+ "embed"
+ "errors"
+ "fmt"
+ "io/fs"
+ "os"
+ "path/filepath"
+
+ "github.com/pressly/goose/v3"
+ _ "modernc.org/sqlite" // SQLite driver
+)
+
+//go:embed migrations/*.sql
+var migrations embed.FS
+
+// DB wraps a *sql.DB connection for the memory SQLite database.
+type DB struct {
+ db *sql.DB
+}
+
+// Open opens (or creates) the memory SQLite database at path.
+func Open(ctx context.Context, path string) (_ *DB, err error) {
+ if err := os.MkdirAll(filepath.Dir(path), 0750); err != nil {
+ return nil, fmt.Errorf("creating database directory: %w", err)
+ }
+
+ dsn := fmt.Sprintf("file:%s?_txlock=immediate", path)
+ sqlDB, err := sql.Open("sqlite", dsn)
+ if err != nil {
+ return nil, fmt.Errorf("opening database: %w", err)
+ }
+
+ success := false
+ defer func() {
+ if !success {
+ if closeErr := sqlDB.Close(); closeErr != nil {
+ err = errors.Join(err, fmt.Errorf("closing database after failure: %w", closeErr))
+ }
+ }
+ }()
+
+ sqlDB.SetMaxOpenConns(1)
+ sqlDB.SetMaxIdleConns(1)
+
+ if err = applyPragmas(sqlDB); err != nil {
+ return nil, err
+ }
+
+ if err = runMigrations(ctx, sqlDB); err != nil {
+ return nil, err
+ }
+
+ if err = sqlDB.PingContext(ctx); err != nil {
+ return nil, fmt.Errorf("verifying connection: %w", err)
+ }
+
+ success = true
+ return &DB{db: sqlDB}, nil
+}
+
+// Close closes the underlying database connection.
+func (d *DB) Close() error { return d.db.Close() }
+
+// DB returns the underlying *sql.DB.
+func (d *DB) DB() *sql.DB { return d.db }
+
+func applyPragmas(db *sql.DB) error {
+ for _, p := range []string{
+ "PRAGMA journal_mode=WAL",
+ "PRAGMA busy_timeout=5000",
+ "PRAGMA synchronous=NORMAL",
+ "PRAGMA foreign_keys=ON",
+ "PRAGMA cache_size=-2000",
+ } {
+ if _, err := db.Exec(p); err != nil {
+ return fmt.Errorf("applying pragma %q: %w", p, err)
+ }
+ }
+ return nil
+}
+
+func runMigrations(ctx context.Context, db *sql.DB) error {
+ migrationsFS, err := fs.Sub(migrations, "migrations")
+ if err != nil {
+ return fmt.Errorf("creating migrations sub-filesystem: %w", err)
+ }
+ provider, err := goose.NewProvider(goose.DialectSQLite3, db, migrationsFS,
+ goose.WithAllowOutofOrder(false),
+ )
+ if err != nil {
+ return fmt.Errorf("creating goose provider: %w", err)
+ }
+ if _, err := provider.Up(ctx); err != nil {
+ return fmt.Errorf("running migrations: %w", err)
+ }
+ return nil
+}
diff --git a/pkg/memory/sqlite/migrations/001_initial.sql b/pkg/memory/sqlite/migrations/001_initial.sql
new file mode 100644
index 0000000000..717f8061d9
--- /dev/null
+++ b/pkg/memory/sqlite/migrations/001_initial.sql
@@ -0,0 +1,60 @@
+-- +goose Up
+
+CREATE TABLE IF NOT EXISTS memory_entries (
+ id TEXT PRIMARY KEY,
+ type TEXT NOT NULL CHECK (type IN ('semantic','procedural')),
+ content TEXT NOT NULL,
+ tags TEXT NOT NULL DEFAULT '[]', -- JSON array
+ author TEXT NOT NULL CHECK (author IN ('human','agent')),
+ agent_id TEXT NOT NULL DEFAULT '',
+ session_id TEXT NOT NULL DEFAULT '',
+ source TEXT NOT NULL CHECK (source IN ('memory','skill')),
+ skill_ref TEXT NOT NULL DEFAULT '',
+ status TEXT NOT NULL DEFAULT 'active'
+ CHECK (status IN ('active','flagged','expired','archived')),
+ trust_score REAL NOT NULL DEFAULT 0,
+ staleness_score REAL NOT NULL DEFAULT 0,
+ access_count INTEGER NOT NULL DEFAULT 0,
+ last_accessed_at TEXT,
+ flagged_at TEXT,
+ flag_reason TEXT NOT NULL DEFAULT '',
+ ttl_days INTEGER,
+ expires_at TEXT,
+ archived_at TEXT,
+ consolidated_into TEXT NOT NULL DEFAULT '',
+ crystallized_into TEXT NOT NULL DEFAULT '',
+ created_at TEXT NOT NULL,
+ updated_at TEXT NOT NULL
+);
+
+CREATE TABLE IF NOT EXISTS memory_revisions (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ entry_id TEXT NOT NULL REFERENCES memory_entries(id) ON DELETE CASCADE,
+ content TEXT NOT NULL,
+ author TEXT NOT NULL,
+ correction_note TEXT NOT NULL DEFAULT '',
+ created_at TEXT NOT NULL
+);
+
+-- Embeddings stored as a JSON array of float32 values.
+-- Queries load all vectors for a type+status combination and compute
+-- cosine similarity in Go. Switch to an external VectorStore provider
+-- for datasets > 100K entries.
+CREATE TABLE IF NOT EXISTS memory_embeddings (
+ entry_id TEXT PRIMARY KEY REFERENCES memory_entries(id) ON DELETE CASCADE,
+ embedding TEXT NOT NULL -- JSON []float32
+);
+
+CREATE INDEX IF NOT EXISTS idx_memory_entries_type_status
+ ON memory_entries(type, status);
+
+CREATE INDEX IF NOT EXISTS idx_memory_entries_expires_at
+ ON memory_entries(expires_at) WHERE expires_at IS NOT NULL;
+
+-- +goose Down
+
+DROP INDEX IF EXISTS idx_memory_entries_expires_at;
+DROP INDEX IF EXISTS idx_memory_entries_type_status;
+DROP TABLE IF EXISTS memory_embeddings;
+DROP TABLE IF EXISTS memory_revisions;
+DROP TABLE IF EXISTS memory_entries;
diff --git a/pkg/memory/sqlite/migrations/002_add_episodic_type.sql b/pkg/memory/sqlite/migrations/002_add_episodic_type.sql
new file mode 100644
index 0000000000..d614ce3d39
--- /dev/null
+++ b/pkg/memory/sqlite/migrations/002_add_episodic_type.sql
@@ -0,0 +1,83 @@
+-- +goose Up
+
+-- SQLite does not support ALTER COLUMN, so we recreate the table with the
+-- updated CHECK constraint to include the 'episodic' memory type.
+
+CREATE TABLE IF NOT EXISTS memory_entries_new (
+ id TEXT PRIMARY KEY,
+ type TEXT NOT NULL CHECK (type IN ('semantic','procedural','episodic')),
+ content TEXT NOT NULL,
+ tags TEXT NOT NULL DEFAULT '[]',
+ author TEXT NOT NULL CHECK (author IN ('human','agent')),
+ agent_id TEXT NOT NULL DEFAULT '',
+ session_id TEXT NOT NULL DEFAULT '',
+ source TEXT NOT NULL CHECK (source IN ('memory','skill')),
+ skill_ref TEXT NOT NULL DEFAULT '',
+ status TEXT NOT NULL DEFAULT 'active'
+ CHECK (status IN ('active','flagged','expired','archived')),
+ trust_score REAL NOT NULL DEFAULT 0,
+ staleness_score REAL NOT NULL DEFAULT 0,
+ access_count INTEGER NOT NULL DEFAULT 0,
+ last_accessed_at TEXT,
+ flagged_at TEXT,
+ flag_reason TEXT NOT NULL DEFAULT '',
+ ttl_days INTEGER,
+ expires_at TEXT,
+ archived_at TEXT,
+ consolidated_into TEXT NOT NULL DEFAULT '',
+ crystallized_into TEXT NOT NULL DEFAULT '',
+ created_at TEXT NOT NULL,
+ updated_at TEXT NOT NULL
+);
+
+INSERT INTO memory_entries_new SELECT * FROM memory_entries;
+DROP TABLE memory_entries;
+ALTER TABLE memory_entries_new RENAME TO memory_entries;
+
+CREATE INDEX IF NOT EXISTS idx_memory_entries_type_status
+ ON memory_entries(type, status);
+
+CREATE INDEX IF NOT EXISTS idx_memory_entries_expires_at
+ ON memory_entries(expires_at) WHERE expires_at IS NOT NULL;
+
+-- +goose Down
+
+-- Revert: drop episodic rows then recreate the narrower constraint.
+DELETE FROM memory_entries WHERE type = 'episodic';
+
+CREATE TABLE IF NOT EXISTS memory_entries_old (
+ id TEXT PRIMARY KEY,
+ type TEXT NOT NULL CHECK (type IN ('semantic','procedural')),
+ content TEXT NOT NULL,
+ tags TEXT NOT NULL DEFAULT '[]',
+ author TEXT NOT NULL CHECK (author IN ('human','agent')),
+ agent_id TEXT NOT NULL DEFAULT '',
+ session_id TEXT NOT NULL DEFAULT '',
+ source TEXT NOT NULL CHECK (source IN ('memory','skill')),
+ skill_ref TEXT NOT NULL DEFAULT '',
+ status TEXT NOT NULL DEFAULT 'active'
+ CHECK (status IN ('active','flagged','expired','archived')),
+ trust_score REAL NOT NULL DEFAULT 0,
+ staleness_score REAL NOT NULL DEFAULT 0,
+ access_count INTEGER NOT NULL DEFAULT 0,
+ last_accessed_at TEXT,
+ flagged_at TEXT,
+ flag_reason TEXT NOT NULL DEFAULT '',
+ ttl_days INTEGER,
+ expires_at TEXT,
+ archived_at TEXT,
+ consolidated_into TEXT NOT NULL DEFAULT '',
+ crystallized_into TEXT NOT NULL DEFAULT '',
+ created_at TEXT NOT NULL,
+ updated_at TEXT NOT NULL
+);
+
+INSERT INTO memory_entries_old SELECT * FROM memory_entries;
+DROP TABLE memory_entries;
+ALTER TABLE memory_entries_old RENAME TO memory_entries;
+
+CREATE INDEX IF NOT EXISTS idx_memory_entries_type_status
+ ON memory_entries(type, status);
+
+CREATE INDEX IF NOT EXISTS idx_memory_entries_expires_at
+ ON memory_entries(expires_at) WHERE expires_at IS NOT NULL;
diff --git a/pkg/memory/sqlite/migrations/003_add_resource_source.sql b/pkg/memory/sqlite/migrations/003_add_resource_source.sql
new file mode 100644
index 0000000000..d1455edf44
--- /dev/null
+++ b/pkg/memory/sqlite/migrations/003_add_resource_source.sql
@@ -0,0 +1,83 @@
+-- +goose Up
+
+-- SQLite does not support ALTER COLUMN, so we recreate the table with the
+-- updated CHECK constraint to include the 'resource' source type.
+
+CREATE TABLE IF NOT EXISTS memory_entries_new (
+ id TEXT PRIMARY KEY,
+ type TEXT NOT NULL CHECK (type IN ('semantic','procedural','episodic')),
+ content TEXT NOT NULL,
+ tags TEXT NOT NULL DEFAULT '[]',
+ author TEXT NOT NULL CHECK (author IN ('human','agent')),
+ agent_id TEXT NOT NULL DEFAULT '',
+ session_id TEXT NOT NULL DEFAULT '',
+ source TEXT NOT NULL CHECK (source IN ('memory','skill','resource')),
+ skill_ref TEXT NOT NULL DEFAULT '',
+ status TEXT NOT NULL DEFAULT 'active'
+ CHECK (status IN ('active','flagged','expired','archived')),
+ trust_score REAL NOT NULL DEFAULT 0,
+ staleness_score REAL NOT NULL DEFAULT 0,
+ access_count INTEGER NOT NULL DEFAULT 0,
+ last_accessed_at TEXT,
+ flagged_at TEXT,
+ flag_reason TEXT NOT NULL DEFAULT '',
+ ttl_days INTEGER,
+ expires_at TEXT,
+ archived_at TEXT,
+ consolidated_into TEXT NOT NULL DEFAULT '',
+ crystallized_into TEXT NOT NULL DEFAULT '',
+ created_at TEXT NOT NULL,
+ updated_at TEXT NOT NULL
+);
+
+INSERT INTO memory_entries_new SELECT * FROM memory_entries;
+DROP TABLE memory_entries;
+ALTER TABLE memory_entries_new RENAME TO memory_entries;
+
+CREATE INDEX IF NOT EXISTS idx_memory_entries_type_status
+ ON memory_entries(type, status);
+
+CREATE INDEX IF NOT EXISTS idx_memory_entries_expires_at
+ ON memory_entries(expires_at) WHERE expires_at IS NOT NULL;
+
+-- +goose Down
+
+-- Revert: drop resource rows then recreate the narrower constraint.
+DELETE FROM memory_entries WHERE source = 'resource';
+
+CREATE TABLE IF NOT EXISTS memory_entries_old (
+ id TEXT PRIMARY KEY,
+ type TEXT NOT NULL CHECK (type IN ('semantic','procedural','episodic')),
+ content TEXT NOT NULL,
+ tags TEXT NOT NULL DEFAULT '[]',
+ author TEXT NOT NULL CHECK (author IN ('human','agent')),
+ agent_id TEXT NOT NULL DEFAULT '',
+ session_id TEXT NOT NULL DEFAULT '',
+ source TEXT NOT NULL CHECK (source IN ('memory','skill')),
+ skill_ref TEXT NOT NULL DEFAULT '',
+ status TEXT NOT NULL DEFAULT 'active'
+ CHECK (status IN ('active','flagged','expired','archived')),
+ trust_score REAL NOT NULL DEFAULT 0,
+ staleness_score REAL NOT NULL DEFAULT 0,
+ access_count INTEGER NOT NULL DEFAULT 0,
+ last_accessed_at TEXT,
+ flagged_at TEXT,
+ flag_reason TEXT NOT NULL DEFAULT '',
+ ttl_days INTEGER,
+ expires_at TEXT,
+ archived_at TEXT,
+ consolidated_into TEXT NOT NULL DEFAULT '',
+ crystallized_into TEXT NOT NULL DEFAULT '',
+ created_at TEXT NOT NULL,
+ updated_at TEXT NOT NULL
+);
+
+INSERT INTO memory_entries_old SELECT * FROM memory_entries;
+DROP TABLE memory_entries;
+ALTER TABLE memory_entries_old RENAME TO memory_entries;
+
+CREATE INDEX IF NOT EXISTS idx_memory_entries_type_status
+ ON memory_entries(type, status);
+
+CREATE INDEX IF NOT EXISTS idx_memory_entries_expires_at
+ ON memory_entries(expires_at) WHERE expires_at IS NOT NULL;
diff --git a/pkg/memory/sqlite/store.go b/pkg/memory/sqlite/store.go
new file mode 100644
index 0000000000..80fe0c9eb0
--- /dev/null
+++ b/pkg/memory/sqlite/store.go
@@ -0,0 +1,364 @@
+// SPDX-FileCopyrightText: Copyright 2025 Stacklok, Inc.
+// SPDX-License-Identifier: Apache-2.0
+
+package sqlite
+
+import (
+ "context"
+ "database/sql"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "time"
+
+ "github.com/stacklok/toolhive/pkg/memory"
+)
+
+// Store implements memory.Store using SQLite.
+type Store struct {
+ db *sql.DB
+}
+
+// NewStore creates a new SQLite-backed Store.
+func NewStore(wrapper *DB) *Store {
+ return &Store{db: wrapper.DB()}
+}
+
+var _ memory.Store = (*Store)(nil)
+
+// Create inserts a new memory entry.
+func (s *Store) Create(ctx context.Context, e memory.Entry) error {
+ tags, err := json.Marshal(e.Tags)
+ if err != nil {
+ return fmt.Errorf("marshalling tags: %w", err)
+ }
+
+ _, err = s.db.ExecContext(ctx, `
+ INSERT INTO memory_entries
+ (id, type, content, tags, author, agent_id, session_id, source, skill_ref,
+ status, trust_score, staleness_score, access_count, last_accessed_at,
+ ttl_days, expires_at, created_at, updated_at)
+ VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)`,
+ e.ID, string(e.Type), e.Content, string(tags),
+ string(e.Author), e.AgentID, e.SessionID, string(e.Source), e.SkillRef,
+ string(e.Status), e.TrustScore, e.StalenessScore, e.AccessCount,
+ nullableTime(e.LastAccessedAt),
+ e.TTLDays, nullableTimePtr(e.ExpiresAt),
+ e.CreatedAt.UTC().Format(time.RFC3339Nano),
+ e.UpdatedAt.UTC().Format(time.RFC3339Nano),
+ )
+ return err
+}
+
+// Get retrieves a single entry by ID, including its revision history.
+func (s *Store) Get(ctx context.Context, id string) (memory.Entry, error) {
+ row := s.db.QueryRowContext(ctx, `
+ SELECT id, type, content, tags, author, agent_id, session_id, source, skill_ref,
+ status, trust_score, staleness_score, access_count, last_accessed_at,
+ flagged_at, flag_reason, ttl_days, expires_at, archived_at,
+ consolidated_into, crystallized_into, created_at, updated_at
+ FROM memory_entries WHERE id = ?`, id)
+
+ e, err := scanEntry(row)
+ if errors.Is(err, sql.ErrNoRows) {
+ return memory.Entry{}, fmt.Errorf("entry %q: %w", id, memory.ErrNotFound)
+ }
+ if err != nil {
+ return memory.Entry{}, err
+ }
+
+ e.History, err = s.loadHistory(ctx, id)
+ return e, err
+}
+
+// Update replaces content and appends the old content to revisions.
+func (s *Store) Update(ctx context.Context, id, content string, author memory.AuthorType, note string) error {
+ tx, err := s.db.BeginTx(ctx, nil)
+ if err != nil {
+ return err
+ }
+ defer rollback(tx)
+
+ var oldContent string
+ if err := tx.QueryRowContext(ctx, `SELECT content FROM memory_entries WHERE id = ?`, id).Scan(&oldContent); err != nil {
+ if errors.Is(err, sql.ErrNoRows) {
+ return fmt.Errorf("entry %q: %w", id, memory.ErrNotFound)
+ }
+ return err
+ }
+
+ if _, err := tx.ExecContext(ctx,
+ `INSERT INTO memory_revisions (entry_id, content, author, correction_note, created_at)
+ VALUES (?, ?, ?, ?, ?)`,
+ id, oldContent, string(author), note, time.Now().UTC().Format(time.RFC3339Nano),
+ ); err != nil {
+ return err
+ }
+
+ if _, err := tx.ExecContext(ctx,
+ `UPDATE memory_entries SET content = ?, updated_at = ? WHERE id = ?`,
+ content, time.Now().UTC().Format(time.RFC3339Nano), id,
+ ); err != nil {
+ return err
+ }
+
+ return tx.Commit()
+}
+
+// Flag marks an entry as potentially stale.
+func (s *Store) Flag(ctx context.Context, id, reason string) error {
+ now := time.Now().UTC().Format(time.RFC3339Nano)
+ _, err := s.db.ExecContext(ctx,
+ `UPDATE memory_entries SET status='flagged', flagged_at=?, flag_reason=?, updated_at=? WHERE id=?`,
+ now, reason, now, id)
+ return err
+}
+
+// Unflag clears the flag on an entry.
+func (s *Store) Unflag(ctx context.Context, id string) error {
+ now := time.Now().UTC().Format(time.RFC3339Nano)
+ _, err := s.db.ExecContext(ctx,
+ `UPDATE memory_entries SET status='active', flagged_at=NULL, flag_reason='', updated_at=? WHERE id=?`,
+ now, id)
+ return err
+}
+
+// Delete permanently removes an entry.
+func (s *Store) Delete(ctx context.Context, id string) error {
+ _, err := s.db.ExecContext(ctx, `DELETE FROM memory_entries WHERE id=?`, id)
+ return err
+}
+
+// List returns entries matching the filter.
+func (s *Store) List(ctx context.Context, f memory.ListFilter) ([]memory.Entry, error) {
+ query := `SELECT id, type, content, tags, author, agent_id, session_id, source, skill_ref,
+ status, trust_score, staleness_score, access_count, last_accessed_at,
+ flagged_at, flag_reason, ttl_days, expires_at, archived_at,
+ consolidated_into, crystallized_into, created_at, updated_at
+ FROM memory_entries WHERE 1=1`
+ var args []any
+
+ if f.Type != nil {
+ query += " AND type=?"
+ args = append(args, string(*f.Type))
+ }
+ if f.Author != nil {
+ query += " AND author=?"
+ args = append(args, string(*f.Author))
+ }
+ if f.Source != nil {
+ query += " AND source=?"
+ args = append(args, string(*f.Source))
+ }
+ if f.Status != nil {
+ query += " AND status=?"
+ args = append(args, string(*f.Status))
+ }
+ if f.CreatedAfter != nil {
+ query += " AND created_at >= ?"
+ args = append(args, f.CreatedAfter.UTC().Format(time.RFC3339Nano))
+ }
+ if f.CreatedBefore != nil {
+ query += " AND created_at <= ?"
+ args = append(args, f.CreatedBefore.UTC().Format(time.RFC3339Nano))
+ }
+
+ query += " ORDER BY created_at DESC"
+ if f.Limit > 0 {
+ query += " LIMIT ? OFFSET ?"
+ args = append(args, f.Limit, f.Offset)
+ }
+
+ rows, err := s.db.QueryContext(ctx, query, args...)
+ if err != nil {
+ return nil, err
+ }
+ defer func() { _ = rows.Close() }()
+
+ var entries []memory.Entry
+ for rows.Next() {
+ e, err := scanEntry(rows)
+ if err != nil {
+ return nil, err
+ }
+ entries = append(entries, e)
+ }
+ return entries, rows.Err()
+}
+
+// Archive transitions an entry to archived status.
+func (s *Store) Archive(ctx context.Context, id string, reason memory.ArchiveReason, ref string) error {
+ now := time.Now().UTC().Format(time.RFC3339Nano)
+ field := consolidatedField(reason)
+ _, err := s.db.ExecContext(ctx,
+ fmt.Sprintf(`UPDATE memory_entries SET status='archived', archived_at=?, %s=?, updated_at=? WHERE id=?`, field),
+ now, ref, now, id)
+ return err
+}
+
+// IncrementAccess increments the access counter and updates last_accessed_at.
+func (s *Store) IncrementAccess(ctx context.Context, id string) error {
+ now := time.Now().UTC().Format(time.RFC3339Nano)
+ _, err := s.db.ExecContext(ctx,
+ `UPDATE memory_entries SET access_count=access_count+1, last_accessed_at=?, updated_at=? WHERE id=?`,
+ now, now, id)
+ return err
+}
+
+// UpdateScores persists recomputed trust and staleness scores.
+func (s *Store) UpdateScores(ctx context.Context, id string, trust, staleness float32) error {
+ _, err := s.db.ExecContext(ctx,
+ `UPDATE memory_entries SET trust_score=?, staleness_score=? WHERE id=?`,
+ trust, staleness, id)
+ return err
+}
+
+// ListExpired returns active entries whose TTL has elapsed.
+func (s *Store) ListExpired(ctx context.Context) ([]memory.Entry, error) {
+ now := time.Now().UTC().Format(time.RFC3339Nano)
+ rows, err := s.db.QueryContext(ctx,
+ `SELECT id, type, content, tags, author, agent_id, session_id, source, skill_ref,
+ status, trust_score, staleness_score, access_count, last_accessed_at,
+ flagged_at, flag_reason, ttl_days, expires_at, archived_at,
+ consolidated_into, crystallized_into, created_at, updated_at
+ FROM memory_entries
+ WHERE expires_at IS NOT NULL AND expires_at <= ? AND status NOT IN ('expired','archived')`, now)
+ if err != nil {
+ return nil, err
+ }
+ defer func() { _ = rows.Close() }()
+ var entries []memory.Entry
+ for rows.Next() {
+ e, err := scanEntry(rows)
+ if err != nil {
+ return nil, err
+ }
+ entries = append(entries, e)
+ }
+ return entries, rows.Err()
+}
+
+// ListActive returns all active and flagged entries for score recomputation.
+func (s *Store) ListActive(ctx context.Context) ([]memory.Entry, error) {
+ rows, err := s.db.QueryContext(ctx,
+ `SELECT id, type, content, tags, author, agent_id, session_id, source, skill_ref,
+ status, trust_score, staleness_score, access_count, last_accessed_at,
+ flagged_at, flag_reason, ttl_days, expires_at, archived_at,
+ consolidated_into, crystallized_into, created_at, updated_at
+ FROM memory_entries WHERE status IN ('active','flagged')`)
+ if err != nil {
+ return nil, err
+ }
+ defer func() { _ = rows.Close() }()
+ var entries []memory.Entry
+ for rows.Next() {
+ e, err := scanEntry(rows)
+ if err != nil {
+ return nil, err
+ }
+ entries = append(entries, e)
+ }
+ return entries, rows.Err()
+}
+
+// ---- helpers ----
+
+type scanner interface {
+ Scan(dest ...any) error
+}
+
+func scanEntry(sc scanner) (memory.Entry, error) {
+ var e memory.Entry
+ var (
+ mtype, author, source, status string
+ tagsJSON string
+ lastAccessed, flaggedAt sql.NullString
+ expiresAt, archivedAt sql.NullString
+ createdAt, updatedAt string
+ )
+ err := sc.Scan(
+ &e.ID, &mtype, &e.Content, &tagsJSON, &author,
+ &e.AgentID, &e.SessionID, &source, &e.SkillRef,
+ &status, &e.TrustScore, &e.StalenessScore, &e.AccessCount, &lastAccessed,
+ &flaggedAt, &e.FlagReason, &e.TTLDays, &expiresAt, &archivedAt,
+ &e.ConsolidatedInto, &e.CrystallizedInto, &createdAt, &updatedAt,
+ )
+ if err != nil {
+ return memory.Entry{}, err
+ }
+ e.Type = memory.Type(mtype)
+ e.Author = memory.AuthorType(author)
+ e.Source = memory.SourceType(source)
+ e.Status = memory.EntryStatus(status)
+ _ = json.Unmarshal([]byte(tagsJSON), &e.Tags)
+ e.CreatedAt, _ = parseTime(createdAt)
+ e.UpdatedAt, _ = parseTime(updatedAt)
+ if lastAccessed.Valid {
+ t, _ := parseTime(lastAccessed.String)
+ e.LastAccessedAt = t
+ }
+ if flaggedAt.Valid {
+ t, _ := parseTime(flaggedAt.String)
+ e.FlaggedAt = &t
+ }
+ if expiresAt.Valid {
+ t, _ := parseTime(expiresAt.String)
+ e.ExpiresAt = &t
+ }
+ if archivedAt.Valid {
+ t, _ := parseTime(archivedAt.String)
+ e.ArchivedAt = &t
+ }
+ return e, nil
+}
+
+func (s *Store) loadHistory(ctx context.Context, entryID string) ([]memory.Revision, error) {
+ rows, err := s.db.QueryContext(ctx,
+ `SELECT content, author, correction_note, created_at
+ FROM memory_revisions WHERE entry_id=? ORDER BY created_at ASC`, entryID)
+ if err != nil {
+ return nil, err
+ }
+ defer func() { _ = rows.Close() }()
+ var revs []memory.Revision
+ for rows.Next() {
+ var r memory.Revision
+ var author, createdAt string
+ if err := rows.Scan(&r.Content, &author, &r.CorrectionNote, &createdAt); err != nil {
+ return nil, err
+ }
+ r.Author = memory.AuthorType(author)
+ r.Timestamp, _ = parseTime(createdAt)
+ revs = append(revs, r)
+ }
+ return revs, rows.Err()
+}
+
+func nullableTime(t time.Time) any {
+ if t.IsZero() {
+ return nil
+ }
+ return t.UTC().Format(time.RFC3339Nano)
+}
+
+func nullableTimePtr(t *time.Time) any {
+ if t == nil {
+ return nil
+ }
+ return t.UTC().Format(time.RFC3339Nano)
+}
+
+func parseTime(s string) (time.Time, error) {
+ return time.Parse(time.RFC3339Nano, s)
+}
+
+func consolidatedField(reason memory.ArchiveReason) string {
+ if reason == memory.ArchiveReasonCrystallized {
+ return "crystallized_into"
+ }
+ return "consolidated_into"
+}
+
+func rollback(tx *sql.Tx) {
+ _ = tx.Rollback()
+}
diff --git a/pkg/memory/sqlite/store_test.go b/pkg/memory/sqlite/store_test.go
new file mode 100644
index 0000000000..8b009d12b6
--- /dev/null
+++ b/pkg/memory/sqlite/store_test.go
@@ -0,0 +1,167 @@
+// SPDX-FileCopyrightText: Copyright 2025 Stacklok, Inc.
+// SPDX-License-Identifier: Apache-2.0
+
+package sqlite_test
+
+import (
+ "context"
+ "fmt"
+ "path/filepath"
+ "testing"
+ "time"
+
+ "github.com/stretchr/testify/require"
+
+ "github.com/stacklok/toolhive/pkg/memory"
+ memorysqlite "github.com/stacklok/toolhive/pkg/memory/sqlite"
+)
+
+func openTestDB(t *testing.T) *memorysqlite.DB {
+ t.Helper()
+ dir := t.TempDir()
+ resolved, _ := filepath.EvalSymlinks(dir)
+ db, err := memorysqlite.Open(context.Background(), filepath.Join(resolved, "memory.db"))
+ require.NoError(t, err)
+ t.Cleanup(func() { _ = db.Close() })
+ return db
+}
+
+func TestMemoryStore_CreateAndGet(t *testing.T) {
+ t.Parallel()
+ db := openTestDB(t)
+ store := memorysqlite.NewStore(db)
+
+ entry := memory.Entry{
+ ID: "mem_test_001",
+ Type: memory.TypeSemantic,
+ Content: "we deploy to us-east-1",
+ Tags: []string{"deployment", "infra"},
+ Author: memory.AuthorHuman,
+ Source: memory.SourceMemory,
+ Status: memory.EntryStatusActive,
+ CreatedAt: time.Now(),
+ UpdatedAt: time.Now(),
+ }
+
+ err := store.Create(context.Background(), entry)
+ require.NoError(t, err)
+
+ got, err := store.Get(context.Background(), "mem_test_001")
+ require.NoError(t, err)
+ require.Equal(t, entry.ID, got.ID)
+ require.Equal(t, entry.Content, got.Content)
+ require.Equal(t, entry.Tags, got.Tags)
+ require.Equal(t, entry.Author, got.Author)
+ require.Equal(t, entry.Status, got.Status)
+}
+
+func TestMemoryStore_Update(t *testing.T) {
+ t.Parallel()
+ db := openTestDB(t)
+ store := memorysqlite.NewStore(db)
+
+ entry := memory.Entry{
+ ID: "mem_test_002",
+ Type: memory.TypeSemantic,
+ Content: "old content",
+ Author: memory.AuthorHuman,
+ Source: memory.SourceMemory,
+ Status: memory.EntryStatusActive,
+ CreatedAt: time.Now(),
+ UpdatedAt: time.Now(),
+ }
+ require.NoError(t, store.Create(context.Background(), entry))
+
+ err := store.Update(context.Background(), "mem_test_002", "new content", memory.AuthorHuman, "corrected")
+ require.NoError(t, err)
+
+ got, err := store.Get(context.Background(), "mem_test_002")
+ require.NoError(t, err)
+ require.Equal(t, "new content", got.Content)
+ require.Len(t, got.History, 1)
+ require.Equal(t, "old content", got.History[0].Content)
+ require.Equal(t, "corrected", got.History[0].CorrectionNote)
+}
+
+func TestMemoryStore_Archive(t *testing.T) {
+ t.Parallel()
+ db := openTestDB(t)
+ store := memorysqlite.NewStore(db)
+
+ entry := memory.Entry{
+ ID: "mem_test_003",
+ Type: memory.TypeProcedural,
+ Content: "check Docker health before E2E tests",
+ Author: memory.AuthorAgent,
+ Source: memory.SourceMemory,
+ Status: memory.EntryStatusActive,
+ CreatedAt: time.Now(),
+ UpdatedAt: time.Now(),
+ }
+ require.NoError(t, store.Create(context.Background(), entry))
+
+ err := store.Archive(context.Background(), "mem_test_003", memory.ArchiveReasonConsolidated, "mem_test_consolidated")
+ require.NoError(t, err)
+
+ got, err := store.Get(context.Background(), "mem_test_003")
+ require.NoError(t, err)
+ require.Equal(t, memory.EntryStatusArchived, got.Status)
+ require.Equal(t, "mem_test_consolidated", got.ConsolidatedInto)
+ require.NotNil(t, got.ArchivedAt)
+}
+
+func TestMemoryStore_List(t *testing.T) {
+ t.Parallel()
+ db := openTestDB(t)
+ store := memorysqlite.NewStore(db)
+
+ ctx := context.Background()
+ for i, content := range []string{"fact A", "fact B", "procedure X"} {
+ mtype := memory.TypeSemantic
+ if i == 2 {
+ mtype = memory.TypeProcedural
+ }
+ require.NoError(t, store.Create(ctx, memory.Entry{
+ ID: fmt.Sprintf("mem_list_%d", i),
+ Type: mtype,
+ Content: content,
+ Author: memory.AuthorHuman,
+ Source: memory.SourceMemory,
+ Status: memory.EntryStatusActive,
+ CreatedAt: time.Now(),
+ UpdatedAt: time.Now(),
+ }))
+ }
+
+ sem := memory.TypeSemantic
+ results, err := store.List(ctx, memory.ListFilter{Type: &sem, Limit: 10})
+ require.NoError(t, err)
+ require.Len(t, results, 2)
+}
+
+func TestMemoryStore_ListTimeRange(t *testing.T) {
+ t.Parallel()
+ db := openTestDB(t)
+ store := memorysqlite.NewStore(db)
+ ctx := context.Background()
+
+ past := time.Now().Add(-2 * time.Hour)
+ recent := time.Now().Add(-30 * time.Minute)
+
+ require.NoError(t, store.Create(ctx, memory.Entry{
+ ID: "mem_old", Type: memory.TypeEpisodic, Content: "old event",
+ Author: memory.AuthorAgent, Source: memory.SourceMemory, Status: memory.EntryStatusActive,
+ CreatedAt: past, UpdatedAt: past,
+ }))
+ require.NoError(t, store.Create(ctx, memory.Entry{
+ ID: "mem_new", Type: memory.TypeEpisodic, Content: "recent event",
+ Author: memory.AuthorAgent, Source: memory.SourceMemory, Status: memory.EntryStatusActive,
+ CreatedAt: recent, UpdatedAt: recent,
+ }))
+
+ cutoff := time.Now().Add(-1 * time.Hour)
+ results, err := store.List(ctx, memory.ListFilter{CreatedAfter: &cutoff})
+ require.NoError(t, err)
+ require.Len(t, results, 1)
+ require.Equal(t, "mem_new", results[0].ID)
+}
diff --git a/pkg/memory/sqlite/vector.go b/pkg/memory/sqlite/vector.go
new file mode 100644
index 0000000000..a0e734833b
--- /dev/null
+++ b/pkg/memory/sqlite/vector.go
@@ -0,0 +1,131 @@
+// SPDX-FileCopyrightText: Copyright 2025 Stacklok, Inc.
+// SPDX-License-Identifier: Apache-2.0
+
+package sqlite
+
+import (
+ "context"
+ "database/sql"
+ "encoding/json"
+ "fmt"
+ "math"
+ "sort"
+
+ "github.com/stacklok/toolhive/pkg/memory"
+)
+
+// VectorStore implements memory.VectorStore using SQLite blob storage and
+// Go-native cosine similarity. Suitable for datasets up to ~100K entries.
+// Use an external VectorStore (Qdrant, pgvector) for larger datasets.
+type VectorStore struct {
+ db *sql.DB
+}
+
+// NewVectorStore creates a new SQLite-backed VectorStore.
+func NewVectorStore(wrapper *DB) *VectorStore {
+ return &VectorStore{db: wrapper.DB()}
+}
+
+var _ memory.VectorStore = (*VectorStore)(nil)
+
+// Upsert stores or replaces the embedding for entry id.
+func (v *VectorStore) Upsert(ctx context.Context, id string, embedding []float32) error {
+ data, err := json.Marshal(embedding)
+ if err != nil {
+ return fmt.Errorf("marshalling embedding: %w", err)
+ }
+ _, err = v.db.ExecContext(ctx,
+ `INSERT INTO memory_embeddings (entry_id, embedding) VALUES (?,?)
+ ON CONFLICT(entry_id) DO UPDATE SET embedding=excluded.embedding`,
+ id, string(data))
+ return err
+}
+
+// Search loads all embeddings matching the filter, computes cosine similarity
+// against query, and returns the topK results in descending score order.
+func (v *VectorStore) Search(
+ ctx context.Context, query []float32, topK int, filter memory.VectorFilter,
+) ([]memory.ScoredID, error) {
+ q := `SELECT e.entry_id, e.embedding
+ FROM memory_embeddings e
+ JOIN memory_entries m ON m.id = e.entry_id
+ WHERE 1=1`
+ var args []any
+ if filter.Type != nil {
+ q += " AND m.type=?"
+ args = append(args, string(*filter.Type))
+ }
+ if filter.Status != nil {
+ q += " AND m.status=?"
+ args = append(args, string(*filter.Status))
+ }
+ if filter.Source != nil {
+ q += " AND m.source=?"
+ args = append(args, string(*filter.Source))
+ }
+
+ rows, err := v.db.QueryContext(ctx, q, args...)
+ if err != nil {
+ return nil, err
+ }
+ defer func() { _ = rows.Close() }()
+
+ qNorm := l2Norm(query)
+ if qNorm == 0 {
+ return nil, fmt.Errorf("query vector has zero magnitude")
+ }
+
+ var scored []memory.ScoredID
+ for rows.Next() {
+ var id, embJSON string
+ if err := rows.Scan(&id, &embJSON); err != nil {
+ return nil, err
+ }
+ var emb []float32
+ if err := json.Unmarshal([]byte(embJSON), &emb); err != nil {
+ continue
+ }
+ sim := cosineSimilarity(query, emb, qNorm)
+ scored = append(scored, memory.ScoredID{ID: id, Similarity: sim})
+ }
+ if err := rows.Err(); err != nil {
+ return nil, err
+ }
+
+ sort.Slice(scored, func(i, j int) bool {
+ return scored[i].Similarity > scored[j].Similarity
+ })
+ if topK > 0 && len(scored) > topK {
+ scored = scored[:topK]
+ }
+ return scored, nil
+}
+
+// Delete removes the embedding for entry id.
+func (v *VectorStore) Delete(ctx context.Context, id string) error {
+ _, err := v.db.ExecContext(ctx, `DELETE FROM memory_embeddings WHERE entry_id=?`, id)
+ return err
+}
+
+func cosineSimilarity(a, b []float32, aNorm float32) float32 {
+ if len(a) != len(b) || aNorm == 0 {
+ return 0
+ }
+ bNorm := l2Norm(b)
+ if bNorm == 0 {
+ return 0
+ }
+ var dot float64
+ for i := range a {
+ dot += float64(a[i]) * float64(b[i])
+ }
+ return float32(dot / (float64(aNorm) * float64(bNorm)))
+}
+
+func l2Norm(v []float32) float32 {
+ var sum float64
+ for _, x := range v {
+ sum += float64(x) * float64(x)
+ }
+ return float32(math.Sqrt(sum))
+}
diff --git a/pkg/memory/sqlite/vector_test.go b/pkg/memory/sqlite/vector_test.go
new file mode 100644
index 0000000000..b613db0edc
--- /dev/null
+++ b/pkg/memory/sqlite/vector_test.go
@@ -0,0 +1,74 @@
+// SPDX-FileCopyrightText: Copyright 2025 Stacklok, Inc.
+// SPDX-License-Identifier: Apache-2.0
+
+package sqlite_test
+
+import (
+ "context"
+ "testing"
+
+ "github.com/stretchr/testify/require"
+
+ "github.com/stacklok/toolhive/pkg/memory"
+ memorysqlite "github.com/stacklok/toolhive/pkg/memory/sqlite"
+)
+
+func TestVectorStore_UpsertAndSearch(t *testing.T) {
+ t.Parallel()
+ db := openTestDB(t)
+ store := memorysqlite.NewStore(db)
+ vectors := memorysqlite.NewVectorStore(db)
+
+ ctx := context.Background()
+
+ entries := []struct {
+ id string
+ embedding []float32
+ }{
+ {"vec_001", []float32{1, 0, 0}},
+ {"vec_002", []float32{0.9, 0.1, 0}},
+ {"vec_003", []float32{0, 0, 1}},
+ }
+ for _, e := range entries {
+ require.NoError(t, store.Create(ctx, memory.Entry{
+ ID: e.id, Type: memory.TypeSemantic, Content: "c",
+ Author: memory.AuthorAgent, Source: memory.SourceMemory,
+ Status: memory.EntryStatusActive,
+ }))
+ require.NoError(t, vectors.Upsert(ctx, e.id, e.embedding))
+ }
+
+ query := []float32{0.95, 0.05, 0}
+ results, err := vectors.Search(ctx, query, 2, memory.VectorFilter{})
+ require.NoError(t, err)
+ require.Len(t, results, 2)
+
+ ids := []string{results[0].ID, results[1].ID}
+ require.Contains(t, ids, "vec_001")
+ require.Contains(t, ids, "vec_002")
+ require.NotContains(t, ids, "vec_003")
+
+ require.GreaterOrEqual(t, results[0].Similarity, results[1].Similarity)
+}
+
+func TestVectorStore_Delete(t *testing.T) {
+ t.Parallel()
+ db := openTestDB(t)
+ store := memorysqlite.NewStore(db)
+ vectors := memorysqlite.NewVectorStore(db)
+
+ ctx := context.Background()
+ require.NoError(t, store.Create(ctx, memory.Entry{
+ ID: "vec_del", Type: memory.TypeSemantic, Content: "c",
+ Author: memory.AuthorAgent, Source: memory.SourceMemory,
+ Status: memory.EntryStatusActive,
+ }))
+ require.NoError(t, vectors.Upsert(ctx, "vec_del", []float32{1, 0, 0}))
+ require.NoError(t, vectors.Delete(ctx, "vec_del"))
+
+ results, err := vectors.Search(ctx, []float32{1, 0, 0}, 5, memory.VectorFilter{})
+ require.NoError(t, err)
+ for _, r := range results {
+ require.NotEqual(t, "vec_del", r.ID)
+ }
+}
diff --git a/pkg/memory/types.go b/pkg/memory/types.go
new file mode 100644
index 0000000000..7369656d3b
--- /dev/null
+++ b/pkg/memory/types.go
@@ -0,0 +1,153 @@
+// SPDX-FileCopyrightText: Copyright 2025 Stacklok, Inc.
+// SPDX-License-Identifier: Apache-2.0
+
+// Package memory defines the types and interfaces for ToolHive's shared long-term memory system.
+package memory
+
+import "time"
+
+// Type distinguishes the two long-term memory namespaces.
+type Type string
+
+const (
+ // TypeSemantic represents factual, aggregated knowledge and world-state memories
+ // (e.g. "company does not sponsor visas"). Contrast with TypeEpisodic.
+ TypeSemantic Type = "semantic"
+ // TypeProcedural represents how-to knowledge and step-based memories.
+ TypeProcedural Type = "procedural"
+ // TypeEpisodic represents time-indexed event records tied to a specific
+ // moment (e.g. "recruiter archived candidate on 2024-03-15 — visa required").
+ // Use CreatedAfter/CreatedBefore in ListFilter to query timelines.
+ TypeEpisodic Type = "episodic"
+)
+
+// AuthorType records whether a memory was written by a human or an agent.
+type AuthorType string
+
+const (
+ // AuthorHuman indicates the memory was written by a human user.
+ AuthorHuman AuthorType = "human"
+ // AuthorAgent indicates the memory was written by an AI agent.
+ AuthorAgent AuthorType = "agent"
+)
+
+// SourceType records whether a memory entry originates from the store or is a
+// read-only index of an installed Skill.
+type SourceType string
+
+const (
+ // SourceMemory indicates the entry originates from the writable memory store.
+ SourceMemory SourceType = "memory"
+ // SourceSkill indicates the entry is a read-only index of an installed Skill.
+ SourceSkill SourceType = "skill"
+ // SourceResource indicates the entry is a UI-managed resource document that
+ // is read-only to agents. Resources are written via the management REST API
+ // and are progressively discovered by agents through memory_search and MCP
+ // Resources protocol (resources/list, resources/read).
+ SourceResource SourceType = "resource"
+)
+
+// EntryStatus is the lifecycle state of a memory entry.
+type EntryStatus string
+
+const (
+ // EntryStatusActive indicates the entry is in normal use.
+ EntryStatusActive EntryStatus = "active"
+ // EntryStatusFlagged indicates the entry has been marked for review.
+ EntryStatusFlagged EntryStatus = "flagged"
+ // EntryStatusExpired indicates the entry has passed its TTL.
+ EntryStatusExpired EntryStatus = "expired"
+ // EntryStatusArchived indicates the entry has been moved to the archive.
+ EntryStatusArchived EntryStatus = "archived"
+)
+
+// ArchiveReason records why an entry was archived.
+type ArchiveReason string
+
+const (
+ // ArchiveReasonConsolidated indicates the entry was merged into a newer entry.
+ ArchiveReasonConsolidated ArchiveReason = "consolidated"
+ // ArchiveReasonCrystallized indicates the entry was promoted to a skill.
+ ArchiveReasonCrystallized ArchiveReason = "crystallized"
+ // ArchiveReasonManual indicates the entry was manually archived.
+ ArchiveReasonManual ArchiveReason = "manual"
+ // ArchiveReasonExpired indicates the entry exceeded its TTL.
+ ArchiveReasonExpired ArchiveReason = "expired"
+)
+
+// Entry is the core domain type representing one stored memory.
+type Entry struct {
+ ID string
+ Type Type
+ Content string
+ Tags []string
+ Author AuthorType
+ AgentID string
+ SessionID string
+ Source SourceType
+ SkillRef string
+ Status EntryStatus
+ TrustScore float32
+ StalenessScore float32
+ AccessCount int
+ LastAccessedAt time.Time
+ FlaggedAt *time.Time
+ FlagReason string
+ TTLDays *int
+ ExpiresAt *time.Time
+ ArchivedAt *time.Time
+ ConsolidatedInto string
+ CrystallizedInto string
+ History []Revision
+ CreatedAt time.Time
+ UpdatedAt time.Time
+}
+
+// Revision records a single correction to a memory entry.
+type Revision struct {
+ Content string
+ Author AuthorType
+ CorrectionNote string
+ Timestamp time.Time
+}
+
+// ListFilter restricts results returned by MemoryStore.List.
+type ListFilter struct {
+ Type *Type
+ Author *AuthorType
+ Tags []string
+ Source *SourceType
+ Status *EntryStatus
+ CreatedAfter *time.Time
+ CreatedBefore *time.Time
+ Limit int
+ Offset int
+}
+
+// VectorFilter restricts similarity search to a subset of entries.
+type VectorFilter struct {
+ Type *Type
+ Status *EntryStatus
+ Source *SourceType
+}
+
+// ScoredID pairs an entry ID with its cosine similarity to a query.
+type ScoredID struct {
+ ID string
+ Similarity float32
+}
+
+// ScoredEntry pairs a full Entry with its similarity to a query.
+type ScoredEntry struct {
+ Entry Entry
+ Similarity float32
+}
+
+// ConflictResult describes a potentially conflicting existing memory returned
+// during a write conflict check.
+type ConflictResult struct {
+ ID string
+ Content string
+ Similarity float32
+ TrustScore float32
+}
diff --git a/pkg/memory/types_test.go b/pkg/memory/types_test.go
new file mode 100644
index 0000000000..8163259793
--- /dev/null
+++ b/pkg/memory/types_test.go
@@ -0,0 +1,31 @@
+// SPDX-FileCopyrightText: Copyright 2025 Stacklok, Inc.
+// SPDX-License-Identifier: Apache-2.0
+
+package memory_test
+
+import (
+ "testing"
+
+ "github.com/stretchr/testify/require"
+
+ "github.com/stacklok/toolhive/pkg/memory"
+)
+
+func TestMemoryTypeConstants(t *testing.T) {
+ t.Parallel()
+ require.Equal(t, memory.Type("semantic"), memory.TypeSemantic)
+ require.Equal(t, memory.Type("procedural"), memory.TypeProcedural)
+ require.Equal(t, memory.Type("episodic"), memory.TypeEpisodic)
+ require.Equal(t, memory.AuthorType("human"), memory.AuthorHuman)
+ require.Equal(t, memory.AuthorType("agent"), memory.AuthorAgent)
+ require.Equal(t, memory.EntryStatus("active"), memory.EntryStatusActive)
+ require.Equal(t, memory.EntryStatus("flagged"), memory.EntryStatusFlagged)
+ require.Equal(t, memory.EntryStatus("expired"), memory.EntryStatusExpired)
+ require.Equal(t, memory.EntryStatus("archived"), memory.EntryStatusArchived)
+ require.Equal(t, memory.SourceType("memory"), memory.SourceMemory)
+ require.Equal(t, memory.SourceType("skill"), memory.SourceSkill)
+ require.Equal(t, memory.ArchiveReason("consolidated"), memory.ArchiveReasonConsolidated)
+ require.Equal(t, memory.ArchiveReason("crystallized"), memory.ArchiveReasonCrystallized)
+ require.Equal(t, memory.ArchiveReason("manual"), memory.ArchiveReasonManual)
+ require.Equal(t, memory.ArchiveReason("expired"), memory.ArchiveReasonExpired)
+}