Architecture

File Layout

Canonical Recall runtime files live under ~/.agents/Recall/. Claude Code, OpenCode, and Grok platform homes contain per-file symlinks back to those canonicals. Pi instead records Recall’s source as a native package and keeps its separately installed MCP adapter/configuration under ~/.pi/agent/.

~/.agents/Recall/                       # Recall install root (canonical files)
├── recall.db                           # SQLite database (FTS5 + WAL mode)
├── recall.db-wal
├── recall.db-shm
├── shared/
│   ├── hooks/                          # Canonical hook files (.ts)
│   │   └── lib/                        # Hook helpers (.ts)
│   ├── skills/                         # Agent Skill canonicals (do-recall-*)
│   └── extract_prompt.md               # Extraction prompt template
├── claude/
│   └── Recall_GUIDE.md                 # Guide for Claude Code
├── opencode/
│   ├── plugins/                        # OpenCode plugin canonicals
│   │   └── lib/                        # Plugin helpers (.ts)
│   └── Recall_GUIDE.md                 # Guide for OpenCode
├── pi/
│   └── Recall_GUIDE.md                 # Canonical guide linked into Pi home
├── grok/
│   └── hooks/
│       ├── RecallLifecycle.json        # Canonical Grok capture hook
│       └── capture.ts                  # Exports the session and calls recall capture
├── MEMORY/                             # Migrated user-authored MEMORY files
│   ├── identity.md                     # L0 identity (user-authored via recall onboard)
│   └── DISTILLED.md                    # All extracted session summaries (full archive)
└── backups/                            # Pre-install + pre-update snapshots

~/.claude/                              # Claude Code home (mostly symlinks back)
├── Recall_GUIDE.md                     # → ~/.agents/Recall/claude/Recall_GUIDE.md
├── MEMORY/
│   ├── identity.md                     # optional managed link → ~/.agents/Recall/MEMORY/identity.md
│   ├── DISTILLED.md                    # → ~/.agents/Recall/MEMORY/DISTILLED.md
│   ├── HOT_RECALL.md                  # Last 10 sessions (fast context loading)
│   ├── SESSION_INDEX.json             # Searchable session metadata lookup
│   ├── DECISIONS.log                  # Architectural decisions (deduplicated)
│   ├── REJECTIONS.log                 # Things to avoid
│   ├── ERROR_PATTERNS.json            # Known error/fix pairs
│   ├── extract_prompt.md              # Extraction prompt template (used by hooks)
│   ├── EXTRACT_LOG.txt                # Extraction run log (checked by recall doctor)
│   └── .extraction_tracker.json       # Per-file extraction state (dedup + retry)
├── hooks/                              # All entries below are symlinks
│   ├── RecallStart.ts                # → ~/.agents/Recall/shared/hooks/RecallStart.ts
│   ├── RecallPreCompact.ts            # → ~/.agents/Recall/shared/hooks/RecallPreCompact.ts
│   ├── RecallExtract.ts               # → ~/.agents/Recall/shared/hooks/RecallExtract.ts
│   ├── RecallBatchExtract.ts           # → ~/.agents/Recall/shared/hooks/RecallBatchExtract.ts
│   ├── RecallTelosSync.ts             # → ~/.agents/Recall/shared/hooks/RecallTelosSync.ts
│   ├── RecallClearExtract.ts          # → ~/.agents/Recall/shared/hooks/RecallClearExtract.ts
│   ├── RecallInSession.ts             # → ~/.agents/Recall/shared/hooks/RecallInSession.ts
│   └── lib/                            # → ~/.agents/Recall/shared/hooks/lib/
└── settings.json                       # Hook registration + MCP server (recall-memory)

~/.grok/hooks/
└── RecallLifecycle.json                # → ~/.agents/Recall/grok/hooks/RecallLifecycle.json

The shared identity resolver and its complete precedence are documented under Tiered RecallStart.

Host Boundaries

Host-neutral CLI and MCP logic lives outside src/hosts/.

Native host adapters own config shapes, paths, transcript parsing, and native command discovery under src/hosts/.

Lifecycle hooks use the same boundary under hooks/lib/hosts/; the generic extraction cascade depends only on the host-neutral Extractor execution-adapter interface.

Recall-owned logs and mutable state resolve from RECALL_HOME (default ~/.agents/Recall) instead of a host configuration directory.

Codex’s preferred attach is the native plugin in hosts/plugins/recall/, discovered through .agents/plugins/marketplace.json. Its .mcp.json registers recall-memory, scripts/build-codex-plugin.ts generates host-adapted skills from the canonical sources, and plugin hooks provide supported transcript capture and session-start context. See Codex Integration.

Grok lifecycle capture is installer-owned. A managed global hook runs hooks/grok/capture.ts, which exports the session and calls recall capture. Grok has no verified automatic injection surface. See Grok Integration.

Ambient text from first-party capture adapters enters through recall capture. Core still scrubs and stores. JCode is not installed unattended: the live probe did not prove a public history watermark or additive hooks. See JCode Integration.

Claude Code’s preferred attach is the native plugin in hosts/plugins/recall-claude/ (skills + MCP). The lifecycle installer continues to own hooks. Stop shells recall capture; without hooks, run bun hooks/lib/hosts/claude/ambient-capture.ts. See Claude Integration.

Pi uses its native package for extensions and skills, with MCP configured separately by the installer. Package discovery and shutdown capture are documented in Pi Integration.

omp uses its native package for main-session turn-completion capture. Setup and capture limits are documented in omp Integration.

OpenCode stays on its markdown drop. Moving it to recall capture is the next adapter after this v1 door. See OpenCode Integration.

Extension surface

A new harness plugs in through existing seams, not a new Cursor marketplace plugin and not a HostDescriptor. The public library is recall-memory/api (Harness API):

• start — recall start / runStart / registerStartFormat (in-process). Same L0/L1 assembler as Claude RecallStart.ts.
• drop — MEMORY/<host>-sessions/ markdown. No registry; the directory name is the extension point.
• capture — Cursor catalogCursorSessions (catalog only). Dump discoverCurrentSession + registerSessionSource. Ambient harness text uses public recall capture. Hidden host-hook is deprecated for that write and remains only for Codex SessionStart injection. Cursor never joins either pipe.
• inject — Cursor templates/cursor/ + mergeCursorHooksJson. Command is unqualified recall start --format cursor. Durable Cursor.app GUI PATH accuracy is pending FM-321/327.

MCP (recall-mcp) and agent-skills/ remain the cross-host agent surfaces.

Database Tables

All FTS5-indexed tables have automatic sync triggers.

Portable JSON, Markdown, and SQL exports contain the seven durable memory and deduplication tables plus host_ingest_generations, host_ingest_generation_messages, host_ingest_embedding_invalidations, host_ingest_state, host_ingest_messages, and loa_message_sources. SQLite exports contain the full database.

The importance column was added in schema migration 7→8 (v0.7.0) on four tables (messages, decisions, learnings, loa_entries). It controls L1 tier ranking at session start. Manage manually with recall pin / recall unpin or backfill from confidence signals with recall importance backfill.

The provenance column was added in schema migration 8→9 on all five memory tables (messages, decisions, learnings, breadcrumbs, loa_entries). It declares how each record was created — verbatim, user_authored, extracted, or derived — and is stamped automatically by every write path, never accepted from callers (see docs/adr/0001-record-provenance-automatic-write-path-metadata.md). Legacy rows stay NULL (unknown) until classified with recall provenance backfill, which only acts on deterministic write-path evidence and never guesses.

The dedup_lineage table was added in schema migration 9→10. recall dedup marks duplicate records non-destructively by writing lineage rows here (survivor table/id, duplicate table/id, reason, similarity, status); marked duplicates stay in their source tables but are hidden from search unless --include-duplicates is passed. Survivor selection follows provenance order (user_authored > verbatim > extracted > derived > unknown), then richness, importance, and recency. Dedup acts within a table only; cross-table candidates are report-only.

Tiered RecallStart (v0.7.0+)

The RecallStart hook injects two tiers at the top of supported sessions:

Automatic terminal-capture LoA entries remain searchable but are excluded from the L1 LoA pool (tags not like automatic-capture,%), so host-generated summaries cannot displace curated knowledge at session start.

The preamble names L2/L3 as on-demand only. They are not injected. Fetch more with memory_search, memory_hybrid_search, or memory_recall (recent LoA, decisions, and breadcrumbs, not raw message history).

Path resolution for identity.md:

1. RECALL_IDENTITY_PATH env var (if set)
2. ./.atlas-recall/identity.md (project-local, if exists)
3. Existing user-owned ~/.claude/MEMORY/identity.md, if it is not the managed canonical link
4. The canonical file under the Recall install root: ~/.agents/Recall/MEMORY/identity.md

recall onboard uses the same resolver, with explicit --out first and --project forcing step 2 even before the file exists. A managed Claude identity link resolves to the canonical target rather than becoming a second storage location.

PreCompact hook (v0.7.0+)

hooks/RecallPreCompact.ts fires before Claude Code compacts its own context. It flushes any in-flight messages to SQLite so nothing is lost during compaction. A byte-offset watermark prevents re-reading and it cooperates with the Stop hook’s extraction lock to avoid races.

Search Architecture

graph LR
    Q[Query] --> FTS[FTS5 Keyword Search]
    Q --> EMB[Ollama Embeddings]
    FTS --> RRF[Reciprocal Rank Fusion]
    EMB --> RRF
    RRF --> R[Ranked Results]

    style Q fill:#3B82F6,color:#fff
    style RRF fill:#10B981,color:#fff
    style R fill:#3B82F6,color:#fff

Lifecycle messages use a separate publication-aware FTS index so a replacement generation never leaks partial rows into search. A search advances at most one bounded repair page; if publication is still pending, it returns available results with a RETRYABLE warning. MCP search tools mark that response as an error when no complete result is available. Retry the search or run recall repair --execute to advance the remaining work.

Extraction Pipeline

graph TD
    A[Session End] --> B[Stop Hook Fires]
    B --> C[RecallExtract.ts]
    C --> D[Read JSONL Conversation]
    D --> E{Size > 120K chars?}
    E -->|Yes| F[Chunk + Meta-Extract]
    E -->|No| G[Single Extraction]
    F --> H[Run Automatic-capture Extractor]
    G --> H
    H --> I{Quality Gate}
    I -->|Pass| J[Store to Memory Files]
    I -->|Fail| K[Log + Retry Window 24h]
    J --> L[DISTILLED.md]
    J --> M[HOT_RECALL.md]
    J --> N[SESSION_INDEX.json]
    J --> O[DECISIONS.log]
    J --> P[ERROR_PATTERNS.json]

    style A fill:#3B82F6,color:#fff
    style I fill:#F59E0B,color:#fff
    style J fill:#10B981,color:#fff
    style K fill:#EF4444,color:#fff

The hook self-spawns in background so the session exits immediately (non-blocking).

Automatic-capture LoA uses the Extractor cascade: default claude-cli (Haiku) then ollama (Recall_OLLAMA_MODEL, default qwen2.5:3b). Curated LoA (recall loa, dump extract) uses the fabric Extractor. Optional per-path config is below. OLLAMA_URL is the shared Ollama endpoint for embeddings and the automatic ollama Extractor; it is not a config.json field.

Extractor config

Optional file: ~/.agents/Recall/config.json. Install never writes it. Missing file = the split defaults above. Unparseable JSON fails both paths. An illegal Extractor ID fails that path closed.

Automatic-capture LoA may use claude-cli or ollama. Curated LoA may use fabric only. File selects IDs and fallback lists. RECALL_FABRIC_MODEL overrides curated model; Recall_OLLAMA_MODEL overrides the automatic Ollama model.

{
  "extractor": {
    "automatic": { "id": "claude-cli", "model": "haiku", "fallback": [{ "id": "ollama", "model": "qwen2.5:3b" }] },
    "curated":   { "id": "fabric", "model": "claude-haiku-4-5" }
  }
}

Harness steps

Optional sibling keys in the same file: query, extraction.automatic, extraction.curated, and cluster. A missing key keeps today’s path. These keys do not widen the Extractor allowlists above, and RECALL_EXTRACTOR does not select them.

A named step is { "id": "pi", "model": "sonnet" } with no argv. Supported ids are claude, pi, opencode, codex, grok, jcode, omp, and cursor. Recall starts Claude print mode, Pi --print, and codex exec. The other ids fail that step. A command step is a non-empty argv array, even when its label is one of those ids. Recall runs that argv with no shell. An empty argv fails that step. It does not guess vendor flags.

A blank claude model uses haiku. Other named ids need a model or that step fails. A query step sends the question alone. Extraction sends the prompt that path already builds. Cluster sends the synthesis prompt. A missing cluster key keeps the existing dated Claude call. A present list that fails does not call Claude afterward.

The file must not contain an API key. Install never writes this file. Extractor config does not read ~/.env.

{
  "query": { "primary": { "id": "pi", "model": "sonnet" }, "fallback": [{ "runner": "local", "mode": "hybrid" }] },
  "cluster": { "primary": { "argv": ["my-harness"] } }
}

Terms: CONTEXT.md. Extractor is the model backend that produces LoA, not a Host and not the ingest/filter/persist path.

Technical Details

Lifecycle Management

• Decision status transitions — decisions move from active → superseded (replaced by a newer decision) or active → reverted (rolled back). The decision_update MCP tool and recall decision CLI command handle these transitions. Superseded decisions are retained for historical context.

• Breadcrumb sweep — at session start, the shared assembler deletes breadcrumbs whose expires_at is already past, and deletes their embeddings in the same transaction. It does not drop rows by importance. recall age sets expires_at on old breadcrumbs; high-importance breadcrumbs with no expiry persist until explicitly removed.

• Prune strategy — recall prune removes consolidated messages and orphaned sessions older than the retention window, superseded/reverted decisions older than a retention window, breadcrumbs whose expires_at is past, old extraction_tracker rows, and extraction_sessions past a 500-row cap, with transactional embedding cleanup for deleted sources. It does not prune by importance. Legacy orphan embeddings are handled by recall repair --execute. Prune is always dry-run by default; pass --execute to commit changes.

• WAL mode allows concurrent reads and writes, though recovery and exclusive locks can still block queries

• Lock waiting — CLI/MCP connections and hook WAL connections set busy_timeout = 5000 before WAL setup so transient recovery locks can clear; persistent contention can still fail

• WAL retention — writable WAL connections set journal_size_limit to 64 MiB. SQLite truncates retained WAL space when the log resets after checkpointing. This is not a hard cap on an active WAL: large transactions or readers that prevent reset can keep it above the limit. Read-only hook connections leave the limit unchanged

• FTS5 full-text search with automatic sync triggers

• Foreign key constraints enforced

• Database and WAL/SHM file permissions set to 0600 (owner read/write only)

• Chunked extraction for sessions >120K characters with meta-extraction merging

• Quality gate rejects extractions missing required sections

• Persistence check marks a quality-passing extraction failed rather than extracted when its SQLite dual-write fails (for example an unwritable or locked database), so a session is never recorded as complete when some of its records did not land

• Retry window of 24 hours for failed extractions (quality-gate, extraction, and persistence failures alike)

• Parameterized queries — no SQL injection vectors

• PRAGMA user_version migration system for schema upgrades

Benchmark harness (v0.7.0+)

docs/benchmarks/runner.ts runs measurement suites and writes results to docs/benchmarks/results/ as JSONL plus a human-readable .md. Suite B (token efficiency) compares v2 wake-up context against v1 and the CLAUDE.md baseline. Methodology is locked in via 5 rules documented in docs/benchmarks/README.md. Run suites via recall benchmark run [suite].

Lifecycle scripts (v0.7.2+)

Three shell scripts in packaging/ manage the full install lifecycle. They share behavior via lib/install-lib.sh so every path (fresh install, update, uninstall) handles settings.json, hooks, MCP registration, and global-link state identically.

All three accept --dry-run to narrate changes without touching anything.

Shared library: lib/install-lib.sh

Sourced by all three scripts. Key functions:

Globals (overridable via env)

Callers can override these before sourcing lib/install-lib.sh:

CLAUDE_DIR="$HOME/.claude"
BACKUP_BASE="$HOME/.agents/Recall/backups"
TIMESTAMP="$(date +%Y%m%d_%H%M%S)"
BACKUP_DIR="$BACKUP_BASE/$TIMESTAMP"
OPENCODE_CONFIG_DIR="${XDG_CONFIG_HOME:-$HOME/.config}/opencode"
PI_CONFIG_DIR="${PI_CODING_AGENT_DIR:-$HOME/.pi/agent}"

All use : "${VAR:=default}" so an override set before source sticks. The test harness uses this to drive the lib against a tmpdir CLAUDE_DIR without touching the real home.

Agent skill: do-recall-update

Check-only. Reads the current version, polls GitHub Releases, and prints the exact cd <path> && ./packaging/update.sh recipe. Never runs update.sh inline — the recall binary lives in the same bun link process tree as the running Claude Code session, and rebuilding mid-session can corrupt in-flight hook invocations. The safe sequence is: exit Claude Code → ./packaging/update.sh → restart.