Persistent Agent Memory Engine π§ ¶
mini-infer includes an embedded, zero-dependency Agent Cognitive Memory Engine (mini_infer.memory). It provides long-term recall, hybrid semantic/lexical retrieval, and automatic prompt context injection across terminal chat sessionsβpreventing context window bloat and eliminating the need for heavy external vector databases.
1. The Agent Memory Problem¶
Autonomous AI agents and local inference engines face a fundamental dilemma:
- Context Window Limitations: Modern open-weight models (such as Qwen 2.5 0.5B/1.5B) have practical context budgets where KV-cache VRAM grows linearly with prompt length. Cramming entire chat histories and file trees into every prompt causes out-of-memory (OOM) failures on consumer GPUs.
- Session Amnesia: Standard terminal chats lose all state once the process exits. User rules, hardware specs, and design decisions must be re-explained in every session.
- Heavy Vector DB Bloat: Existing agent memory frameworks (e.g., LangChain, MemGPT, ChromaDB) require multi-gigabyte dependency trees, C++ compilers, external REST servers, or cloud API keysβviolating
mini-infer's zero-dependency philosophy.
The Solution: Embedded Zero-Dependency Cognitive Storage¶
mini-infer solves this with an integrated SQLite WAL storage layer combined with a deterministic, subword-hashed dense vector embedder and Okapi BM25 lexical ranker running in pure PyTorch/Python standard library.
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β mini-infer Interactive Chat / Agent Runner β
βββββββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββ
β User Query: "What quantization?"
βΌ
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β MemoryEngine Orchestrator β
β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β Hybrid Retriever β β
β β β β
β β Dense Cosine Similarity Okapi BM25 Lexical Matching β β
β β (256-dim subword hashing) + (Term & Inverted Document Freq) β β
β β \ / β β
β β βΌ βΌ β β
β β Score Fusion: Ξ± Β· S_dense + (1-Ξ±) Β· S_BM25 β β
β β β β β
β β βΌ β β
β β Modulate by Importance & Exponential Time Decay β β
β ββββββββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββββ β
β β β
β βΌ β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β Dynamic Prompt Context Injector β β
β β Injects top relevant verified facts into system prompt budget β β
β ββββββββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββββ β
ββββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββ
β ACID Commit / WAL Fetch
βΌ
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Embedded SQLite Store (.mini_infer/memory.db) β
β Indexed on `category`, `last_accessed`, `id` (Zero network overhead) β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
2. Memory Taxonomy¶
Memories are categorized into a 4-tier cognitive hierarchy:
| Category | Enum | Decay Behavior | Description | Example |
|---|---|---|---|---|
| Core | core |
None (\(D(t) \equiv 1.0\)) | Permanent hardware constraints, developer preferences, safety rules. | "RTX 3050 has 6GB VRAM; never exceed 2048 KV blocks" |
| Project | project |
Half-life = 30 days | Codebase architecture, module contracts, CLI signatures. | "INT4 quantized weights packed in 4-bit uint8 nibbles" |
| Decision | decision |
Half-life = 21 days | Architectural Decision Records (ADRs) and trade-off rationale. | "Selected BM25+dense hybrid over pure vector for symbol names" |
| Episodic | episodic |
Half-life = 7 days | Conversational history, temporary tool outputs, user prompts. | "User ran servebench against port 8000 on 2026-10-27" |
3. Mathematical Foundations¶
3.1. Subword-Hashed Dense Embeddings¶
To avoid downloading a multi-gigabyte BERT or Sentence-Transformers checkpoint, mini-infer computes deterministic 256-dimensional feature embeddings directly in PyTorch:
where \(D = 256\) is the embedding dimension and \(\mathbf{e}_k\) is a unit one-hot basis vector. The resulting vector is \(L_2\)-normalized:
Dense cosine similarity between query \(q\) and document \(d\) is computed via dot product:
This captures morphological similarity (e.g., quantization \(\leftrightarrow\) quant_linear, vram \(\leftrightarrow\) memory) in \(<0.2\text{ ms}\) on CPU.
3.2. Okapi BM25 Lexical Ranking¶
Dense embeddings can occasionally miss exact variable names or unique file paths. Okapi BM25 guarantees that exact identifier matches receive top priority:
where default parameters are \(k_1 = 1.5\) and \(b = 0.75\). BM25 scores are normalized to \([0, 1]\).
3.3. Hybrid Score Fusion & Time Decay¶
The final retrieval score combines dense semantic score, lexical BM25 score, memory importance weight (\(W_{\text{imp}} \in [0.1, 2.0]\)), and recency decay:
where default \(\alpha = 0.55\).
Exponential Decay with Frequency Reinforcement¶
For non-CORE categories, time decay follows an exponential half-life curve boosted by total access hits:
This mimics human long-term potentiation: frequently accessed memories are retained longer even as time elapses.
4. Deduplication & Consolidation¶
When adding memories via MemoryEngine.add(), incoming content is checked against all existing memories in the category:
- If cosine similarity \(S_{\text{dense}}(\text{new}, \text{existing}) \ge 0.88\), the engine reinforces the existing memory (incrementing its access hit count and refreshing its timestamp) rather than creating a duplicate entry.
MemoryEngine.consolidate()runs cognitive garbage collection, automatically pruning decayed memories when storage exceeds target capacity while permanently protectingCOREitems.
5. CLI & Chat Usage¶
5.1. Interactive Chat Shortcuts¶
When running mini-infer chat, the memory engine is active by default:
# Start chat with active Agent Memory
mini-infer chat --model Qwen/Qwen2.5-0.5B-Instruct --memory
# In-chat shortcuts:
User: /remember RTX 3050 has 6GB VRAM and runs INT4 inference
mini-infer: [Memorized] (core): RTX 3050 has 6GB VRAM and runs INT4 inference
User: /memory
mini-infer: Persistent Memories (1 items):
- [core] RTX 3050 has 6GB VRAM and runs INT4 inference
5.2. CLI Memory Management¶
mini-infer memory provides complete standalone CRUD and inspection commands:
# Add a verified fact
mini-infer memory add "Use group_size=64 for INT4 sub-byte quantization" --category project --importance 1.5
# List stored memories
mini-infer memory list
# Search memories using hybrid ranking
mini-infer memory search "quantization settings"
# Display database footprint and telemetry
mini-infer memory stats
# Garbage collect decayed items
mini-infer memory consolidate --max-items 500
# Purge memories
mini-infer memory clear --force
5.3. Environment Configuration¶
The storage location defaults to .mini_infer/memory.db in the current workspace. To use a custom path or shared global database across projects: