No LLM in the loop. The consolidation pass is deterministic:
Pull the N most recent active memories (default 30) with embeddings
Pairwise cosine similarity, threshold 0.85
For each similar pair, check if they share extracted entities
Shared entities + similarity 0.85-0.98 → flag as potential contradiction (same topic, maybe different facts)
No shared entities + similarity > 0.85 → redundancy (mark for consolidation)
Second pass at 0.65 threshold specifically for substitution-category pairs (e.g., "MySQL" vs "PostgreSQL" in otherwise-similar sentences) — these are usually real contradictions even at lower similarity
Consolidation then collapses the redundancy set into canonical memories with combined importance/certainty. No LLM call, no randomness. Reproducible, cheap, runs in a background tick every ~5 minutes.
The LLM could improve this (better merge decisions, better entity alignment) but the tradeoff is cost and non-determinism. v1 is deterministic on purpose.
Source: crates/yantrikdb-core/src/cognition/triggers.rs and consolidate.rs next to it.
alt Hacker News
No LLM in the loop. The consolidation pass is deterministic:
Pull the N most recent active memories (default 30) with embeddings Pairwise cosine similarity, threshold 0.85 For each similar pair, check if they share extracted entities Shared entities + similarity 0.85-0.98 → flag as potential contradiction (same topic, maybe different facts) No shared entities + similarity > 0.85 → redundancy (mark for consolidation) Second pass at 0.65 threshold specifically for substitution-category pairs (e.g., "MySQL" vs "PostgreSQL" in otherwise-similar sentences) — these are usually real contradictions even at lower similarity Consolidation then collapses the redundancy set into canonical memories with combined importance/certainty. No LLM call, no randomness. Reproducible, cheap, runs in a background tick every ~5 minutes.
The LLM could improve this (better merge decisions, better entity alignment) but the tradeoff is cost and non-determinism. v1 is deterministic on purpose.
Source: crates/yantrikdb-core/src/cognition/triggers.rs and consolidate.rs next to it.