Guide

Agent memory for AI fleets: System-One control that stays cheap

By Atlas AI Resources · Published October 2, 2026 · How we make these guides

Long-running bots accumulate preferences, task history, and environment facts faster than any context window can hold. The win is not “more tokens” — it is treating memory control as its own cheap decision layer, then calling a full model only when you need to write the answer.

1. Context alone is not a memory system

Persistent agents (coding assistants, personal bots, research workflows) keep writing new observations. Past a point, stuffing the whole history into the prompt fails: models miss middle positions, retrieval gets noisy, and every turn pays for tokens you do not need. Agentic memory means preserving useful experience outside the prompt, organizing it, and recovering the right pieces when a new question arrives.

Official Grok Bot docs make a related point for fleets: bot memory is not a substitute for an authoritative source. Put changing facts in a system of record; put durable safety rules in the bot description. External memory research is about the store and retrieval path between those two.

2. Split cheap control from expensive reasoning

A September 2026 paper from UT Dallas researchers introduces Jev-Mem, an open MIT-licensed architecture inspired by System-One / System-Two cognition. System One handles fast, bounded decisions (labels, scores, route choices). System Two handles slower, open-ended generation. Most memory operations — typing a new note, judging a relation, routing a query, scoring candidates, deciding when to stop — fit System One. Final answer synthesis stays on System Two.

That separation matters for fleets because memory decisions fire constantly during construction and retrieval. Putting a full autoregressive model on every one of those steps burns latency and tokens. Jev-Mem’s controller uses typed probabilistic decisions (via Jev / TypeSafe-style interfaces, with optional local Laya support) so frequent control stays structured and batched.

3. One shared store, four relational views

Jev-Mem keeps canonical observations (text, timestamp, provenance) and links them across semantic, temporal, causal, and entity graphs. The same node can participate in several views. Deterministic links (timestamp order, exact shared IDs) are created in code; inferred links go through the controller only for a bounded candidate set, so cost does not grow with every pairwise compare.

The default write path preserves observations rather than discarding them at ingest. Selectivity happens when relations are built and when retrieval decides what evidence is enough — so a detail that looked unimportant can still surface later.

4. Retrieval as a loop: route, assess, stop

Instead of a fixed top-k dump, Jev-Mem treats retrieval as closed-loop control. The controller estimates which graph views the query needs, allocates a search budget, pulls hybrid vector/keyword anchors, expands neighbors, scores candidates (relevance, novelty, support), and reassesses evidence sufficiency. It can stop when evidence looks sufficient — or when further search looks unlikely to help — under explicit limits on depth, nodes, edges, controller calls, and time.

For a small fleet, the pattern to steal is simple: do not keep retrieving until the context is full. Decide what “enough evidence” means, bound the search, and hand a short evidence pack to the model that writes the reply.

5. Author-reported results (treat as paper measurements)

On the LoCoMo long-conversation benchmark with GPT-4o-mini as the answer model, the authors report an overall LLM-as-a-Judge score of 0.777 — an 11.0% relative improvement over the strongest baseline they compare (MAGMA at 0.700). They also report memory construction in 158 seconds (a 6.6× speedup over the fastest competing memory system in their table) and average query latency of 0.93 seconds (a 36.7% reduction versus the fastest memory baseline). Those numbers are the paper’s measurements on that setup, not a guarantee for your bots.

The code is early research software (MIT). Use it as a reference architecture and experiment harness, not as a drop-in production memory product.

What to steal for your own fleet

• Keep authoritative, changing facts outside any single chat memory.
• Move high-frequency memory decisions (type, relate, route, score, stop) off the generative path when you can.
• Store observations with provenance and time; connect them for “what / when / why / who,” not only embedding similarity.
• Bound retrieval: allocate a budget, assess evidence, stop early when you have enough.
• Reserve the full model for synthesis and hard reasoning — not every memory housekeeping call.
• If you try the open code, treat results as research: pin versions, read the paper’s limits, and verify on your own workloads.

Useful public resources

Related: Small AI fleets: handoffs · Grok Bot for engineering · Skills before routines · What is a one-job Grok Bot?