OKF Agent Memory – Git-native persistent memory for AI coding agents

Posted by okf_memory 3 days ago

Comments

Comment by langs 2 days ago

I don't get it. Why benchmark the latency instead of recall/precision? Optimizing for millisecond-level latency is meaningless in the context of LLM calls. Accuracy is the tool's greatest value, yet there is no testing for it?

Comment by esafak 2 days ago

Has anyone else benchmarked all these tools for precision/recall? I too want to know if agent memory is something I should add. I only do session memory for now and that is quite useful.

Comment by vshulcz 1 day ago

Benched cold start recently: 19k LongMemEval sessions laid down in the real ~/.claude and ~/.codex layouts, 100 questions whose answer sits in exactly one session, scored by whether that session comes back (bias: I built deja, one of the six)

deja: 29s to index, 24ms query, 18/100 hit@1, 67 found@50. Plain BM25, no vectors

agentmemory: 95s import, 14 hit@1, 65 found@50, plus a worker and engine on four ports

MemPalace: ~3h mining, 2.6s query, 14 hit@1

CASS: 56m index, every NL query fails with "query fuel exhausted" on the release build (fixed on their main)

claude-mem: no-op out of the box, only records forward from install

funes: the documented 1 min first pass indexed 189 of 19k sessions (0/100); full index still embedding, ~3 sessions/s

Numbers look low because 19k sessions is brutal; on the standard 500-session LongMemEval-S the same BM25 gets ~85% hit@1

The funny thing is BM25 basically ties embeddings here at 1/100th the cost. The real cliff is reranking (found@50 67 vs hit@5 35) and staleness. Vector search has zero concept of "superseded info" only fix I found was letting explicit user corrections outrank the transcript.

Repro scripts and corpus: https://vshulcz.github.io/deja-vu/guide/day-zero.html

@skeledrew: cross-agent across 23 harnesses, but yeah, it's an index over logs, not a source of truth :)

Comment by opwizardx 1 day ago

I did similar tests on my own corpus when was considering whether to keep semantic search in default path for pond.

On 3 months of my own sessions I’ve seen that BM25 search was finding the correct answer in ~61%, where semantic had shown only ~37% of success.

After that it was easy for me to make the decision.

Got all info on how I did evals in here, if interested: https://github.com/tenequm/pond/tree/main/docs/researches/26...

Comment by glub 1 day ago

Yes. See LongMemEval, LoCoMo. Tons of research here.

But precision/recall is relatively "solved". What nobody has gotten close to solving is maintenance and provenance - what goes into memory, what qualifies as truth, how stale memory gets invalidated/superseded.

We're now in the phase of re-discovering 30+ years of pain of knowledgebases.

Comment by olenzma 19 hours ago

Invalidation is not an algorithm, but a judgment made in context

Comment by nedomolkovivan 7 hours ago

[flagged]

Comment by entity002 7 hours ago

[dead]

Comment by kimseungyong 2 days ago

I like the idea.

To avoid losing context, I mainly conduct the planning session and the implementation session separately.

From the standpoint of building enterprise products, what worries me most is whether the agent we are implementing may not have understood a completely different context.

If okf_memory maintains domain knowledge very well, it is expected that implementation will be possible in unit functional units within a consistently smooth session.

However, there is a risk in applying this idea directly to practical work, so I’ll have to test it separately on a personal project.

Comment by iJohnDoe 3 days ago

This one was posted before?

https://github.com/fellowgeek/mcp-memory

https://news.ycombinator.com/item?id=49286073

Comment by straumat 1 day ago

Nice tool! If you want to publish your okf bundles for humans to read (like on a github pages or your intranet), i developped an open source solution to do that : https://github.com/oak-invest/kiso - It's like Hugo for OKF

Comment by calebkaiser 3 days ago

Love seeing projects like this. The performance benchmarks are nice to see. Have you done any benchmarks against approaches like OpenAI's Symphony for things like token usage or task completion?

Comment by ankitmathur01 2 days ago

[flagged]

Comment by rogeliodh 3 days ago

How does it compare to https://github.com/scaccogatto/okf-skills ?

Comment by swordsith 2 days ago

Personally I've always seen AI 'memory' as a pain point for people in their experience using LLMs than a benefit from the agent remembering the last unrelated thing you were working on. It wastes context similarly to 'skills'. The most efficient workflow imo is having a few well written (not by ai) md files across a clean codebase.

Comment by continuitykit 2 days ago

[flagged]

Comment by skeledrew 3 days ago

I want this, but also for cross-project memory. Save me from building my own, which I have planned but figure something would eventually pop up in HN...

Comment by opwizardx 1 day ago

I had been building https://github.com/tenequm/pond for this exact purpose for a while, and it works like a charm in my case.

Give it a try, hope it will help you to solve your need without injecting anything in your context all the time.

Comment by vshulcz 1 day ago

https://github.com/vshulcz/deja-vu

Comment by techgnosis 3 days ago

https://github.com/huggingface/funes

Maybe?

Comment by skeledrew 2 days ago

Not really. Could be a starter, but I specifically want everything to be stored as plain text and no vector search.

Comment by opwizardx 1 day ago

plain text + fts covers all the basic needs, adding vector search, in my experience, adds more hassle and failures then helps

Comment by _ink_ 2 days ago

Or https://github.com/mempalace/mempalace

Comment by DvirNiv 1 day ago

[flagged]

Comment by opwizardx 1 day ago

Why do you need an MCP server at all when the spec itself is so simple?

Comment by practicalsystem 3 days ago

cool going to check it out, I've got an opensource project that might compliment it that I'm excited to try

https://github.com/ucsandman/declick

Comment by triyambakam 3 days ago

How well does the model adhere to using this in a harness like Codex where it may be directed to use the built in memory tooling? Maybe I'll need to try an experiment directing it to save to its native memory to use OKF instead

Comment by mbreese 2 days ago

This is usually my main concern with tooling like this that isn’t a first party project. Anthropic can tune Opus, Fable, etc and their harness to use their memory format or preferred method of tool calling. I have had mixed results getting LLMs to consistently use third party tools.

I’m very much in favor of things like OKF wikis for memory or knowledge storage/retrieval. So I too would love to know how well this really integrates into one of the coding harnesses (Claude code or Codex mainly).

Comment by steammaho 2 days ago

honestly didn't notice much degrading of standard compact. Have several sessions which already lasts for several month and they are perfectly fine

Comment by hankbond 3 days ago

How are you using this compared to more explicit approaches where you lay out the project documentation in certain formats and conventions?

Comment by nullbio 3 days ago

Now that Astra is moving to a new compaction model (aka, ditching compaction altogether), is there any need for this sort of thing still?

Comment by pdimitar 2 days ago

Huh? Can you show source on the "it will not compact"? Very interesting.

Comment by esafak 2 days ago

"With Astra, we’re introducing a new way for Codex to preserve and retrieve context when the context window fills. Historically, models have used compaction to summarize work during long sessions, such as when debugging complex issues or tackling large refactors. Each compaction can leave out details about why a fix failed or how a component behaves. In Codex, Astra can keep notes across context windows, preserving accumulated details without repeatedly compressing them into a single summary. Earlier context windows remain searchable, so Astra can find requirements or test results from previous messages and tool outputs—even if that information wasn’t captured in its notes. You can enable this experimental feature in your Codex config.toml, (opens in a new window) and it will become the default for Astra in the coming weeks."

https://openai.com/index/gpt-6-astra/

Comment by bayesianbot 2 days ago

This clearly is Codex-specific, not as much a feature of the model (though obviously they probably have trained it to be great at working with their own tools). Sounds somewhat similar to pi-observational-memory I'm using with Pi.

Comment by svyatov 2 days ago

The performance is cool and all, but what about capture/retrieval quality? Are there any benchmarks for that?

Comment by muhammadwaqar1 2 days ago

[dead]

Comment by okf_memory 3 days ago

Hey HN,

We built OKF Agent Memory because we were frustrated with how AI coding agents (Claude Code, Cursor, Windsurf, local models) handle long-term project context.

Every time a context window closes or a session resets, the agent forgets architectural decisions, domain discoveries, and operational rules. The existing solutions fall into two extremes: 1. Ad-hoc flat files (CLAUDE.md, AGENTS.md, .cursorrules) that inevitably balloon into 20k-token monoliths, degrade agent focus, and cause "lost-in-the-middle" attention failure. 2. Vector databases / background daemons (Mem0, Letta, Zep) that introduce heavy runtimes (Python/Node), docker containers, proprietary storage silos, and recurring embedding API costs (adding 200–800ms per retrieval call).

Our approach: The "LLM Wiki" in pure Go.

OKF Agent Memory (v0.1.0) is a single, zero-dependency Go binary that turns your Git repository into a structured, self-validating knowledge corpus based on Google's Open Knowledge Format (OKF) v0.2 specification:

• In-Memory BM25 Search (<300µs): Fast lexical ranking across titles, YAML metadata, tags, and bodies directly in memory. No embedding APIs, zero network overhead, zero runtime cost. • Progressive Disclosure: Slashes prompt overhead by up to 90%. Instead of loading thousands of lines of context, the agent searches the bundle index and pulls only the exact 300-token concept required for the current task. • 100% Git-Native: Everything lives in `knowledge/` as human-readable Markdown. You audit your agent's memory via `git diff`, `git blame`, and code reviews. • Built-In Stdio MCP Server: `okf mcp knowledge` exposes native Model Context Protocol tools (`okf_search`, `okf_show`, `okf_create`, `okf_validate`) directly to Claude Code and Cursor. • Trust Tiers: Distinguishes authoritative human law (`verified: human:...`) from agent-generated drafts (`generated: agent:...`). • Sub-4ms Cold Starts (<15MB RSS): Starts in milliseconds with no VM spin-up.

Try it in 30 seconds: $ brew install okf-memory/tap/okf $ cd your-project && okf bootstrap .

GitHub: https://github.com/okf-memory/okf-agent-memory Docs & Landing Page: https://okf-memory.dev

We'd love your feedback on the architecture, the Go implementation, and how your coding agents behave with progressive disclosure memory!

Comment by lukevp 3 days ago

I love the idea! OKF 0.2 solves for the problem of the AIs generating massive amounts of documentation (far more than humans ever created) and giving it equal-weight over what a human actually approved and committed to. I've been using it to attribute my decisions with specific directions on how to ensure that it takes strong direction from my explicit decisions and clarifies implicit / AI-driven decisioning.

Adding on progressive disclosure to this is brilliant, and I love the idea of a fast, in-memory, single-binary tool. This is a great way to approach the solution to this problem.

One thing I will say though - I would never be able to use this in my enterprise. It would just be too much of an uphill battle to purchase something that is so niche in utility - this tool is not a ton different than just having the md files locally and having it use ripgrep to search over them, and telling CLAUDE to write the OKF files as well as an index when it makes changes, is it? is the index generated dynamically / is anything about the progressive disclosure different than just having the agent manage it while it documents?

If you're going for smaller teams that can buy tools without a ton of approval / procedural overhead, I think that might have some success. another possible solution would be to make the cross-repo search something that you can handle with OSS but you have to self-host, and then pay for support. if you got enough usage and penetration within an enterprise from the teams just using OSS and self-hosting, they might consider buying support after-the-fact.