Then I found mini-SGLang, the stripped-down version. And there, right in the code, was radix trees.

You know that moment when you learn something new and suddenly spot it everywhere? Like buying a specific car model and then seeing it on every street? That’s exactly what happened. Once I saw radix trees in SGLang, I started finding them in places that had absolutely nothing to do with LLMs. Turns out radix trees solve a specific class of problems so well that they keep showing up across decades and completely unrelated domains.

What Is a Radix Tree? (The Basics)

A radix tree (sometimes called a compressed trie or Patricia tree) is what you get when you take a trie and stop wasting space.

In a regular trie, each node represents one character. “CAST” and “CASH” share the C→A→S prefix, but still need separate nodes for T and H. A standard trie uses 5 nodes total. Still wasteful.

A radix tree compresses single-child chains. Those two words share “CAS”, so you get one node for “CAS” and two children for “T” and “H”. Three nodes instead of five.

What makes them useful:

• O(k) operations where k is key length (not O(n) where n is number of keys)
• Space compression — single-child edges get merged
• Built-in hierarchy — prefix relationships are just there

This isn’t academic. These are the data structures powering systems you use every day.

IP Routing: Where It All Started

The original use case: longest prefix match for packet forwarding.

Your routing table has entries like:

• 10.0.0.0/8 → interface A
• 10.1.0.0/16 → interface B
• 10.1.2.0/24 → interface C

When a packet arrives for 10.1.2.5, you need the most specific match. Hash tables can’t do this. You need a tree.

Network cards implement radix trees in hardware for line-rate performance. We’re talking about a data structure from 1960, published by Fredkin before the internet existed, now routing the internet’s traffic.

I never connected the dots before. I’ve configured routing tables, worked with network equipment, and had no idea what data structure was hiding underneath. That’s the thing about good abstractions: they work so well you forget they’re there.

Redis: Streams

But radix trees didn’t stay in networking. They migrated into places you’d never expect.

Redis uses radix trees to power Streams: its append-only log data structure introduced in Redis 5.0.

The problem Streams solve: you need an ordered, time-indexed log that supports range queries by ID and efficient consumer group tracking. Hash tables can’t give you ordered iteration. Skip lists (what sorted sets use) would work but waste memory for append-mostly workloads.

XRANGE mystream 1526985054069-0 1526985054079-0  # Give me entries in this ID range

antirez wrote Rax, a radix tree implementation, specifically for this. Stream entries are delta-compressed into listpack macro nodes, and those nodes are indexed by a radix tree keyed on entry IDs. Because IDs are time-based and stored in big-endian order, nearby entries share long prefixes. Exactly what radix trees compress well.

The results are dramatic. Storing one million entries in a Stream uses ~17 MB. The same data in a sorted set + hash takes ~220 MB. That’s 13x less memory, thanks to the radix tree plus listpack combination.

I’ve used Redis in production for years. Streams were just “another data type” to me. I never looked under the hood to see that the same data structure routing my packets was also indexing my event logs.

Ethereum: The Cryptographic Twist

Ethereum uses a Merkle Patricia Trie: a radix tree where every node is a cryptographic hash of its children.

This isn’t for storage. It’s for proofs.

When you want to verify that an account exists in Ethereum’s state, you don’t download the whole database. You download the path from root to leaf, and the hashes along the way. Each node’s hash is computed from its children, so if any data is tampered with, the root hash changes. The tree structure lets you prove membership without revealing everything. Light clients can verify state without running a full node.

The Ethereum yellow paper (section 4.3) describes this in detail. Every account, every contract storage slot, organized as a trie.

Same data structure that’s routing your packets and sorting your Redis keys, now securing billions in cryptocurrency.

SGLang: LLM Inference

Now we’re back to where I started.

SGLang uses radix trees for KV cache prefix caching. Here’s the problem:

When you run LLM inference, the attention mechanism needs to keep track of all previous tokens (the KV cache). If you have 100 requests with the same system prompt, you’re computing the same KV cache 100 times. Wasteful.

SGLang stores token sequences as keys in a radix tree:

• Key: [101, 203, 77, 55] (the system prompt tokens)
• Value: pointer to the GPU memory holding the KV cache

When request #42 arrives with the same system prompt, the tree finds the matching path. Zero recomputation. But here’s the kicker: radix trees also handle partial overlaps. If request #43 has [101, 203, 77, 55, 88] and request #44 has [101, 203, 77, 55, 99], the tree splits after the fourth token. Both share the first four tokens, but diverge after.

For long prompts, recomputing KV cache can dominate inference time. SGLang’s prefix caching eliminates redundant work across requests.

Hash tables can’t do this. Earlier approaches like vLLM’s block-level hashing (16-token blocks) require exact block alignment. SGLang’s token-level radix tree catches sharing opportunities that block hashing misses.

I saw this firsthand. Running SGLang with multiple concurrent requests, all sharing common prompts. The throughput jumped. GPU memory usage dropped. The radix tree was doing the heavy lifting. The RadixAttention paper has the benchmarks.

The Common Thread: Hierarchical Prefix Matching

So what do IP routing, Redis Streams, Ethereum state, and LLM inference have in common?

• Hierarchical data (IP prefixes, time-based entry IDs, account states, token sequences)
• Variable-length prefixes (not fixed blocks)
• Need for prefix operations (longest match, range queries, membership proofs, cache sharing)

Radix trees solve all of these naturally. The structure is the solution.

This is why the pattern keeps winning. Engineers aren’t lazy and keep copying code. Radix trees are just the right tool for this specific class of problems. When you have hierarchical data with shared prefixes, you’ll find yourself reaching for a radix tree.

The Takeaway

Don’t take away “learn radix trees.” Take away learn to recognize patterns.

I didn’t connect SGLang to IP routing. I didn’t see the connection between Redis Streams and Ethereum state. These were separate problems in my head until I learned the underlying data structure.

Pattern matching is one of the best skills you can develop in software engineering.

Bloom filters show up in distributed databases, web crawlers, and blockchain light clients. B-trees power databases, filesystems, and even Git. Skip lists are in Redis, LevelDB, and concurrency libraries.

When you see the same pattern in unrelated systems, you’re not seeing coincidence. You’re seeing how certain solutions just fit certain problems.

I was reading about “cache chunks” in SGLang docs, completely lost. Then I found what was under the hood. That’s the thing about patterns — once you see them, you can’t unsee them. Next time you’re evaluating an inference server and someone mentions “prefix caching,” you’ll know to ask what data structure they’re using. Next time you’re designing a system with hierarchical data and shared prefixes, a radix tree might be waiting for you.

This is what happened to me with routing tables and Redis. I used them for years without understanding what made them tick. Now I see the pattern everywhere. The data structure you just learned, powering half the systems you use — that’s not a bug. It’s just good engineering.

Re-ranking is different. It’s where a language model actually reads the results and judges whether they’re relevant.

This is the piece that makes modern search feel almost magical. Retrieval finds candidates quickly but makes mistakes. Re-ranking catches those mistakes.

Fast and Shallow vs Slow and Deep

BM25 and vector search are fast because they use precomputed indexes. BM25 looks up terms in an inverted index. Vector search compares embeddings that were computed during indexing. Neither method reads the actual document at query time - they work from preprocessed representations.

This is a tradeoff. Speed requires simplification. The indexes capture something about each document, but not everything. A vector embedding compresses a document into a few hundred numbers. Nuance gets lost.

The result: retrieval methods sometimes return documents that seem relevant but aren’t. The embedding for “coffee machine maintenance” might be close to “coffee brewing techniques” because both involve coffee. But if you’re searching for maintenance guides, brewing techniques aren’t helpful.

This is where re-ranking comes in. A re-ranker actually reads the candidate documents and the query, then makes a judgment: is this document actually relevant to what the user asked?

How Re-ranking Works

A re-ranker is a small language model trained specifically to judge relevance. It’s not a general chatbot - it’s a specialist. Models like Qwen3-Reranker or BGE-Reranker are typically 500MB-1GB.

For each candidate document, the re-ranker receives:

• The original query
• The document content (or the matching chunk)

It outputs a simple judgment: yes or no, with a confidence score. “Yes, this document answers the query” or “No, it doesn’t.”

The confidence scores let the system adjust rankings. A high-confidence “yes” boosts a document up. A high-confidence “no” pushes it down. Uncertain judgments have less effect.

This is fundamentally different from retrieval. BM25 and vectors compute similarity metrics - statistical measures that correlate with relevance. The re-ranker makes a semantic judgment - it understands language well enough to assess whether the content actually addresses the query.

Two-Stage Retrieval

You might wonder: if the re-ranker is so good at judging relevance, why not use it for everything?

Because it’s slow.

Running a language model takes time - even a small one. If you have 10,000 documents, running the re-ranker on all of them would take far too long. BM25 can search 10,000 documents in milliseconds. The re-ranker might take seconds per document.

The solution is two-stage retrieval:

1. Stage 1 (retrieval): Use fast methods (BM25, vectors, RRF) to find the top 50-100 candidates
2. Stage 2 (re-ranking): Use the slow but accurate re-ranker to judge those candidates

This is the classic speed-accuracy tradeoff. Stage 1 casts a wide net quickly, accepting some false positives. Stage 2 filters carefully, using more expensive computation on a much smaller set.

The key assumption: if a document is truly relevant, stage 1 will probably find it. The retrieval methods don’t need to be perfect - they just need to not miss good results. Stage 1 handles recall; stage 2 handles precision.

Position-Aware Blending

Sophisticated systems don’t simply replace retrieval scores with re-ranker scores. They blend them, with the blend ratio depending on position.

Why different weights at different positions?

The top results from RRF are strong signals - documents that ranked highly in multiple retrieval methods. If both BM25 and vectors agree a document is relevant, it probably is. The re-ranker might disagree, but retrieval has a strong track record at the top. So the blend favors retrieval.

Lower positions are less certain. Maybe a document ranked #50 in BM25 but #5 in vectors. RRF puts it somewhere in the middle, but we’re not confident. Here, the re-ranker’s judgment matters more. It can rescue genuinely relevant documents that retrieval undervalued, or demote false positives that slipped through.

This is a hedge. We don’t fully trust either signal, so we blend them. The blend shifts based on how confident we are in the retrieval signal at each position.

The Full Pipeline

A complete hybrid search pipeline looks like this:

1. Query expansion: Generate variations of the query using a local model
2. Parallel retrieval: Run all queries against both BM25 and vector indexes
3. RRF fusion: Combine all ranked lists into unified scores
4. Re-ranking: Run the re-ranker on top candidates
5. Position-aware blending: Combine retrieval and re-ranker scores
6. Return results: Final ranked list

Each stage serves a purpose:

• Query expansion catches terminology mismatches
• BM25 catches exact keyword matches
• Vectors catch semantic similarity
• RRF combines signals from different methods
• Re-ranking filters false positives and promotes true relevance
• Position-aware blending hedges between retrieval and re-ranker confidence

It’s a lot of machinery for a search command. But the result is search that handles both “ECONNREFUSED error” (exact match) and “why is networking broken” (semantic query) gracefully.

Running Locally

All of this can run on your machine. A typical setup uses three models:

Total: about 2GB. Not tiny, but manageable. I’ve written before about small models and what they can do. These are specialized tools - not general chatbots, but experts at their narrow tasks. They run on CPU if needed, though GPU acceleration helps.

The local-first approach matters. Your documents don’t leave your machine. There’s no API to pay for, no rate limits, no service to go down. The search is as reliable as your laptop.

QMD is one implementation of this pipeline. It assembles BM25, vectors, RRF, and re-ranking into a coherent tool that runs entirely locally. But the concepts apply broadly - any modern search system worth its salt uses some combination of these techniques.

The Takeaway

Modern search is layered. Fast methods find candidates. Slow methods judge them. Fusion combines multiple signals. Each layer compensates for the limitations of the others.

BM25 is fast but only matches words. Vectors understand meaning but can be fooled by surface similarity. RRF combines them but can’t tell if a document actually answers the question. Re-ranking can, but it’s too slow to run on everything.

Together, they form a pipeline that’s greater than the sum of its parts. You ask a question in natural language, and documents appear ranked by genuine relevance - not just keyword density or embedding distance, but something closer to “does this actually help?”

The techniques aren’t new. BM25 is from 1994. Vector search has been around for years. RRF was published in 2009. Re-ranking with language models is newer but well-established. The insight is that these old ideas, thoughtfully assembled, produce something that feels like magic.

That’s often how useful software gets made. The components exist; someone just needs to put them together. Understanding what each piece does - and why - helps you appreciate the engineering, and maybe build something of your own.

But what if you want the same brain answering you on Telegram while you’re commuting, on Slack while you’re working, and on iMessage when your laptop is closed?

That’s the problem OpenClaw solves. One agent, multiplexed across every channel you use.

The Multiplexer Problem

Pi is stateless between sessions. It reads a conversation file, generates a response, writes the updated file. The conversation could come from anywhere — a terminal, a Slack webhook, a Telegram bot.

But Pi doesn’t handle the routing. Something else has to receive that Telegram message, figure out which conversation it belongs to, load the right session file, run Pi, take the response, and send it back to Telegram. That’s not agent work — it’s plumbing.

OpenClaw is the plumbing.

Routing: Which Agent, Which Session

When a message arrives from any channel, OpenClaw makes two decisions: which agent handles it, and which session it continues.

The binding system handles the first question. You configure rules:

• Telegram DMs → personal agent
• Work Slack → work agent
• Discord server → coding agent
• Everything else → default agent

Bindings match on channel, peer (who’s talking), account (which bot account received it), guild (Discord server), or team (Slack workspace). Specific matches override general ones. A binding for a specific Telegram chat overrides the catch-all Telegram binding.

Threads inherit their parent’s binding. If you’re in a Slack channel bound to your work agent and someone starts a thread, that thread routes to the same agent — even though technically it has a different peer ID. The binding system checks the parent context first, then falls back to direct matching.

A concrete config:

bindings:
  - match:
      channel: telegram
      peer:
        kind: dm
        id: "12345678"  # Specific contact
    agentId: personal
  - match:
      channel: slack
      teamId: "T0123WORK"
    agentId: work
  - match:
      channel: discord
      guildId: "coding-server"
    agentId: coding

The hierarchy is evaluated top to bottom. First match wins. The final implicit rule is “everything else → default agent.”

Session Isolation

Once OpenClaw knows which agent handles a message, it needs to know which session to continue. This determines what context the agent sees.

OpenClaw constructs session keys from agent ID, channel, and peer. The dmScope setting controls how conversations group together:

• main — all DMs collapse into one session (unified context)
• per-peer — each contact gets their own session (isolated conversations)
• per-channel-peer — same person on Telegram vs WhatsApp gets separate sessions (platform-isolated)

The trade-off is context bleed vs context isolation. With main, your agent knows you mentioned a project to your coworker and can reference it when your boss asks — useful for a unified assistant, awkward if you wanted those conversations separate. With per-peer, what you discussed with one person stays in that context.

Neither is universally right. It depends on how you want your assistant to behave.

Multi-Agent

OpenClaw supports multiple agents in one gateway. Each agent has:

• Its own model configuration (Opus for complex work, Sonnet for routine tasks)
• Its own sandbox settings (Docker container, filesystem access)
• Its own skills directory
• Its own sessions

The work agent doesn’t see personal conversations. The coding agent’s filesystem access doesn’t extend to your documents. Isolation isn’t just convenience — it’s the architecture.

A practical setup might look like:

• Personal agent on Opus for open-ended conversation
• Work agent on Sonnet with company Slack integration
• Coding agent on Sonnet with Docker sandbox and workspace access

Each agent is a Pi instance with different configuration. OpenClaw routes traffic to the right one.

Provider Failover

Cloud providers fail. Rate limits hit. Bills exceed quotas. A production assistant can’t just stop working when Anthropic returns 429.

OpenClaw maintains auth profiles — ordered lists of providers with automatic rotation. When one fails, it marks that profile as in cooldown and tries the next. Your config might list:

1. Anthropic Claude (primary)
2. OpenAI GPT-4 (fallback)
3. Local Ollama (emergency)

The failover is transparent. The agent doesn’t know which provider answered. The conversation continues on whatever’s available.

This enables cost optimization too. Route simple messages to the cheap model, complex ones to the expensive model. Use cloud providers for capability, local models for privacy. The routing layer makes these policies configurable.

Command Gating

Not everyone who can message your agent should be able to reset its memory or switch models. OpenClaw implements command gating — certain commands require approval.

Gated commands include:

• /reset, /new — clear session state
• /compact — force context compaction
• /model — switch the underlying model
• /think — adjust reasoning level

Who can run these commands? You configure an allowlist. For personal use, that’s just your user IDs. For team deployments, it might be specific Slack users or Discord roles. Everyone else can talk to the agent but can’t touch its configuration.

Media Staging

When someone sends an image or file, it has to reach the agent somehow. OpenClaw stages media into the sandbox workspace before Pi runs.

An image sent via Telegram:

1. OpenClaw downloads it from Telegram’s servers
2. Writes it to a temporary path in the sandbox workspace
3. Includes the path in the message context
4. Pi can read the file using its normal read tool

This keeps Pi’s tools minimal — it doesn’t need platform-specific media APIs. It just reads files. The infrastructure handles the translation from “Telegram photo” to “file at /workspace/tmp/image-123.jpg.”

What Happens to a Message

Trace a Telegram message through the system:

1. Receive — Telegram bot API delivers the message to OpenClaw
2. Route — Binding system matches channel + peer → agent
3. Session — Session key constructed, session file located
4. Stage — Attachments staged into sandbox workspace
5. Gate — Slash commands checked against allowlist
6. Directives — Inline directives extracted (/think high, /model gpt-4)
7. Run — Pi executes with the session and message
8. Chunk — Response split to fit platform limits (Slack: 40KB)
9. Send — Response delivered back to Telegram

The agent — Pi — handles step 7. Everything else is infrastructure. The infrastructure exists so the agent can be simple.

The Gateway

All this runs in a single process: the gateway. It maintains channel connections, routes messages, manages sessions, handles hot-reload.

The gateway runs on your hardware. Sessions live on your disk. Credentials stay on your machine. When you turn it off, it’s off.

This is different from agent-as-a-service. With hosted agents, the provider holds your sessions, sees your conversations, decides when to update the model. With OpenClaw, you hold everything. The trade-off is operational overhead — you’re running infrastructure. The benefit is control.

Pi in Production

Pi’s bet is that minimal agents are better agents — less bloat, more focus, clearer failures.

OpenClaw doesn’t contradict that bet. It validates it. Pi stays minimal. The infrastructure stays separate. When you’re debugging why the agent gave a bad response, you’re looking at Pi. When you’re debugging why a message didn’t arrive, you’re looking at OpenClaw. The concerns don’t mix.

This is what happens when you take a minimal agent and ask “how do I use this everywhere?” The answer isn’t making the agent bigger. It’s building infrastructure around the agent.

One brain, multiple eyes. The brain stays simple. The eyes multiply.

The obvious next question: why not use both?

Modern search systems do. They run BM25 and vector search in parallel, then combine the results. But combining ranked lists is harder than it sounds. The technique that makes it work - Reciprocal Rank Fusion - is elegant enough to be worth understanding on its own.

The Problem With Scores

Let’s say you search for “authentication flow” and get these results:

BM25 Results:

1. meeting-notes.md (score: 12.4)
2. auth-design.md (score: 8.7)
3. api-spec.md (score: 6.2)

Vector Results:

1. auth-design.md (score: 0.89)
2. login-flow.md (score: 0.84)
3. meeting-notes.md (score: 0.71)

Which document is most relevant overall?

The naive approach is averaging scores. But that doesn’t work - BM25 scores and cosine similarity scores are completely different things. A BM25 score of 12.4 doesn’t mean the same thing as a vector score of 0.89. They’re not on the same scale, they’re not measuring the same thing, and adding them together is meaningless.

You could try normalizing - convert both to 0-1 ranges and then combine. But normalization is tricky. What range do you normalize to? How do you handle documents that only appear in one list? The choices are arbitrary and affect results in ways that are hard to predict.

Reciprocal Rank Fusion sidesteps all of this by ignoring scores entirely. It only looks at ranks.

Ranks, Not Scores

RRF’s insight: position in a ranked list tells you something, regardless of the scoring function that produced it.

If a document is ranked #1 by BM25, that means BM25 thinks it’s the most relevant. If the same document is ranked #2 by vector search, that means vector search thinks it’s nearly the most relevant. We don’t need to understand or compare the scores - the ranks carry the signal.

RRF computes a combined score based purely on positions:

RRF(document) = sum of 1/(k + rank) for each list

Where k is a constant (typically 60) and rank is the document’s position in each list.

Let’s work through the example:

meeting-notes.md:

• BM25 rank 1: 1/(60+1) = 0.0164
• Vector rank 3: 1/(60+3) = 0.0159
• RRF: 0.0323

auth-design.md:

• BM25 rank 2: 1/(60+2) = 0.0161
• Vector rank 1: 1/(60+1) = 0.0164
• RRF: 0.0325

api-spec.md:

• BM25 rank 3: 1/(60+3) = 0.0159
• Vector rank: not present = 0
• RRF: 0.0159

login-flow.md:

• BM25 rank: not present = 0
• Vector rank 2: 1/(60+2) = 0.0161
• RRF: 0.0161

Combined ranking:

1. auth-design.md (0.0325)
2. meeting-notes.md (0.0323)
3. login-flow.md (0.0161)
4. api-spec.md (0.0159)

auth-design.md wins because it ranked highly in both lists. meeting-notes.md ranked #1 in BM25 but only #3 in vectors, so it comes second. Documents appearing in only one list score lowest.

Why k=60?

The constant k controls how much rank position matters.

With a small k (say, 1):

• Rank 1: 1/(1+1) = 0.50
• Rank 2: 1/(1+2) = 0.33
• The gap between #1 and #2 is huge

With a large k (say, 60):

• Rank 1: 1/(60+1) = 0.0164
• Rank 2: 1/(60+2) = 0.0161
• The gap between #1 and #2 is small

A larger k means positions matter less relative to appearing in multiple lists. Being #1 in one list isn’t much better than being #2. But appearing in both lists, even at mediocre ranks, beats appearing at the top of only one.

The original RRF paper (2009) found k=60 worked well empirically across various datasets. It’s become the standard default. The intuition: it rewards consensus across methods while still giving some credit to top positions.

Query Expansion: Casting a Wider Net

RRF combines results from different search methods. But you can go further by generating variations of the query itself.

Instead of just searching for “authentication flow,” you use a small language model to generate alternative phrasings. Maybe “user login process” or “identity verification steps.”

Then you run four searches, not two:

1. Original query → BM25
2. Original query → vectors
3. Expanded query → BM25
4. Expanded query → vectors

All four result lists feed into RRF. The original query gets weighted higher to keep it dominant, but the expansion helps catch documents using different terminology.

This addresses a limitation I mentioned in the BM25 post: you might not remember the exact words used. If your notes say “login” but you search for “authentication,” the expansion might generate “login” and catch those documents.

Query expansion is essentially automated synonym matching, powered by a language model that understands how concepts can be rephrased.

Why Hybrid Beats Pure

You might wonder: if vector search understands meaning, why bother with BM25 at all?

Because they have different failure modes.

BM25 fails when:

• You use different words than the document (“auth” vs “authentication”)
• You’re searching for concepts, not terms
• You can’t remember exact phrasing

Vector search fails when:

• You want exact matches (error messages, function names)
• The terms are rare or domain-specific
• The document is tangentially related but not actually relevant (high similarity, low relevance)

Consider searching for “ECONNREFUSED timeout.” BM25 will find documents containing that exact error string. Vector search might return documents about network errors generally - related, but not specifically about ECONNREFUSED.

Or consider searching for “how we handle user sessions.” Vector search will find documents about session management even if they never use the word “session.” BM25 would miss them if the terminology differs.

Hybrid search gets both. RRF ensures documents matching both methods rise to the top, while documents matching only one method still appear - just ranked lower.

Putting It Together

A hybrid retrieval pipeline looks like this:

1. Query expansion: Generate variations of the original query
2. Parallel retrieval: Run all queries against both BM25 and vector indexes
3. RRF fusion: Combine all ranked lists into one
4. Return results: Ranked by combined RRF score

The whole thing can run locally. The expansion model, the embedding model, the BM25 index - all on your machine. No API calls, no network latency, no data leaving your control.

For most searches, this is what you want. It’s slightly slower than pure BM25 (model inference takes time), but the results are substantially better for anything beyond exact-match queries.

RRF Beyond Search

RRF is useful anywhere you need to combine ranked lists from different sources. Election aggregation, recommendation systems, meta-analysis of studies. The principle is the same: ranks carry information even when scores don’t compare.

The elegance is in what RRF ignores. It doesn’t try to understand the scoring functions. It doesn’t normalize or calibrate. It just observes: this document ranked highly according to multiple independent methods. That’s a signal worth trusting.

It’s a reminder that sophisticated problems sometimes have simple solutions. Combining search rankings could involve complex machine learning, learned weighting schemes, elaborate normalization procedures. Or it could involve adding up reciprocals of positions. The simple approach often wins.

The Takeaway

BM25 and vector search answer different questions. BM25 asks “does this document contain these words?” Vector search asks “is this document about the same thing?” Hybrid search asks both.

RRF combines their answers without needing to understand or compare their scores. Documents that both methods like rise to the top. Documents that only one method likes still appear, just lower.

Query expansion catches terminology mismatches by searching for variations you didn’t think of.

Together, these techniques turn keyword search and semantic search from competing approaches into complementary ones. The result is search that handles both precise technical queries and vague conceptual ones - the best of both worlds.

If you’re extending an agent at runtime, mistakes are inevitable. A skill with a bug. A command that hangs. A change that corrupts state. Traditional agents either crash or carry corrupted context forward. Pi does something different.

Sessions aren’t logs. They’re trees.

The Problem with Logs

Most chat systems store conversation as a flat log. Message 1, message 2, message 3, appended forever. When you need to recover from a mistake, your options are limited: clear the whole session, or manually edit the log file and hope you don’t corrupt it.

Flat logs assume conversations are linear. But agent workflows aren’t linear. You try something, it fails, you backtrack. You explore a tangent, learn something, return to the main thread. You want to test a risky operation without committing to it.

Pi’s SessionManager stores conversations as trees.

Tree Structure

Pi sessions are JSONL files where every entry has an id and parentId:

{"type":"session","version":3,"id":"abc123","cwd":"/workspace"}
{"id":"e1","parentId":null,"type":"message","role":"user","content":"..."}
{"id":"e2","parentId":"e1","type":"message","role":"assistant","content":"..."}
{"id":"e3","parentId":"e2","type":"message","role":"user","content":"..."}

The parentId links create a tree. Most of the time, the tree is just a linear chain — normal conversation. But when you branch, the tree structure enables recovery.

Branching

Pi’s SessionManager exposes createBranchedSession(leafId). You pick any point in the conversation history, and Pi forks a new branch from there.

The branch inherits everything up to the fork point. After that, it diverges. You can experiment in the branch — test a dangerous command, try a different approach, debug a broken skill. The main session remains untouched.

When the branch work is done, you have options:

• Discard — the experiment failed, throw away the branch
• Summarize — extract learnings into a summary, apply to main session
• Continue — the branch becomes the new main line

Branch summaries get persisted as a special entry type:

{"id":"bs1","parentId":"e47","type":"branch_summary","summary":"Debugged deploy skill: config template had unescaped quotes, added YAML validation step before write"}

The summary is dense — just the learnings, not the debugging conversation. When you return to the main session, the agent sees the summary without the noise of trial and error.

This is what I called “intentional compaction” but with actual architecture supporting it. You’re not manually copying context between sessions. The tree structure makes branching a first-class operation.

Compaction as Tree Pruning

Context windows fill up. When they do, you’re in the dumb zone — the model drifts and forgets instructions. You need to drop old content. Most agents just truncate — keep the last N messages, drop everything else.

Pi’s compaction is smarter. It summarizes older conversation into a persistent compaction entry:

{"id":"c1","parentId":"e50","type":"compaction","summary":"...","firstKeptEntryId":"e51","tokensBefore":45000}

The summary isn’t just dropped context — it’s compressed knowledge that remains in the tree. Future turns see the compaction summary plus messages after firstKeptEntryId. The information is preserved, just in denser form — compression as journey, not just mechanism.

Auto-compaction triggers when:

contextTokens > contextWindow - reserveTokens

The settings are configurable:

{
  "compaction": {
    "enabled": true,
    "reserveTokens": 20000,
    "keepRecentTokens": 20000
  }
}

reserveTokens is headroom for prompts and the next response. keepRecentTokens controls how much recent context survives compaction. OpenClaw enforces a floor of 20,000 tokens for reserveTokens — enough room for multi-turn housekeeping before compaction becomes unavoidable. Set it lower and OpenClaw bumps it up.

Pre-Compaction Memory Flush

OpenClaw adds a safety net: before compaction triggers, the system runs a silent agentic turn — tackling the same persistent memory problem that Beads tried to solve with dependency graphs, but with a simpler approach.

The flush runs when context crosses a “soft threshold” — below Pi’s compaction threshold but close enough to warrant concern. The system injects a message asking the agent to persist critical state to disk. The agent writes to memory/YYYY-MM-DD.md in the workspace.

The turn is silent — NO_REPLY at the start suppresses delivery to the user. You don’t see the housekeeping. But when compaction runs, the durable state has already been saved.

Configuration:

{
  "compaction": {
    "memoryFlush": {
      "enabled": true,
      "softThresholdTokens": 4000
    }
  }
}

The flush runs once per compaction cycle. If the session stays below threshold, no flush. If it crosses threshold multiple times, one flush per cycle. The workspace file survives compaction — you don’t lose critical context just because the window filled up.

Hot-Reload

Trees make self-extension recoverable. But OpenClaw adds another layer: hot-reload for the infrastructure around Pi.

OpenClaw runs two watch systems in parallel:

Skills watcher — monitors skills/ directories using chokidar (a cross-platform file watcher), debounces changes (250ms default), bumps snapshot versions when skills change. When you edit a skill, it’s available immediately. No restart.

Config watcher — rule-based reload for configuration. Some changes can hot-reload; others require gateway restart:

• Hooks and cron jobs hot-reload instantly
• Browser control settings hot-reload
• Gateway and plugin changes require restart
• Skills have their own dedicated watcher

This separation prevents unsafe reloads while enabling fast iteration on safe changes.

Recovery Workflow

Here’s a concrete scenario. You’re working with mom on a deployment task. mom writes a new deploy skill to automate your release process. You invoke it, and the skill has a bug — it writes corrupted config to your staging environment.

With a flat-log agent, your options are bad: start over and lose all context, or try to have the agent fix its own mess while carrying the corrupted state forward. Both paths hurt.

With Pi’s tree structure, you branch from before the damage:

1. /branch e47 — fork from the message before the deploy
2. Debug in the branch — the main session stays untouched
3. Fix the skill, test it, verify it works
4. /summary — extract the learnings into a dense summary
5. Return to main — the summary persists, the debugging noise doesn’t

The skill is now fixed and hot-reloaded (no restart needed). You can return to main and run the deploy again. The branch gave you a safe space to recover without contaminating your primary context.

Trees give you actual recovery. Flat logs give you “start over” or “hope for the best.”

The JSONL Format

Pi’s choice of JSONL is deliberate. It’s append-only, which means writes are atomic — no risk of corrupting the file mid-write. It’s human-readable, so you can inspect sessions with standard tools. It’s line-based, so you can grep for specific content.

The tree structure lives in the parentId links, not in nested JSON. This means you can still process the file line-by-line. Tools that don’t understand trees just see a flat log. Tools that do understand trees can reconstruct the full structure.

What This Enables

Tree-structured sessions enable workflows that flat logs can’t support:

Parallel exploration. Fork multiple branches to try different approaches simultaneously. Compare results, pick the best one.

Safe experimentation. Test risky operations in branches. If they work, merge. If they fail, discard.

Debugging isolation. When something breaks, branch to debug without polluting the main context with error messages and failed attempts.

Recoverable self-extension. Agents can extend themselves knowing that mistakes are recoverable. This is what makes mom’s self-writing skills viable in practice.

The Architecture

Malleable code — agents that write and modify their own capabilities — requires architecture that makes mistakes recoverable. Pi’s session trees provide that architecture.

Flat logs assume perfect execution. Trees assume failure is normal and recovery is essential.

For self-extending agents, the second assumption is obviously correct. And once you have trees, you stop fearing experimentation. Branch, try something risky, recover if it fails. The architecture makes courage cheap.

Search for “authentication” and BM25 won’t find documents that say “login.” To BM25, those are different strings. It has no concept that they mean related things.

Vector search fixes this. It finds documents by meaning, not just by words.

Meaning as Coordinates

Here’s the core idea: what if we could place words in space, where similar meanings are close together?

Imagine a map where “dog” and “puppy” are neighbors, “cat” and “kitten” are nearby, and “refrigerator” is off in a distant corner. If you could build such a map, then searching for “puppy” would naturally find documents about “dogs” - they’re in the same neighborhood.

This is what embeddings do. They convert text into coordinates - lists of numbers that represent position in a high-dimensional space. Similar meanings end up with similar coordinates.

The dimensions aren’t things you can name, like “animal-ness” or “size.” They’re abstract features learned from patterns in massive amounts of text. But the result is intuitive: words that appear in similar contexts end up near each other.

“Coffee” and “tea” appear in similar sentences - people drink them in the morning, they’re served hot, they’re caffeinated. So their embeddings are close. “Coffee” and “democracy” rarely share context, so they’re far apart.

How Embeddings Get Made

An embedding model reads text and outputs a vector - a list of, say, 768 numbers. Those numbers are the coordinates.

The model learns these representations during training. It sees billions of sentences and learns to predict words from context. In the process, it develops internal representations where similar things cluster together.

You don’t train these models yourself. You use pre-trained ones. Modern embedding models like Gemma or E5 are small enough to run locally - a few hundred megabytes. Feed them a sentence, get back coordinates.

The key insight: if you embed both your documents and your search query using the same model, you can find documents by asking “which document coordinates are closest to my query coordinates?”

Cosine Similarity: Which Way Are You Pointing?

Once you have coordinates, you need a way to measure closeness. The standard approach is cosine similarity.

Think of each vector as an arrow pointing from the origin to its coordinates. Cosine similarity measures the angle between two arrows. If they point in the same direction, similarity is 1. If they’re perpendicular, it’s 0. If they point opposite ways, it’s -1.

Why angles instead of distances? Because it handles magnitude differences gracefully. A long document and a short document about the same topic will have vectors pointing the same direction, even if one vector is “longer” (has larger coordinate values). The direction captures meaning; the length is mostly noise.

When you search “how users prove their identity,” vector search:

1. Embeds your query into a vector
2. Compares that vector against all stored document vectors
3. Returns documents with the highest cosine similarity

Documents about authentication, login, credentials, and identity verification all end up near each other in the embedding space. Your query about “proving identity” lands in the same neighborhood, so they match - even though the words are different.

The Problem With Big Documents

There’s a catch. Embedding models have context windows - limits on how much text they can process at once. A typical embedding model might handle 512 or 2048 tokens (roughly words).

Your documents are often longer. A meeting transcript might be 10,000 words. A technical spec might be 5,000. You can’t just feed the whole thing into the embedding model.

But what if you could? Would you even want to? A single embedding for a 10,000-word document would be a blurry average of everything in it. Search for “authentication” and you’d match a document that mentions authentication once in a sea of unrelated content - because the embedding represents the whole document, not the relevant part.

The solution is chunking: splitting documents into smaller pieces, each with its own embedding.

Chunking: The Art of Slicing Text

A common approach chunks documents into pieces of about 800 tokens each. That’s roughly 600 words, or about a page of text.

Why 800? It’s a tradeoff:

• Too small (100 tokens): You lose context. A chunk about “it” doesn’t tell you what “it” refers to.
• Too large (4000 tokens): You’re back to blurry averages. Specific topics get diluted.
• 800 tokens: Big enough to contain a coherent idea, small enough to be specific.

But there’s a problem with slicing text: you might cut right through an important passage. If a paragraph about authentication gets split between two chunks, neither chunk has the complete picture.

The fix is overlap. Each chunk shares some content (typically 10-20%) with the next chunk. The end of chunk 1 overlaps with the beginning of chunk 2.

This means text near chunk boundaries appears in two chunks. If your search matches that text, you’ll find it - it won’t fall through the cracks.

More overlap means better boundary coverage but more storage and slower indexing. Less overlap means faster indexing but more risk of missing boundary content. 15% is a reasonable middle ground.

What Gets Stored

When you index documents for vector search:

1. Each document gets split into chunks with overlap
2. Each chunk gets embedded into a vector (768 or so numbers)
3. Vectors get stored in an index alongside the original text

A 5,000-word document might become 8-10 chunks, each with its own embedding. When you search, the system checks all chunks and returns the ones whose vectors are closest to your query.

The results show which chunk matched, not just which document. This is useful - you see the specific passage that’s relevant, not just “somewhere in this giant file.”

Where Vector Search Shines

Vector search excels at conceptual queries:

• “How users prove their identity” finds authentication docs even if they say “login”
• “Making containers talk to each other” finds networking docs regardless of terminology
• “Why is the build failing” matches troubleshooting guides phrased differently

It’s also robust to phrasing. “User authentication flow,” “how login works,” and “verifying user identity” all land in similar regions of embedding space. Vector search finds the same documents regardless of how you phrase the question.

Where Vector Search Struggles

Vector search isn’t perfect. It has blind spots that BM25 handles better:

Exact terms: Search for “ECONNREFUSED” and vector search might return documents about network errors in general. BM25 would find the exact error message.

Rare technical terms: Embedding models are trained on common text. Obscure jargon, internal code names, or domain-specific terminology might not be well-represented in the embedding space.

Precision vs recall: Vector search casts a wide net. Sometimes you want exactly what you typed, not semantically related content.

This is why the best search systems offer both approaches. BM25 when you know the exact terms. Vectors when you’re exploring concepts. And increasingly, hybrids that combine both signals.

The Local Advantage

I’ve written before about the advantages of local models. For search, the benefits are clear:

• Privacy: Your documents never touch external servers
• Speed: No network latency, no rate limits
• Cost: After setup, every query is free
• Reliability: Works offline, no service dependencies

Modern embedding models are small enough to run on modest hardware. They download once and cache locally. The tradeoff is that they’re less powerful than massive cloud models - but for document search, they’re more than sufficient.

The Takeaway

Embeddings convert meaning into coordinates. Similar meanings cluster together. Cosine similarity measures how close two meanings are.

Chunking handles long documents by splitting them into pieces small enough to embed meaningfully, with overlap to avoid losing content at boundaries.

Together, these techniques let you search by concept rather than by keyword. “Authentication,” “login,” and “proving identity” all find the same documents - because they mean the same thing, even though the words are different.

BM25 asks: “Does this document contain these words?”

Vector search asks: “Is this document about the same thing as my query?”

Both questions are useful. The best search systems answer both.

The answer is mom — short for “Master Of Mischief.” It’s a Slack bot Mario built on Pi that does something I haven’t seen elsewhere: it manages itself.

The Setup Problem

Traditional AI agents require you to configure everything upfront. Install dependencies. Set up credentials. Define tools. Configure integrations. The agent is powerful, but only after you’ve done the work to make it powerful.

mom flips this. You give it Slack access and some API keys. It handles the rest.

Need git? mom runs apk add git inside its Docker container. Need to call an API? mom asks for the credentials, stores them appropriately, and writes a CLI wrapper. Need a capability that doesn’t exist? mom writes a skill for it.

The agent becomes responsible for its own environment.

The Workspace

mom’s self-management works because of a carefully designed workspace structure. Since mom is a Slack bot, the hierarchy maps to Slack channels — but the pattern works for any multi-context setup:

./data/
├── MEMORY.md              # Global context across all Slack channels
├── settings.json          # Configuration
├── skills/                # Reusable tools (global)
├── C123ABC/               # Per-channel directory (Slack channel ID)
│   ├── MEMORY.md          # Channel-specific memory
│   ├── log.jsonl          # Message history (source of truth)
│   ├── context.jsonl      # What the LLM sees
│   ├── attachments/
│   ├── scratch/
│   └── skills/            # Channel-specific tools

Two things matter here.

First, the two-file memory system. There’s a global MEMORY.md that persists across all channels, and a channel-specific MEMORY.md for local context. mom reads both before every response and can update them on request. This isn’t RAG or vector search — it’s just markdown files the agent knows to check.

Second, the skill hierarchy. Skills in /workspace/skills/ are global. Skills in /workspace/{channelId}/skills/ are channel-specific and override global ones. mom discovers available skills by reading SKILL.md files from both locations.

Self-Installing Tools

mom runs in Docker with only the data directory mounted. The container mounts only the data directory — the host filesystem is invisible. The container starts minimal — Alpine Linux with almost nothing installed.

When mom needs a tool that isn’t there, it installs it:

apk add git jq curl

The system prompt instructs mom to determine the appropriate package manager based on the container OS. Alpine uses apk. Debian uses apt. The agent figures it out.

Changes persist across sessions because the container stays running. mom installs git once, and it’s there for every future conversation. If the container gets recreated, mom has logged what it installed to SYSTEM.md and can restore the environment.

Self-Creating Skills

Here’s where it gets interesting. mom doesn’t just use skills — it creates them.

A skill is a directory containing a SKILL.md file and optional scripts:

/workspace/skills/note/
├── SKILL.md
└── note.sh

The SKILL.md has YAML frontmatter defining metadata:

---
name: note
description: Create and manage notes in the workspace
---

# Note Skill

Usage: `./note.sh <action> [args]`

Actions:
- `add <text>` - Add a new note
- `list` - List all notes
- `search <query>` - Search notes

mom discovers skills by scanning for SKILL.md files. The frontmatter goes into context (~100 tokens per skill). The full documentation only loads when mom decides to use the skill.

When mom encounters a task it can’t handle with existing skills, it can write a new one. Create the directory, write the SKILL.md, write the script, and the skill is immediately available. No restart, no configuration, no approval flow.

The Context Sync

mom maintains two JSONL files per channel. JSONL is “JSON Lines” — one JSON object per line, easily appendable and greppable. Each line is a complete message or event.

• log.jsonl — the source of truth, containing every message
• context.jsonl — what the LLM actually sees, synced from log.jsonl before each response

This separation enables something clever. When context fills up, mom compacts context.jsonl by summarizing older messages. But log.jsonl remains complete. If mom needs to recall something from before the compaction, it can grep log.jsonl directly.

This is what I was describing as intentional compaction — but built into the architecture rather than requiring manual intervention.

The Docker Model

mom’s isolation model is simple: the container mounts only the data directory.

docker run -d --name mom-sandbox \
  -v $(pwd)/data:/workspace \
  alpine:latest tail -f /dev/null

mom --sandbox=docker:mom-sandbox ./data

mom executes commands inside the container via docker exec. It can install packages, write files, run scripts — but only within the mounted workspace.

This isn’t perfect security. Credentials stored in the workspace are accessible to the container. A malicious model could exfiltrate them. There’s no built-in secrets manager or credential vault — just files in the workspace. But it’s a reasonable trade-off: contain the blast radius without pretending you can prevent all harm.

Skill Hierarchy in Practice

The two-level skill system — global and channel-specific — enables useful patterns.

Say you have a global github skill that wraps gh CLI. Works fine for most channels. But your work channel needs to use a different GitHub org with different credentials. You create a channel-specific skill that overrides the global one:

/workspace/skills/github/SKILL.md         # Global: personal GitHub
/workspace/C123ABC/skills/github/SKILL.md # Override: work GitHub

mom checks channel-specific skills first. If it finds a match, it uses that. Otherwise, it falls back to global. The override is implicit — no configuration, just file placement.

Event-Driven Wake-ups

mom isn’t limited to responding to messages. It supports three types of scheduled events:

• Immediate — one-shot execution right now
• One-shot — scheduled for a specific future time
• Periodic — cron-based recurring execution (cron is the Unix scheduler format: 0 9 * * * means “9am every day”)

A DevOps channel might schedule mom to check deployment status every morning at 9am. mom wakes up, runs the health check, writes findings to MEMORY.md, and goes back to sleep. When you check in later, the summary is already there.

Webhooks work similarly. External systems can POST to mom’s endpoint, triggering an agent run with the webhook payload as context. CI/CD pipelines, monitoring alerts, calendar events — anything that can send HTTP can wake mom.

The agent runs, handles the event, and persists state to the workspace. No human in the loop required.

What This Enables

mom’s self-management enables workflows that would be tedious to configure manually:

Credential accumulation. The first time you mention GitHub, mom asks for a token, stores it in the workspace, and configures gh CLI. Next time, it just works. Over time, mom accumulates the credentials and tools for your specific workflow. (Yes, this means credentials live as files in the workspace — the security model is “contain the blast radius,” not “prevent all access.”)

Context-specific skills. A channel for DevOps work might have skills for Kubernetes and AWS. A channel for writing might have skills for grammar checking and research. Each channel’s skill directory reflects its purpose.

Autonomous maintenance. Scheduled wake-ups mean mom can do background work — checking logs, updating dashboards, sending reminders — without waiting for you to ask.

What Power Users Build

Armin Ronacher runs Pi with a collection of custom extensions that demonstrate the pattern at scale. His setup includes:

• /answer — reformats agent questions into dialog boxes for cleaner interaction
• /todos — manages to-do lists as markdown files in the workspace
• /review — code review interface with branching for iterative feedback
• /control — multi-agent experimentation, running parallel approaches
• /files — lists changed/referenced files with quick-look support

He also replaced MCP servers entirely with CLI-based skills. Browser automation via Chrome DevTools Protocol (CDP is how tools like Puppeteer talk to Chrome — Vercel’s Agent Browser takes the same approach, reducing token usage by 93% compared to Playwright MCP). Changelog management. Commit tooling. All implemented as skills the agent reads on demand rather than tools loaded at startup.

The pattern works because skills are just files. Write a SKILL.md, maybe a script, and you’ve extended the agent. No plugin API to learn, no registration system, no restart required.

The Taste Problem

I wrote about the taste problem — how vibe-coded contributions often lack the judgment that makes software coherent. mom’s self-extension could easily become a mess of poorly-designed skills.

The mitigation is structure. Skills have a defined format. The SKILL.md requires you to think about when the skill triggers (frontmatter) and how it executes (documentation and scripts). mom can extend itself, but the extension points are constrained.

This doesn’t guarantee good skills. But it raises the floor. A bad skill is at least a bad skill with proper metadata and documentation, not a random script mom decided to run.

The Philosophy

Traditional software is rigid. You ship features, users consume them, changes require releases. mom treats code as clay — reshaping at runtime based on what’s needed.

This isn’t new. Emacs has been self-extending for decades. But mom applies the philosophy to AI agents: the agent itself decides what capabilities to add, implements them, and uses them.

The question is whether you trust the agent’s judgment. mom requires that trust. In return, it handles the tedium of environment setup and lets you focus on what you actually want to do.

What Makes This Work

Self-managing agents aren’t magic. They’re just agents with three things: good defaults, a workspace they’re allowed to modify, and constraints that channel the chaos.

The skill format is the key constraint. Without structure, mom would create a mess of one-off scripts with no documentation and inconsistent interfaces. The SKILL.md requirement forces organization. The frontmatter forces you to think about discovery. The documentation forces you to think about usage.

The pattern is malleable code within rigid boundaries. The agent can write whatever skills it needs, but those skills must follow a format. The agent can install whatever tools it wants, but those changes are logged and reproducible. The agent can accumulate credentials, but they live in a known location with a known security model.

This is the opposite of the “AI will figure it out” approach. It’s more like: AI will figure it out, within guardrails that keep the figuring-out coherent.

BM25 is where search starts. Understanding it explains why some queries work beautifully and others return garbage.

The Grep Problem

Let’s start with the simplest possible search: grep. You have files, you want to find which ones contain a word. Grep scans through them and returns matches.

This works when you remember exactly what you’re looking for. If I know I wrote “ECONNREFUSED” in some error handling code, grep finds it instantly.

But most searches aren’t like that. I vaguely remember a conversation about authentication. Did I call it “auth”? “Authentication”? “Login flow”? “Identity verification”? Grep needs me to guess right. If I guess wrong, I get nothing.

Even when grep finds matches, they’re unsorted. A file that mentions “authentication” once in passing ranks the same as a file entirely about authentication. That’s not helpful when you have hundreds of matches.

BM25 solves the ranking problem. It scores results by relevance using surprisingly simple intuitions about what makes a document match a query well.

Rare Words Matter More

Imagine searching for “authentication error handling” across your notes.

The word “the” probably appears in every document. Finding “the” tells you nothing - it doesn’t help distinguish relevant documents from irrelevant ones.

The word “authentication” appears in maybe 5% of your documents. Finding it is meaningful - this document is probably about authentication.

The word “ECONNREFUSED” appears in maybe 0.1% of your documents. Finding it is very meaningful - this document almost certainly discusses that specific error.

BM25 formalizes this intuition. It weights rare terms higher and common terms lower. The technical name is “inverse document frequency” - the less frequently a term appears across all documents, the more it matters when it does appear.

This is why search engines handle stopwords (the, a, an, is, are) gracefully. They’re not removed or ignored - they just contribute almost nothing to the ranking because they appear everywhere.

Repetition Has Limits

If a document mentions “authentication” once, that’s a signal. If it mentions “authentication” fifty times, that’s a stronger signal. But is it fifty times stronger?

You might think so, but no. A document that says “authentication” fifty times isn’t necessarily more relevant than one that says it ten times. At some point, you’re just being repetitive.

BM25 handles this with “term frequency saturation.” The first few occurrences of a word boost relevance significantly. Additional occurrences help less and less. Eventually, more repetition barely moves the needle.

This prevents keyword stuffing from gaming the rankings. A document that artificially repeats search terms won’t dominate just because it has high raw counts.

Document Length Matters

Consider two documents:

• Document A: 200 words, mentions “authentication” 5 times
• Document B: 10,000 words, mentions “authentication” 5 times

Which is more relevant to a search for “authentication”?

Probably Document A. It’s short and 2.5% of it is about authentication. Document B is long and authentication is just 0.05% of the content - probably a passing mention in a larger piece.

BM25 normalizes for document length. A match in a short document counts for more than the same match in a long document. This prevents lengthy documents from dominating results just because they contain more words.

The Formula You Don’t Need to Memorize

BM25 combines these three intuitions into a single relevance score:

1. Rare terms get more weight (inverse document frequency)
2. Repetition helps, but with diminishing returns (saturating term frequency)
3. Matches in shorter documents count more (length normalization)

The actual formula has tunable constants, but the intuitions are what matter. BM25 asks: “How much does finding these specific words in this specific document tell me about relevance?”

A document mentioning rare search terms multiple times without being padded with filler content scores well. A document barely touching on common words in a sea of unrelated text scores poorly.

Why This Still Matters

BM25 was published in 1994. In tech years, that’s ancient. Why is it still foundational?

Because it’s fast and it works.

BM25 doesn’t require machine learning. It doesn’t need GPUs or training data. You can implement it with basic data structures and run it on any hardware. SQLite’s FTS5 (Full-Text Search 5) extension implements BM25, which means any application with SQLite gets competent search almost for free.

For precise searches - error messages, function names, specific technical terms - BM25 is hard to beat. It does exactly what you want: finding documents that contain the words you typed, ranked by how relevant those matches seem.

Where BM25 Falls Short

BM25 has a fundamental limitation: it only understands words, not meaning.

If you search for “authentication,” BM25 will not find documents that only use the word “login.” To BM25, those are completely different strings. It has no concept of synonyms, related concepts, or semantic similarity.

BM25 fails when:

• You don’t remember the exact terminology (“auth” vs “authentication”)
• The document uses different words for the same concept (“credentials” vs “password”)
• You’re searching for a concept rather than specific words (“how users prove their identity”)

BM25 also can’t handle typos. Search for “authentcation” (missing an ‘i’) and you’ll get nothing, even if you have dozens of relevant documents.

These limitations are fundamental to how BM25 works. It compares character sequences, not meaning. For thirty years, this was an acceptable tradeoff - the speed and simplicity were worth the occasional missed result.

Now there are other options. Vector search understands meaning rather than just words. The old algorithm isn’t obsolete; it’s just no longer alone.

The Takeaway

BM25 embodies a few simple truths about text search:

• Rare words are more informative than common words
• Repetition matters, but not linearly
• Shorter documents with matches are probably more focused

These intuitions are timeless. They applied in 1994 and they apply today. Understanding BM25 helps you understand why search behaves the way it does - why some queries return great results and others miss obvious matches.

When a query fails, it’s usually because you’re asking for meaning and BM25 only knows words. That’s not a flaw in the algorithm - it’s a limitation of the approach. One that newer techniques address, but never fully replace. Sometimes you really do want exact word matching, and a thirty-year-old formula will be there when you do.

The result? Four tools. A system prompt under 1,000 tokens. No MCP. No plugin ecosystem. It’s YAGNI applied to agent architecture.

The Spaceship Problem

Mario built Pi because existing coding agents became, in his words, “a spaceship with 80% of functionality I have no use for.” Claude Code, Cursor, Windsurf — they all ship with dozens of specialized tools, elaborate planning modes, and pre-built integrations. The assumption is that more capabilities mean more power.

Pi bets the opposite. Constraints are liberating. They’re architecture.

Four Tools

The entire foundation is four tools:

• read — examine files with configurable line limits
• write — create files or complete overwrites
• edit — surgical text replacement via old/new string pairs
• bash — command execution with no guardrails

That’s it. No glob, no grep, no specialized search. If you want to find something, you bash a find or rg command. If you want to understand a codebase, you read files directly.

The tool definitions are minimal:

{
  name: "read",
  description: "Read file contents",
  parameters: {
    path: { type: "string", description: "File path" },
    limit: { type: "number", description: "Max lines to read" }
  }
}

No elaborate schemas. No nested options. The model figures out how to use read because it’s read millions of examples of file reading in training. The tool just exposes the capability.

The reasoning: frontier models have been RL-trained extensively on coding tasks. They inherently understand what a coding agent is. You don’t need thousands of tokens of system prompt explaining how to be helpful. The model already knows.

The System Prompt

Most agent harnesses ship with massive system prompts. Claude Code’s runs thousands of tokens — tool descriptions, safety guidelines, behavioral rules, edge case handling, formatting instructions.

Pi’s system prompt is under 1,000 tokens. Here’s roughly what’s in it:

• Tool descriptions (~400 tokens for all four tools)
• Working directory and environment info (~100 tokens)
• Project context file reference (AGENTS.md) (~50 tokens)
• Basic behavioral guidelines (~200 tokens)

No elaborate persona. No safety theater. No “you are a helpful assistant who…” preamble. The model knows it’s a coding agent because it’s being asked to code.

This isn’t minimalism for its own sake. Every token in your system prompt is a token that isn’t available for actual work. If you burn 5,000 tokens on instructions, you’ve shrunk your effective context window by 5,000 tokens. Do that across a long session with compaction, and the overhead compounds.

Armin Ronacher, who switched to Pi after trying most alternatives, puts it directly: the system prompt is “the shortest of any agent.” He runs Pi almost exclusively now, replacing Claude Code, Cursor, and custom setups. The minimal prompt isn’t a limitation — it’s why the agent stays focused.

YOLO by Default

Here’s where Pi gets controversial: no permission checks.

Most coding agents implement elaborate approval flows. “The agent wants to run rm -rf. Allow?” The theory is safety. The practice is theater.

Once you give an agent code execution, artificial restrictions are futile. A sufficiently capable model will find ways around guardrails — or you’ll click “approve” a hundred times and stop reading what you’re approving. Either way, the safety is illusory.

Pi acknowledges this. Unrestricted filesystem access. Any command without permission checks. The “sandbox” is your judgment about what you ask it to do, not a popup you’ll learn to ignore.

Against MCP

I’ve written about MCP and context bloat before. Pi takes a strong stance: no MCP support at all.

The argument is architectural. MCP servers load tools into your context at startup. Take Playwright MCP for browser automation:

• 21 tools loaded at startup
• 13,700 tokens in system prompt
• Present whether you need browser automation or not

That’s 13.7k tokens of your context window gone before you’ve typed anything. Add a few more MCP servers and you’re burning 30-40k tokens on tool descriptions alone.

Mario’s alternative is embarrassingly simple: CLI tools with README documentation.

• Shell script: playwright-browse.sh
• README: ~225 tokens explaining usage
• Loaded only when the agent reads the README

When the agent needs browser automation, it reads the README, understands the interface, and calls the CLI via bash. When it doesn’t need browser automation, those 225 tokens don’t exist in context.

The math is stark. A session that might use browser automation once:

• MCP approach: 13,700 tokens for the entire session
• CLI approach: 225 tokens when actually needed, 0 otherwise

The difference isn’t just token count. It’s progressive disclosure. MCP front-loads everything. CLI tools load on demand. For agents that do many different things across a session, the savings compound.

No Built-in Planning

Pi doesn’t have a planning mode. No to-do lists. No automatic task decomposition.

The reasoning: planning modes create model state-management overhead. The agent has to track what it planned, what it completed, what changed. That’s cognitive load that could go toward the actual work.

If you need a plan, write it to a file. The file persists across sessions. You can edit it. The agent can read it. But the state lives in the filesystem, not in some hidden UI abstraction.

This applies to sub-agents too. Pi doesn’t have built-in orchestration. If you want to spawn a sub-agent, you bash it — spawn another Pi instance with a specific task. The parent sees the subprocess output. The delegation is visible, not hidden.

Benchmark Results

You’d expect a minimal agent to underperform feature-rich alternatives. The benchmarks say otherwise.

Terminal-Bench 2.0 tested Pi with Claude Opus 4.5 against specialized harnesses — Codex, Cursor, Windsurf. Pi held its own despite having a fraction of the tooling.

More interesting: the benchmark team’s own Terminus agent provides only tmux access — even more minimal than Pi. It also competed effectively. The conclusion: sophisticated tooling might matter less than we assumed.

The Philosophy

Mario summarizes it as five principles:

1. Models self-supervise coding tasks — frontier models need minimal prompting about their role
2. Sub-agents risk coherence — parallel feature implementation via spawned agents produces fragmented codebases
3. File-based state beats UI state — markdown artifacts outlast sessions and enable cross-session collaboration
4. Observability over orchestration — hidden delegations destroy debuggability
5. Token efficiency matters — context discipline beats feature maximalism

This isn’t advice for all agents. It’s a specific bet: the best coding agent for experienced developers is one that gets out of the way.

What Pi Powers

Pi isn’t just a solo project. It’s what powers OpenClaw — the multi-channel assistant that’s been generating both excitement and concern lately. OpenClaw adds the infrastructure layer: WhatsApp, Telegram, Slack, Discord, Signal, iMessage, Matrix. But the brain is Pi.

It’s also what powers mom (Master Of Mischief), Mario’s self-managing Slack bot that installs its own tools and writes its own skills. The minimal foundation turns out to be surprisingly extensible.

Feeling the Difference

I spent half an hour with Pi today and I’m still processing what happened. Understanding something intellectually is different from feeling it.

The agent felt snappy running a local model. Not “acceptable for local.” Actually snappy.

GLM-4.7-Flash-Q5-200K running locally on my Framework Desktop (Strix Halo, Ryzen AI MAX+ 395). No cloud API. No internet required. Just a quantized model doing actual work.

I pointed it at my Nix dotfiles repo — flakes, home-manager configs, NixOS hosts, custom packages. Within seconds, it read flake.nix, ran ls -la, read the README, listed the hosts directory. Then delivered a comprehensive breakdown: multi-host management, configuration layers, custom packages, secrets management. It understood the architecture.

No verbose preamble. No elaborate planning. Just read, understand, explain.

Then I pushed it: “I’d love to have a plan mode for yourself, can you help me add that?”

The agent explored the Nix store where Pi lives, found the packages/coding-agent/docs/ directory, read extensions.md to understand the extension API. Then it one-shot created a complete plan mode extension:

import type { ExtensionAPI } from "@mariozechner/pi-coding-agent";

export default function (pi: ExtensionAPI) {
  let inPlanMode = false;
  let planQueue: Array<{ toolName: string; input: any }> = [];

  pi.registerCommand("plan", {
    description: "Toggle plan mode - preview actions before execution",
    handler: async (args, ctx) => {
      // ... complete implementation
    },
  });

  pi.on("tool_call", async (event, ctx) => {
    if (inPlanMode) {
      ctx.ui.notify(`[PLAN] Would execute: ${event.toolName}`, "info");
      return { block: true, reason: "Plan mode: blocked execution" };
    }
  });
}

Command registration, event interception, state management, UI feedback, documentation, a demo script. It figured out from the docs that Pi uses @mariozechner/pi-coding-agent for types and @sinclair/typebox for schemas. It understood the event system, the context object, the UI API.

A local quantized model one-shot created a working extension for an agent framework it had never seen before, by reading the source code and documentation.

And here’s the kicker: I didn’t even restart the agent. /reload, and the new plan mode was live. This is malleable software in practice — the system reshaping itself in response to a conversation. The gap between “I wish this existed” and “now it does” collapsed to minutes.

The minimal system prompt matters more than I understood. With a local model, you feel every token. There’s no massive datacenter hiding the latency. Less system prompt means faster first response, more room for context, more of the model’s attention on your actual task.

The capability floor with a minimal agent is higher than expected. The agent’s performance is bottlenecked by the model, not the harness. A minimal harness lets the model show its actual capability. A bloated harness adds overhead that masks it.

The Trade-offs

Pi isn’t for everyone. The YOLO approach requires trust — in the model, in yourself, in your backups. The minimal tooling means you’re often writing bash one-liners that a specialized tool would handle automatically.

But the trade-off is clarity. When something goes wrong, you know exactly what happened. There’s no hidden orchestration layer, no mysterious context injection, no tools you forgot were loaded.

I’m sold. Not just intellectually — I was already there after reading about it. I’m sold experientially. This is what the future feels like: not more features, not more tokens, not more complexity. Less. Less prompt. Less scaffolding. Less overhead. More actual capability.

Four tools. A tiny prompt. A model that knows what it’s doing. That’s enough.

I was shocked. This wasn’t a rejection because my code was bad or my approach was wrong. The door was simply closed to everyone. The maintainer offered to re-implement the fix themselves, which felt strange - they’d essentially take my idea and write it again from scratch. I asked them to at least document the policy in the README so other people don’t waste their time discovering this the hard way.

That experience stuck with me, and then I started seeing the pattern everywhere.

The Pattern

Days later I saw tweets about major GitHub repositories stopping external contributions. The reason: unprecedented levels of AI-generated slop flooding their pull request queues.

tldraw announced they would automatically close pull requests from external contributors going forward:

“Like many other open-source projects on GitHub, we’ve recently seen a significant increase in contributions generated entirely by AI tools.”

The submissions suffer from incomplete context and fundamental misunderstandings of how the codebase works. Worse, the people submitting them rarely stick around for the back-and-forth that any real contribution requires. An open pull request represents a commitment from maintainers to review it seriously, to engage with the contributor, to shepherd the change through to completion. When the signal-to-noise ratio collapses, that commitment becomes impossible to sustain.

I get why maintainers are doing this. Open source maintenance is exhausting even under the best circumstances. Every PR needs careful review, testing, communication, and often multiple rounds of revision. When most of what lands in your queue is low-quality AI slop from people who won’t be there for the follow-up, the cost-benefit math completely breaks down. But something valuable is being lost in the process.

The Ladder We Climbed

Open source contributions gave me a lot of exposure early in my career. It’s how I got my name out there, learned how real projects work, and connected with people who were better than me.

The process was straightforward but powerful. You find a project you actually use. You hit a bug that annoys you. You dig into the codebase, figure out what’s going wrong, and fix it. You submit a PR. The maintainer reviews it, maybe asks for changes, and you go back and forth until the code is ready. Through that process you learn how real codebases are structured, how to communicate with other developers asynchronously, how to write code that works with existing patterns rather than against them.

That door is closing now, and I worry about what happens to the next generation of developers.

When I mentored junior engineers, my advice was always the same: contribute to open source. Pick projects you genuinely care about and fix problems that annoy you. The learning that happens through that process is something no tutorial or bootcamp can replicate. You develop an intuition for how software systems work at scale, how to navigate unfamiliar code, how to communicate technical ideas clearly to people you’ve never met.

If that path is no longer available, we’re making things significantly harder for people trying to enter this field. The ladder we climbed is being pulled up behind us.

The Vibe Coding Problem

Even so, part of me understands why this is happening.

The tools make it dangerously easy. You can point Claude at a repository, describe a bug vaguely, and get a pull request in minutes. You don’t need to understand the codebase architecture. You don’t need to diagnose the root cause of the problem. You just paste the output and click submit. The friction that used to filter for genuine engagement has been almost entirely removed.

I’ve written before about vibe coding and what it means when code becomes disposable. Steve Yegge built Beads - 225,000 lines of Go - without ever looking at the code himself. It shipped, it delivered real value to real people, but the technical debt ran deep. That approach works when you own the whole thing and can iterate freely. It falls apart when you’re contributing to someone else’s project.

The same dynamic is now playing out across open source. People submit PRs they don’t understand, to fix problems they haven’t actually diagnosed, using code they couldn’t maintain if their lives depended on it. Sometimes the PR even works on the surface. But when it doesn’t - when it breaks something subtle, when it needs iteration, when the maintainer asks a clarifying question - there’s nobody home. The person who submitted it has already moved on to their next AI-assisted drive-by.

You can’t check your brain at the door when contributing to open source. If you don’t understand what you’re submitting, you’re not helping - you’re just shifting the burden onto maintainers who now have to figure out whether your contribution is sound, often with zero assistance from you. That’s not collaboration. That’s dumping work on volunteers.

There’s an irony here. Claude Code itself is reportedly 100% vibe coded - built entirely through AI-assisted development. And it works, it’s actually a good product. The approach seems to be: iterate until it works, worry less about the intermediate steps.

I’ve been wrestling with this tension myself. In my beads post I admitted I was trying to let go of caring so much about code quality - treating code as disposable rather than precious. But I keep coming back to the same conclusion: I’m not ready to completely stop caring about what’s being generated. The leverage comes from collaboration with the AI, not from blind delegation. When you submit something to an open source project, you’re asking maintainers to trust your judgment. If you’ve outsourced that judgment entirely to an AI you didn’t supervise, you’re wasting everyone’s time.

The Fun Part

I’m having more fun coding now than I have in years.

AI has unlocked experimentation in ways that weren’t practical before. Complex algorithms and data structures that would have taken weeks to implement correctly can now be explored in an afternoon. I recently wanted to understand CRDTs and Lamport clocks better, so I started building Sterna, my own experiment at a beads replacement. The idea might be overkill, might even be stupid, but it was genuinely fun to build and the thing actually works.

I think that’s what antirez is getting at. The creator of Redis recently wrote about his own experience with AI-assisted development. In a single week of prompting and inspecting code, he modified his linenoise library to support UTF-8, fixed transient Redis test failures, created a pure C library for BERT inference in 700 lines, and reproduced weeks of Redis Streams work in about 20 minutes.

“It is now clear that for most projects, writing the code yourself is no longer sensible, if not to have fun.”

And on the passion for building:

“What was the fire inside you, when you coded till night to see your project working? It was building. And now you can build more and better, if you find your way to use AI effectively. The fun is still there, untouched.”

I’m more bullish than antirez on this. The productivity gains are real and I use these tools constantly. I’ve written about staying in the “smart zone” and managing context effectively. When you know what you’re doing, the gains are insane.

But I diverge from antirez on one point: I still care about understanding the output. He says “writing code is no longer needed for the most part.” I’d frame it differently - writing code is changing, not disappearing. The leverage comes from collaboration, not from turning your brain off and letting the AI drive.

The Taste Problem

I’m not claiming my taste in code is better than anyone else’s. But I do have opinions - about how code should look, how systems should be structured, when a solution is elegant versus when it’s a hack waiting to bite you. Those opinions came from 20+ years of writing code the hard way, making mistakes, and learning what actually works in production.

That experience is what lets me evaluate AI output meaningfully. How do you develop that taste if you never write code yourself? If you never struggle through a problem manually? If you never have a PR rejected and have to figure out why?

Taste requires exposure and iteration. You develop heuristics through experience - writing bad code, having it rejected, learning what actually works in production versus what just looks plausible. If junior developers can’t get their PRs accepted anywhere, if they’re just prompting AI and submitting the output without understanding it, where’s the feedback loop that builds judgment?

No Answers

I don’t have solutions. I’m concerned and working through it myself.

Open source contribution used to be a door into this industry. It was how you proved yourself when you had no professional experience. It was how you learned from people better than you by actually engaging with their code and their feedback. It was how communities formed around shared problems and collective ownership.

That model assumed contributors understood their contributions. It assumed maintainers could trust that someone would be there to iterate when issues came up. It assumed the cost of review was worth the benefit of community input. AI broke those assumptions - not because AI is bad, but because it removed the friction that used to filter for genuine engagement. When contributing is as easy as “prompt and submit,” you get a flood of contributions from people who aren’t really there.

The maintainers closing their gates are responding rationally. They’re protecting their time and their projects and their sanity. I don’t blame them at all.

But we need to think about what we’re losing. Maybe the answer is new contribution models with more structured onboarding, proof-of-understanding requirements, or tiered access based on track record. Maybe the answer is accepting that open source contribution as we knew it was a historical anomaly, enabled by specific conditions that no longer exist. Or maybe we need entirely new ways for people to build taste and prove competence before they contribute - ways that work in a world where AI can generate plausible-looking code on demand.

I don’t know. I’m still working it out. But watching those gates close feels like watching something important slip away. As antirez puts it: “Skipping AI is not going to help you or your career… Test these new tools, with care, with weeks of work, not in a five minutes test where you can just reinforce your own beliefs.” The fire he talks about - the passion for building - that’s still there. The question is how we pass it to the next generation when the paths we took are being walled off.