Orrery, a Knowledge Graph That Adapts to Its Corpus

We have been thinking about what an engineering organization looks like in 2026 when much of the work is done by agents. On the “performing work” side we have a suite of projects (Tracker, dippin-lang, Barnstormer) aimed at increasing the velocity at which any one person can get things done. But velocity is only half of an agentic org. The other half is memory. How do agents build on the work and learnings of teammates, human and agent alike, so the organization gets smarter as it works instead of rediscovering everything from scratch?

Orrery is our attempt at an answer. When we asked ourselves what a knowledge graph should look like in 2026, we landed on two convictions.

1. The graph should adapt to its corpus. Not just grow as documents are added, but get deeper, developing more specialized understanding of the domains you actually feed it. A graph that only knows generic entities can only give generic answers, so as your questions get more specific, retrieval quality quietly falls behind.
2. The human interface and the agent interface are separate design spaces. They sit on the same substrate, but you can (and should) optimize them independently. An interface built to serve both tends to serve neither, and agents genuinely do not care what things look like.

The rest of this post is about how those two ideas play out. How the graph refines its own extraction over time, what it takes to maintain a living graph, and why the humans get a galaxy to look at while the agents get an API to work through.

Doing the Work Ahead of Time

Most of the recent graph work rests on one belief: the useful work should happen when documents arrive, not when questions are asked. Standard RAG re-derives connections on every query. If instead you compile connections once at ingestion, extracting entities, linking documents, and building structure, then every downstream agent and teammate inherits those learnings for free.

The approaches differ in how much of that work they do when documents arrive, and how much they save for query time. Microsoft’s GraphRAG derives the graph and writes community summaries proactively, which enables answering higher-order questions about a whole corpus but is expensive enough that Microsoft themselves swung back with LazyGraphRAG, which does fast, cheap entity extraction up front and defers the heavier reasoning to retrieval time. The two techniques bracket the effort spectrum, and both treat the ingestion pipeline as something you design once and hold fixed.

Karpathy’s LLM wiki is the closest philosophical neighbor to what we are doing. His pattern has an LLM incrementally build and maintain a persistent, interlinked wiki as sources arrive. The knowledge gets compiled once and kept current, with the human curating sources and directing the sessions. We share the underlying belief completely. The place we diverge is the format itself. A wiki couples the storage layer to a human-shaped rendering. Pages and links are a graph. Karpathy even points people at Obsidian’s graph view to see the shape of theirs. But the structure itself is built for a human reader, who navigates prose pages through an index. That is a fine constraint for a personal knowledge base. It is less obvious that a team of agents should have to operate through the same lens, which is a thread we will pull on later in this post.

At the other extreme sit the industrial-scale graphs, like Pinterest’s PinSage or the LinkedIn and Facebook social graphs. These are enormous, but sterile. The entity types and edge types are fixed in advance, the interactions are known, and everything gets fit into that structure. Pinterest ran a good chunk of their recommendation stack for years on a graph of pins and boards, and that was the whole vocabulary.

So the field gave us three options. Graphs that are rich but frozen at build time, graphs shaped around a human reader, or graphs that scale but only by fixing their structure in advance. What we wanted was something that stays open-ended, with free-form extraction so unexpected connections can develop, while adapting its own structure to whatever corpus it is given. That second idea comes from work we did earlier this year on self-honing workflows with Simmer, and it is the piece that makes Orrery different from a pipeline you configure once.

Refining Entities Adaptively

Here is the failure mode we designed against. Say you start a knowledge graph around startup founding and VC rounds. Your extraction naturally pulls entities about investors, funding, and pitching. Then the graph starts absorbing ops documents, or engineering discussions, or ongoing business. If your extraction does not adapt, every new domain gets squeezed through the old entity vocabulary, and the graph’s understanding stays frozen at whatever you configured on day one. That failure mode is what the adaptive half of Orrery exists to prevent.

In Orrery, documents are classified into a hierarchical domain taxonomy that builds itself incrementally, and every document gets immediate entity extraction from a general-purpose spec, so the graph is queryable from the first upload. But when a conceptual domain accumulates enough content, it triggers the interesting half of the system. Orrery investigates that domain and builds a custom extraction spec for it, determining what specialized entities would give better insight into that particular area than the general superset does. The specificity of the graph adapts to the user and their data. Feed it more of something, and it learns to see that something in finer detail.

Two examples from our live instance. The general spec extracts at the level of software engineering and architecture topics. When our budgeting and finance domain hit critical mass, its refined spec started pulling entities the general vocabulary had no concept of, things like compliance standards, funding stages, financial metrics, and the specific SaaS services we pay for. When a software architecture domain around command and event state hit the same threshold, its spec went the opposite direction, a level below architecture into the code itself, extracting individual fields, functions, and error conditions. The general spec can tell you a repo does event sourcing. The refined spec can tell you which function replays events and that duplicate event ids during replay are a known failure mode.

The refinement mechanism is a descendant of Simmer, a feedback loop mixing traditional metrics with LLM judgment on open-ended criteria, using a small board of judges rather than a single one for more varied critique. We run it in two stages. The first stage determines what the appropriate specialized entities for a domain should be, effectively building a golden dataset. The second stage hones an extraction prompt against that dataset, optimizing for a prompt that a small model can execute reliably, where the golden set makes clean traditional metrics like taxonomy coverage and precision/recall possible.

From Agentic Loop to Workflow

Why two stages instead of one big agentic loop? We tried the loop first. It worked, in the sense that a frontier-model generator and judge iterating on entity specs produces good results, but each run took 60+ minutes and cost $10-15 in agent time. Worse, the judge had to critique two different things at once (are these the right entities? is this prompt extracting them well?), and that push-pull feedback let the agents waffle around the problem space across the handful of iterations an agentic loop realistically gets.

The conversion strategy was to run the agentic version repeatedly across varied domains and isolate the common behavior from the traces. What the agents were actually doing decomposed cleanly into those two stages, so we rebuilt it as focused LLM calls with the information each stage needs front-loaded instead of foraged for. Each stage now has one clear objective, the feedback is focused, and the problem became tractable for much smaller models. The workflow version runs in 10-20 minutes at around $0.50, roughly a 30x cost reduction, and it works well enough on small local models (Gemma-class, ~30B parameters) that the entire Orrery service, several thousand documents and 100+ GitHub repos with continuous normalization and maintenance, currently runs on local models. That general pattern (using agents to explore a problem space, then compiling their traces into workflows) is worth a dedicated post.

Adding to and Maintaining a Graph Over Time

Something much less common in the literature is how to maintain an active graph. Most research builds a dataset, computes on it, and stops, while the industrial graphs avoid the problem by fixing their structure in advance. A graph doing free-form extraction does not get to avoid it.

The first risk of extracting everything open-endedly is an explosion of entities, many of which overlap or are variants of one another, which makes retrieval harder than it needs to be. Current versions of Orrery spin up workers when the system is idle to scan for normalization candidates. We are deliberately targeting the lowest-hanging fruit first, things like “Harper Reed” vs. “harperreed” spacing and capitalization variants or “2389” vs. “twenty three eighty nine” disambiguation, and staying conservative about merging higher-order concepts. It is not obvious where the break-even point is for collapsing conceptual near-duplicates, so for now we keep the worst offenders from overpopulating the graph and can layer on more aggressive normalization as we learn.

The second risk is that some portion of nodes will simply be wrong. Rather than scheduling broad sweeps that re-fact-check the top slice of the graph on a regular cadence, which mostly re-verifies things that have not changed, Orrery lets agents and users submit issues against nodes, GitHub-style. A separate pipeline picks the issues up, compares the claims, and adjudicates. Quality maintenance concentrates on the parts of the graph that are actively used, which is where correctness actually matters. This is very much an ongoing area of study for us.

One more maintenance decision is more aesthetic than functional, but we care about it. Most knowledge graph visualizations change shape dramatically over time, because any update triggers a full layout recomputation. Watching your graph reorganize itself daily is like looking up and seeing the constellations rearranged every night. Technically fine, deeply off-putting. Orrery anchors its layout around the conceptual domains, gravitational-well style. Domains stay in place and everything else shifts around them, so the galaxy grows and evolves without ever losing its landmarks.

A Tale of Two Interfaces

Which brings us back to the second conviction. The substrate does not have to be human-shaped, and once you accept that, you get to design each interface for its actual audience, humans on one side and agents on the other.

For agents, Orrery exposes standard graph operations through APIs, including semantic and lexical search, node traversal, and local neighborhood exploration. Models turn out to be quite comfortable with a densely populated graph, arguably more comfortable than humans are, because it approximates the kind of over-connected associative structure they handle well. The same extraction and enrichment flows that build the graph power RAG tasks for our agents, giving data just enough structure that agents can build on each other’s work.

For humans, none of that is fun to look at. We wanted an interface that makes a person want to keep exploring, and the mental model that does that is stars and galaxies, which is where the name comes from. An orrery is a mechanical model of the heavens, a machine you build because watching the motion is the point. So the front end renders the graph as a galaxy, with domains as regions of space laid out semantically, documents and entities as stars, and the whole thing growing as the corpus grows. When a domain accumulates enough mass to trigger adaptive refinement, you can see it, the star-shaped cluster changing character as its custom entities come online.

My favorite part is watching the graph work. As agents query the graph, their searches trigger cascades of light, from the documents they hit, through those documents’ entities, up to the conceptual domains above them. The agents do not need this feature and do not know it exists. All of it is built to make an Orrery instance feel alive, and to show a human what work is being done and where, as it happens.

You could render this same substrate as a wiki, and for some teams that projection might be the right one. The point is not galaxy versus wiki. The point is that the rendering is a projection of the graph rather than the graph itself, so you are free to pick whichever projection serves the audience in front of it.

What Do You Do With a Graph?

Oddly, this is not a well-studied problem. Most graph research ends at building the graph and running retrieval benchmarks on it. There is very little on what an organization does with a living one, and even less on whether you can meta-learn on top of a graph and use those learnings for downstream tasks nobody benchmarks.

That is where our current experiments are. We are combining company documents, meeting transcripts, git repos, Claude Code sessions, and artifacts from our Tracker and dippin-lang flows into the graph, so agents can draw on work done by other team members. One early example is that ingesting Tracker runs lets agents dig into them and perform retros on the work that was done. In a small experiment, an agent given those retro documents while planning a new run reasoned through design decisions that a generic agent with only the spec library never engaged with, things like how to tier models for cost and where pipelines tend to fail. The point is not that the resulting plan was better. If you have infinite budget, throwing a frontier model at every node works fine, if overkill. The point is that the graph captured design decisions from past runs and applied them to future work without being told to. Different team members are strong at different things (some of us optimize aggressively for cost, others build more robust pipelines), and the graph becomes the medium through which those practices propagate instead of staying locked in individual heads.

The open questions from here are mostly about scale and governance. Do we run one graph or many? How do we review artifacts that agents write back into the graph? What does maintenance even mean at the scale of an entire engineering org? If we can get good at maintaining an organization’s knowledge graph at that level of activity, that seems worth doing. We will report back.

Try It

Orrery is open source, and runs fully locally if you want it to.

git clone https://github.com/2389-research/orrery.git
cd orrery
cp .env.example .env
docker compose up

Point it at Anthropic’s API, AWS Bedrock, or a local Ollama install. The README has the details.