Why the assistant is a graph traversal, not a chat.

The first version of the assistant was a chat box. You typed something, a model returned tokens, sometimes those tokens included a tool call, and the tool call hit one of our internal APIs. It demoed beautifully. It also broke in roughly six ways the moment it met a real workspace with real data.

The deeper problem was not the model. The deeper problem was the framing. A chat assistant assumes the user is the conductor and the assistant is a clever scribe. In an enterprise workspace, the user usually does not want to conduct. They want the work to happen. The assistant needs to start from the work and walk outward.

The four failure modes of "chat plus tools"

We kept running into the same shape of failure:

1. Context bloat. To answer a question about an account, the model wanted the contacts, the open tasks, the recent docs, the meeting notes, and the contract. We were paying for tens of thousands of input tokens per turn, and the relevant slice was tiny.
2. Stale retrieval. A vector index over the corpus was always one minute behind reality. A user would close a task and ask the assistant "what's left for the Acme rollout?", and the answer still included the closed task.
3. No path back. The model would propose an action like "send the recap" and we had no way to show the user what would happen. The right preview required walking the same data the model walked.
4. Permissions leakage. Retrieval indexes did not enforce ACLs at query time. We added a post-filter, and the model started hallucinating answers from the documents we filtered out.

Around month four, the reframe finally landed: this is a graph problem dressed up as a chat problem. I stopped trying to make chat work and started writing down what the assistant actually does on a workspace.

What it actually does

At the layer below the chat surface, every assistant action is one of three primitives:

• Locate: find a node in the work graph that matches a description in natural language.
• Expand: from a known node, walk one or more edges to fetch related context.
• Act: create, edit, or delete a node or edge, transactionally and with audit.

That's it. The chat surface is a presentation layer. The state machine underneath is a graph walker with three operations, a budget, and a permission predicate.

How the walker is actually implemented

The walker is a small state machine that wraps a tool-using model. The model only sees the current node, a budget of how many more steps it can take, and a set of available operations on that node. It does not see the entire workspace context.

Pseudo-code, cleaned up for readability:

def walk(prompt, start, subject, budget=12):
    state = State(prompt=prompt, focus=start, budget=budget)
    while state.budget > 0 and not state.done:
        ops = ops_for(state.focus, subject)
        choice = model.pick(state, ops)
        if choice.kind == "locate":
            state.focus = graph.find(choice.q, subject)
        elif choice.kind == "expand":
            state.facts.append(graph.expand(state.focus, choice.label, subject))
        elif choice.kind == "act":
            state.proposals.append(choice.action)
            state.done = True
        state.budget -= 1
    return state

Two things are doing a lot of work here. ops_for(focus, subject) returns only the operations the requesting subject is allowed to perform on the focus node. The model is never asked to choose an operation it could not be permitted to take. And graph.expand is the same call the product surface makes: same indexes, same query planner, same ACL predicate. There is no shadow retrieval system.

What we tried before this stuck

We took three swings at this before the walker. The history is worth keeping for anyone trying to build a similar system.

Attempt one: function-calling chat

A loop of "model picks a function, function runs, result appended to context, repeat." Familiar shape. Two problems. First, context grew without bound, because the model would re-fetch the same data when it forgot what it had. Second, the model would call functions in orders that didn't make sense, because nothing constrained the set of allowable calls at each step.

Attempt two: ReAct with a flat tool registry

We tried a more disciplined version of attempt one, with reasoning traces and a controlled tool registry. It was better, but the registry was still flat: every tool was available every step, and the model would routinely pick a global tool when a local one would do.

Attempt three: a planner-executor split

A second model would produce a plan up front, and an executor would carry it out. The planner hallucinated steps. Whenever a step failed, the executor had no recourse, because the plan was already committed. Composability across the modules also fell apart, because the planner could not see the per-module operation surface.

The walker won because it made the model's job small at every step. Pick one operation on one node, from a tightly-scoped menu. The model is very good at that. It is much less good at "produce a plan to send a recap, given the entire workspace."

The chat surface is still there

From a user's perspective, the assistant still looks like a chat. They type "send the recap" and they see a recap drafted. The graph traversal is hidden under the surface. The reason we built the surface this way, and the reason we kept it, is the same: chat is the lowest-friction way to express intent.

What changed is what happens between the user's intent and the model's first token. The chat string is translated into a starting node. From there, the walker takes over.

What we gained

The improvements that mattered most were structural rather than a matter of tuning. Input tokens per turn fall sharply, because the walker only ever loads the current node and the facts it has gathered, not the whole workspace. Time to first useful action drops, because there is far less to read before acting. And incorrect actions get rarer, because at each step the model chooses one operation from a small, permitted menu instead of planning over everything at once. The biggest qualitative win is that you can now point at a specific node ("this account") and the assistant starts there. The walker treats the context window as a budget instead of a backpack.

What we lost

The walker is not as good at open-ended brainstorming. Asking it "what should I think about for Q3?" returns a competent but boring answer, because the model is constrained to operations on real nodes. We added a small "blue-sky" mode that drops the constraints; it's clearly marked, it doesn't take actions, and the recall behavior is more like a plain chat. It is there for the open-ended moments and used sparingly, which is about what I would expect.

What we'd do differently

Two things. First, the budget logic was too simple at the start. We modeled it as a flat counter, and the model would burn budget on cheap expansions and run out before it could act. The current version weights operations by cost and stops early when the marginal information per step is below a threshold. Second, we waited too long to add a dry-run preview. Once we did, the reaction flipped: people will trust an action they can preview. They will not trust an action they cannot.

What's next

We want the walker to learn from its own trajectories. We have the data: every step is recorded, every outcome is observable, every action has a downstream signal (was the email opened, was the task completed, was the doc edited again). We don't yet use that signal to tune the operation-selection model. That's the work for the next six months.