Jev Plays Pokemon

TL;DR: I tried Jev as a per-step navigator inside a Pokemon Red agent and it added nothing over plain BFS. Promoted one level up to picking which movement policy we route under, with a trimmed down context, it works well: ~160 ms and ~$0.00002 per decision, cheap enough that routing an entire playthrough costs fractions of a cent.

This week Jev launched, and the main draws are very fast inference and low price, with the caveat that it specializes in choosing between 1 of N choices at a given time. Jev is a non-generative “System One” model: rather than producing text, it evaluates a structured state and returns a typed 1-of-N choice with calibrated probabilities, which is what makes the “1 of N” framing literal. The appealing part is that it could sit as a router or a classifier within a large system to help route requests or use specific models at a given point without needing a larger slower model, reaching for other zero-shot techniques, or building a dataset to finetune something like a BERT model.

Instead of something more business shaped I had decided unrelatedly to have an agent play through Pokemon Red to test a few other ideas, so I added Jev into this flow as a way to choose the policy by which we move around in the game while other models set the higher level goals of what we should be doing and where we should go.

At the moment I am in fairly early points of this experimentation where I am working out what the proper game state and action spaces should be as well as how to do long term planning correctly and manage the long term context within the system.

Three levels of decision making

High level: What Should We Do?

Since the characters in pokemon can adventure around quite a bit, at the highest level I use a GLM model to do planning about what the agents should be aiming to do at a given point in time and what the acceptance criteria for completion is. Basically I let it set quests on its own based on what is happening in game and what counts as completing them. Having this get set at the highest level means that whenever a sub level agent thinks that a quest is completed we can concretely check it against these acceptance criteria.

For instance the agents know that they need to fetch a particular item and set the acceptance criteria of being able to find this item in their inventory.

At this level there is a populated Orrery knowledge base with information about the early locations, pokemon types, and items. The agents at this level are allowed to send search queries to enrich their understanding and planning.

Middle layer: where do we go?

Middle layer is given game state information and attempts to set x,y coordinates that the agents should try and navigate towards and is backed by a DeepSeek 4.1 Flash model. Using PyBoy we read the game’s RAM directly and decode it ourselves (the loaded map’s tile and collision data, plus a small tile catalog we pulled from the pokered disassembly) so without needing the images we can give the bot a decent picture of its surroundings via an ASCII representation (walkable tiles, walls, grass, ledges, and doors). The goal of this model is mostly that knowing its high level quests, the current game state, and what it has been doing it needs to pick where to head to next.

The specific models here, GLM for planning and DeepSeek 4.1 Flash for the middle layer, are mostly arbitrary picks and nothing structural depends on them. Both are served by LunaRoute, our model provider (and good friends of ours).

Low Level Routing: Jev!

At this level we have where we should be going, and Jev decides how we should get there. Initially this was just BFS.

The first thing I tried was letting Jev pick the actual step-by-step direction toward the target each frame. But I was also handing it the BFS shortest-path suggestion as a hint, and a confident Jev conditioned on that basically agreed with raw BFS nearly every step (94% agreement in an A/B run), so at the per-step level it wasn’t really adding anything over just running BFS.

Re-reading the Jev docs (which stress handing it exactly the state a discrete decision depends on) pushed me to use it one level up instead: a deterministic weighted pathfinder does the actual routing, and Jev chooses which policy we route under. That let me add a few movement policies on top of BFS:

• shortest: plain BFS, the direct path (normal travel)
• health preservation: avoid spending steps on grass tiles to skip wild battles; Jev tends to pick this when the party is low on health
• farm exp: deliberately pace back and forth through grass to trigger battles and grind; Jev picks this when the party is healthy but below the level threshold it thinks it needs to beat the next gym leader

So while the game itself is a toy, the pattern isn’t: a fast calibrated model picking 1 of N policies while slower models handle the open ended planning is the shape of a lot of real systems.

How much context does Jev actually need?

One of the open questions I had when I folded Jev in was how much context it required. While testing the step-by-step version I gave it the same full game-state preview the middle layer gets, then A/B’d that against a minimal slice of just what the decision immediately depends on. The decisions came out identical either way (94% BFS agreement in both arms), but the full map context used 1,324 input tokens against 601 for the minimal one, more than double the cost for the same choices, and confidence was actually slightly worse (0.82 vs 0.87). The extra state wasn’t helping, it was just noise the model had to look past.

That lesson carried straight into the policy picker. It only gets handed the decision-relevant signals (party health, level versus the target it thinks it needs for the next gym, and whether grass is even nearby) rather than the whole game state. A nice side effect of the calibration is that confidence works as a smell test for how you’ve framed the decision, and a natural next step is to route on it: if Jev is torn between policies, escalate the choice up to the middle layer model.

In the example below you can see Jev choose a farming policy while the party is healthy, then fall back to a dodge-grass policy once health drops and it needs to make progress.

What it costs

Each policy decision is ~423 input tokens and ~160 ms (median 159 ms, range 146-540). At Jev’s $0.042/M input pricing with output free that works out to ~$0.00002 a call. In our loop Jev fires about once every 20 steps, roughly 1.3 calls a minute at our 25 steps/minute gameplay rate, so Jev’s share of an hour of play is about $0.0014. Even at a Doom-like 10 decisions a second, our 423 token state works out to ~$0.64/hour.

For comparison, the same tier of decision on the middle layer’s generative model came back at 418 ms median, and that was on a tiny 62 token prompt, so it understates the gap on real sized prompts, and it gets metered on output tokens besides. Jev’s job here is the routing decision, and the point is keeping that decision off the bigger models. Because it’s fast and output-free, making that call as often as the game needs it is essentially free. The loop’s 2.38 s/step pace is dominated by the larger middle layer calls rather than driven by Jev.

Conclusions and Next Steps

Jev is a nice addition to this pipeline as the intelligent router between 1 of N policies: fast enough and cheap enough that the routing decision stops being something to economize on. Over the next few days I will try and kick off a series of longer runs and see if it is able to navigate its way through the first few starter areas and hopefully fight a gym battle.

The full agent is open source if you want to poke at it: github.com/2389-research/jev-plays-pokemon.