AIventure: Agentic NPCs

There's a pit of lava between you and a switch, and the only thing that can cross it is a small robot who follows you around like an overly keen puppy.

You can't walk it over there. You have to ask. In plain English. "Go and flip the switch." And then you stand back and watch it think.

This is the third room of AIventure, an Angular + PhaserJS adventure that teaches Gen AI concepts by making you play with them. It was an idea I built earlier this year with my colleague Juyeong Ji from Deepmind DevRel. A sort of spirirtual successor to my old Adventure conference technology, except focused on gamified learning and less on social/multiplayer.

The first rooms are about single tool calls – say the words, a door opens. The robot is where it becomes an agent: it gets a goal, not an instruction, and has to loop until it's done.

And – my favourite feature – it has a hard limit on how long it'll keep trying before it sits down and sobs.

What the robot is told

The whole persona is one line:

NPC persona:
"a helpful bot who can handle the switch"

And it gets exactly two skills:

[
  {
    "name": "move_to",
    "description": "move agent to x,y coordinate",
    "parameters": {
      "type": "object",
      "properties": { "x": { "type": "integer" }, "y": { "type": "integer" } },
      "required": ["x", "y"]
    }
  },
  {
    "name": "find_switch",
    "description": "search the room for available switch, will return (x,y) coordinate"
  }
]

Notice what's missing. There's no flip_switch. There's no map in the prompt. The robot doesn't know where the switch is until it asks, and it can't flip anything until it's standing next to it. The goal only gets solved by chaining the tools together – which is the whole point of the lesson.

The loop: ask, act, report back, ask again

The solution page describes the robot's loop in three words: retrieve instructions, execute, re-prompt. Here's how that plays out in the code when Gemma 4 is running locally through LM Studio:

flowchart LR
    P["Player: 'flip the switch'"] --> M["Gemma 4<br/>(LM Studio)"]
    M -- "tool call:<br/>find_switch / move_to" --> G["Game rules<br/>PuzzleRules.ts"]
    G --> R["AgenticNPC<br/>moves on grid"]
    R -- "model-tool-execution-result" --> S["lmstudio.service.ts<br/>appends tool result"]
    S -- "re-prompt with history" --> M

1. The model gets the goal and the two tools, and calls find_switch.
2. The game's rules table handles it – there's a rule that maps find_switch to a FIND_SWITCH action and move_to to MOVE_AGENT.
3. The result comes back to the service as a model-tool-execution-result event.
4. The service pushes it into the conversation as a role: "tool" message and, once every pending call has an answer, fires the whole history back at the model.

// From src/app/services/lmstudio.service.ts (trimmed)
async handleToolResult(result: any) {
  const lastMessage = this.history.at(-1);
  const callTarget = lastMessage.tool_calls.at(-this.waitingToolCallCount);
  this.history.push({
    role: "tool",
    tool_call_id: callTarget.id,
    content: JSON.stringify(result.output),
  });

  this.waitingToolCallCount--;
  if (this.waitingToolCallCount > 0) return;

  // All tools answered – ask the model what to do next
  const response = await fetch(this.apiUrl, { /* same history, same tools */ });
  // ...
}

That's the agent loop. No framework, no orchestration layer – just a chat history that grows by one tool result at a time, and a model deciding whether it's finished yet.

The agent is a state machine with feelings

On the game side the robot is an AgenticNPC, which extends the regular MovingNPC. Its brain is an enum:

// From src/game/core/AgenticNPC.ts
export enum AgentState {
  FOLLOW,
  BUMPED,
  THINKING,
  EXECUTE,
  VERIFY,
  GOAL,
  FAILED,
}

By default it's in FOLLOW, chasing you around the room. Once you've given it a job, THINKING and EXECUTE are the states that matter – those are when its tool calls get counted (more on that in a second).

It also has a thought queue – timed messages that pop up as bubbles over its head. "Bumped!" when it walks into you. "Waiting.." when it's paused. It sounds like a cosmetic touch, but it's doing real work: the player can see what the agent is thinking without opening a log. That passes the five-foot test – someone looking over your shoulder on a train can tell what the robot is up to.

The bit I'd steal for every agent: a tool budget

Here's the part that deserves more attention than it'll get:

// From src/game/core/AgenticNPC.ts
incrementToolUse() {
    this.toolUseCount++;
    if (this.toolUseCount >= CONSTANTS.AGENT_MAX_TOOL_USE) {
        this.setState(AgentState.FAILED);
        this.commandTarget = null;
        this.setThought("I've reached my limit...", CONSTANTS.THOUGHT_BUBBLE.LONG_DURATION);
    }
}

Every time a model function call arrives, InteractionSystem.ts finds whichever agent is currently THINKING or EXECUTE, bumps its count, and – if the agent has just tipped into FAILED – drops the call on the floor instead of passing it on to the trigger system. The counter resets whenever the robot starts a fresh round of thinking.

Once it's failed, the robot sits there with a "Sob Sob" thought bubble. Which is honestly how I feel about most runaway loops I've debugged.

It's a small local model running on a laptop. Sooner or later it'll pick a bad coordinate, or ask for the switch again when it already knows where it is. Rather than pretend that won't happen, the game gives it a budget and a graceful way to say "I can't do this". A visible failure beats an invisible infinite loop.

Movement is dumb on purpose

When the model says move_to(x, y), you might expect A* pathfinding. You'd be wrong – and that's actually one of the quiz questions on the solution page.

The robot does the simplest thing that could possibly work: step along X towards the target, then along Y. If the next tile is blocked, try the other axis once. It asks a GridManager whether a tile is free before moving, so it doesn't end up sharing a square with anything else.

// From src/game/core/AgenticNPC.ts (trimmed)
// Simple pathfinding: X then Y
let nextGx = currentGx + stepX;
let nextGy = currentGy;
if (stepX === 0) {
  nextGy = currentGy + Math.sign(dy);
}
if (!this.gridManager.isBlocked(nextGx, nextGy, "npc")) {
  this.moveTo(nextGx, nextGy);
} else if (stepX !== 0) {
  // simple retry on the other axis
}

Surprisingly hard to make a robot this unsophisticated – you keep wanting to add things. But the lesson here isn't pathfinding. The intelligence lives in the loop, not in the legs. If the robot gets stuck, the model sees that in the next tool result and can pick a different coordinate. That's the agent doing its job.

Conclusion

• Give agents goals and small tools, not scripts. Two functions and a one-line persona were enough to need a genuine multi-step plan.
• Budget your tool calls. A hard cap plus a clear FAILED state protects you from the loops you didn't imagine.
• Show the agent's thinking where the user already is. Thought bubbles in the game world beat a console nobody's reading.

The full robot lives in src/game/core/AgenticNPC.ts on GitHub. Go and make it cry.

Article published on: 30 Sep, 2026