---
date: 2026-09-30
title: "AIventure & Agentic NPCs"
state: published
youtubeId: "6yjJ7Prt-RI"
tags: "games", "AI", "demos"
---

# AIventure: Agentic NPCs

There's a pit of lava between you and a switch, and the only thing that can cross it is a small robot who follows you around like an overly keen puppy.

You can't walk it over there. You have to _ask_. In plain English. "Go and flip the switch." And then you stand back and watch it think.

This is the third room of [AIventure](https://github.com/bebechien/AIventure), an Angular + PhaserJS adventure that teaches Gen AI concepts by making you play with them. It was an idea I built earlier this year with my colleague Juyeong Ji from Deepmind DevRel. A sort of spirirtual successor to my old Adventure conference technology, except focused on gamified learning and less on social/multiplayer.

The first rooms are about single tool calls – say the words, a door opens. The robot is where it becomes an agent: **it gets a goal, not an instruction, and has to loop until it's done.**

And – my favourite feature – it has a hard limit on how long it'll keep trying before it sits down and sobs.

## What the robot is told

The whole persona is one line:

> **NPC persona:**  
> _"a helpful bot who can handle the switch"_

And it gets exactly two skills:

```json
[
  {
    "name": "move_to",
    "description": "move agent to x,y coordinate",
    "parameters": {
      "type": "object",
      "properties": { "x": { "type": "integer" }, "y": { "type": "integer" } },
      "required": ["x", "y"]
    }
  },
  {
    "name": "find_switch",
    "description": "search the room for available switch, will return (x,y) coordinate"
  }
]
```

Notice what's missing. There's no `flip_switch`. There's no map in the prompt. The robot doesn't know where the switch is until it asks, and it can't flip anything until it's standing next to it. **The goal only gets solved by chaining the tools together** – which is the whole point of the lesson.

## The loop: ask, act, report back, ask again

The solution page describes the robot's loop in three words: retrieve instructions, execute, re-prompt. Here's how that plays out in the code when Gemma 4 is running locally through LM Studio:

```mermaid
flowchart LR
    P["Player: 'flip the switch'"] --> M["Gemma 4<br/>(LM Studio)"]
    M -- "tool call:<br/>find_switch / move_to" --> G["Game rules<br/>PuzzleRules.ts"]
    G --> R["AgenticNPC<br/>moves on grid"]
    R -- "model-tool-execution-result" --> S["lmstudio.service.ts<br/>appends tool result"]
    S -- "re-prompt with history" --> M
```

1. The model gets the goal and the two tools, and calls `find_switch`.
2. The game's rules table handles it – there's a rule that maps `find_switch` to a `FIND_SWITCH` action and `move_to` to `MOVE_AGENT`.
3. The result comes back to the service as a `model-tool-execution-result` event.
4. The service pushes it into the conversation as a `role: "tool"` message and, once every pending call has an answer, fires the whole history back at the model.

```typescript
// From src/app/services/lmstudio.service.ts (trimmed)
async handleToolResult(result: any) {
  const lastMessage = this.history.at(-1);
  const callTarget = lastMessage.tool_calls.at(-this.waitingToolCallCount);
  this.history.push({
    role: "tool",
    tool_call_id: callTarget.id,
    content: JSON.stringify(result.output),
  });

  this.waitingToolCallCount--;
  if (this.waitingToolCallCount > 0) return;

  // All tools answered – ask the model what to do next
  const response = await fetch(this.apiUrl, { /* same history, same tools */ });
  // ...
}
```

That's the agent loop. No framework, no orchestration layer – **just a chat history that grows by one tool result at a time**, and a model deciding whether it's finished yet.

## The agent is a state machine with feelings

On the game side the robot is an `AgenticNPC`, which extends the regular `MovingNPC`. Its brain is an enum:

```typescript
// From src/game/core/AgenticNPC.ts
export enum AgentState {
  FOLLOW,
  BUMPED,
  THINKING,
  EXECUTE,
  VERIFY,
  GOAL,
  FAILED,
}
```

By default it's in `FOLLOW`, chasing you around the room. Once you've given it a job, `THINKING` and `EXECUTE` are the states that matter – those are when its tool calls get counted (more on that in a second).

It also has a thought queue – timed messages that pop up as bubbles over its head. "Bumped!" when it walks into you. "Waiting.." when it's paused. It sounds like a cosmetic touch, but it's doing real work: **the player can see what the agent is thinking without opening a log.** That passes the five-foot test – someone looking over your shoulder on a train can tell what the robot is up to.

## The bit I'd steal for every agent: a tool budget

Here's the part that deserves more attention than it'll get:

```typescript
// From src/game/core/AgenticNPC.ts
incrementToolUse() {
    this.toolUseCount++;
    if (this.toolUseCount >= CONSTANTS.AGENT_MAX_TOOL_USE) {
        this.setState(AgentState.FAILED);
        this.commandTarget = null;
        this.setThought("I've reached my limit...", CONSTANTS.THOUGHT_BUBBLE.LONG_DURATION);
    }
}
```

Every time a model function call arrives, `InteractionSystem.ts` finds whichever agent is currently `THINKING` or `EXECUTE`, bumps its count, and – if the agent has just tipped into `FAILED` – **drops the call on the floor** instead of passing it on to the trigger system. The counter resets whenever the robot starts a fresh round of thinking.

Once it's failed, the robot sits there with a "Sob Sob" thought bubble. Which is honestly how I feel about most runaway loops I've debugged.

It's a small local model running on a laptop. Sooner or later it'll pick a bad coordinate, or ask for the switch again when it already knows where it is. Rather than pretend that won't happen, the game gives it a budget and a graceful way to say "I can't do this". **A visible failure beats an invisible infinite loop.**

## Movement is dumb on purpose

When the model says `move_to(x, y)`, you might expect A\* pathfinding. You'd be wrong – and that's actually one of the quiz questions on the solution page.

The robot does the simplest thing that could possibly work: step along X towards the target, then along Y. If the next tile is blocked, try the other axis once. It asks a `GridManager` whether a tile is free before moving, so it doesn't end up sharing a square with anything else.

```typescript
// From src/game/core/AgenticNPC.ts (trimmed)
// Simple pathfinding: X then Y
let nextGx = currentGx + stepX;
let nextGy = currentGy;
if (stepX === 0) {
  nextGy = currentGy + Math.sign(dy);
}
if (!this.gridManager.isBlocked(nextGx, nextGy, "npc")) {
  this.moveTo(nextGx, nextGy);
} else if (stepX !== 0) {
  // simple retry on the other axis
}
```

Surprisingly hard to make a robot this unsophisticated – you keep wanting to add things. But the lesson here isn't pathfinding. **The intelligence lives in the loop, not in the legs.** If the robot gets stuck, the model sees that in the next tool result and can pick a different coordinate. That's the agent doing its job.

## Conclusion

- **Give agents goals and small tools, not scripts.** Two functions and a one-line persona were enough to need a genuine multi-step plan.
- **Budget your tool calls.** A hard cap plus a clear FAILED state protects you from the loops you didn't imagine.
- **Show the agent's thinking where the user already is.** Thought bubbles in the game world beat a console nobody's reading.

The full robot lives in `src/game/core/AgenticNPC.ts` on [GitHub](https://github.com/bebechien/AIventure). Go and make it cry.