Demo
Full Level 1 playthrough showing proactive SLM hints, dynamic difficulty adjustment, interactive Q&A, and the debug overlay.
The Problem
In the median mobile game, only 4% of players return after a week. Among the causes on-device AI can address: difficulty walls, missed mechanics, and lack of contextual help.
Tutorials are static and skippable. Players who miss mechanics struggle silently and churn.
Difficulty settings are one-size-fits-all and don't adapt to individual player behavior in real time.
Rule-based systems handle common scenarios. The long tail of hundreds of state combinations goes unaddressed.
Why an On-Device SLM?
One fine-tuned model on the player's device. Zero per-call cost on-device, sub-200ms latency, works offline. Works best when rules can't enumerate the input space and the output is language: contextual hints, dialogue, Q&A.
What the SLM does
- Reasons over complex game state to decide what action to take
- Generates contextual text: hints, dialogue, commentary
- Answers player questions using game knowledge and session history
- Adapts to each player's behavior in real-time
Why not a large model?
- Frontier closed models (Claude, GPT, Gemini) can't be fine-tuned by customers
- Large open-weight models (Llama 70B, Qwen 235B) are fine-tunable but heavy to train and to serve
- Cost per call scales with player count
- Latency too high for real-time game loops
- A small fine-tune beats a general large model on structured tasks for one game
- Works offline, no network dependency
See also: NVIDIA Research: SLMs are the Future of Agentic AI
By the Numbers
Measured comparison: on-device SLM vs cloud LLM for the same game assistant task.
per player session on-device
vs $0.42 per session on Claude Sonnet 4.6 (Bedrock, measured)
response time on-device
vs 4-5 seconds cloud LLM round-trip (measured)
to fine-tune the on-device SLM
Amazon SageMaker, under 3 hours end-to-end
Session = 10 min gameplay, ~150 AI calls. Cloud LLM measured on Amazon Bedrock (Claude Sonnet 4.6), April 2026. On-device measured on Apple M1 Pro.
Use Cases
All use cases share one base model in memory. Each behavior is a per-use-case fine-tune (40-80 MB) that swaps in at runtime.
Proactive Game Assistant
Invisible AI that watches gameplay and decides when and how to help. Structured tool-call output with game-side guardrails.
Interactive Q&A (Ask Pip)
Player asks anything about the game world. RAG-powered answers using embedded lore + session context.
NPC Dialogue
Multiple characters with their own voice and knowledge. Each speaks in-character, in conversation with the player, grounded in the current scene.
Gameplay Summarization
End-of-level recap text from the player's actual session.
Architecture
Game State JSON (every 2-7s)
→ On-Device SLM (base model + per-use-case fine-tune)
→ Tool call / text / Q&A response
→ Game executes
On-device by default, cloud as a fallback for devices that can't fit the model locally. The game code is identical across both; only the transport layer changes.
On-Device
Model runs locally on CPU, GPU, or NPU. Zero latency, zero per-call cost, works offline. This is what the demo shows.
Self-Hosted Cloud
The same fine-tuned model runs in AWS on ECS or EKS with GPU instances. Tested with ECS for this demo; the downloadable game builds use the on-device path only.
AWS also offers managed paths through Amazon Bedrock. A fine-tuned Amazon Nova Micro (a small model in the same size class as the on-device one) can be served pay-per-call. Each fine-tune is its own endpoint, which fits studios that prefer serverless inference.
Guardrails
The model is trained to be conservative, but the game validates every response before it reaches the player.
Hallucination filter blocks suggestions about powers or items the player doesn't have yet.
Per-tool cooldowns prevent spam. Hints, highlights, and difficulty adjustments are rate-limited separately.
Difficulty adjustment only triggers after repeated deaths when the player isn't improving, hard-capped at 3 per level.
Model Stack
| Component | Size | Purpose |
|---|---|---|
| Base model (Qwen 3.5 2B, quantized) | 1.2 GB | Text generation, always loaded |
| Fine-tuned behaviors | 40-80 MB each | One per use case, swapped in at runtime |
| Embedding model | 20 MB | Vector search for game lore (RAG) |
| Vector database | 5 MB | In-engine vector store |
| Total on-device | 1.3 GB | Fits on mobile (4+ GB RAM) |
This demo runs a 2B-parameter base model because it covers tool calling, Q&A, and RAG in one fine-tune. Smaller specialist models exist for narrower tasks (tool calling without reasoning), and we're evaluating them alongside other base models to fit a wider range of devices.
Training
Amazon SageMaker for fine-tuning, Amazon Bedrock for data enrichment. Each training run is under $7. Cheap enough to iterate.
Let a large AI decide everything
Inconsistent, small model can't learn from noise
Deterministic labeling
Consistent decisions the model can learn
Hybrid approach
Consistent labels + AI-generated text variety
Same training data, only the labeling approach changed. Accuracy measured on a frozen human-curated test set.
At 2B parameters: structured output and RAG-grounded text work. Ungrounded free text doesn't, yet.
Tech Stack
Game
- Godot 4.6 (GDScript)
- Custom llama.cpp GDExtension
- In-engine RAG (embeddings + vector store)
- Ninja Adventure asset pack (CC0)
Training & Infrastructure
- Amazon SageMaker (GPU training)
- Amazon Bedrock (teacher enrichment)
- MLflow (experiment tracking)
- Kiro IDE (development)