The challenge
GoodAI set out to create AI People – a game where players interact with AI-powered characters that feel alive: they think, speak, and respond naturally in real-time. To realize this, multiple technical challenges had to be addressed:
- Local vs cloud LLM deployment: LLMs would ideally run on players’ machines. If the player’s PC couldn’t support running the LLM locally alongside the game, a cloud option had to be provided to the player.
- Real-time voice: Players would speak to AI characters and hear them respond naturally.
- Security: Prevent players from manipulating the underlying prompts or generating inappropriate content.
- Uncensored responses: Game characters needed authentic personalities without corporate AI safety constraints that would break immersion, while still filtering out inappropriate content.
This wasn’t just about integrating an API – it required building a complete AI gaming experience from scratch.
My approach
Fine-tuning LLMs for better performance and character authenticity
I fine-tuned open-source LLMs to improve their performance and to create characters that felt real rather than corporate-safe. For making characters which felt real, the challenge was removing the political correctness baked into most models while maintaining appropriate boundaries for a game environment. This involved:
- Custom fine-tuning approaches to adjust model behavior
- Testing different model architectures and quantization levels for local deployment performance
- Balancing model size against response quality and speed
Implementing cutting-edge voice technology
When a new TTS model was released (literally less than a week old), I moved fast:
- Integrated the model into our system
- Built a complete server infrastructure around it
- Packaged everything with PyInstaller so it could run locally on players’ machines
I’ve also integrated local and cloud ASR solutions.
Engineering safety without breaking immersion
I developed security measures that worked invisibly:
- Prompt injection prevention to stop players from manipulating the AI system instructions
- Content moderation filters that caught inappropriate scenarios before they reached the LLM
- Systems that enabled core game mechanics
Infrastructure and deployment
Alongside a coworker I maintained cloud infrastructure, ensuring the complete pipeline – from player input through speech recognition, LLM processing, and text-to-speech output – worked smoothly.
Results
- Delivered alpha release to real players with functioning AI characters
- Enabled local gameplay – no API costs, no latency from cloud calls
- Demonstrated technical feasibility of consumer-grade hardware running sophisticated LLM-powered gaming experiences
To see it in action, take a look at this video, where together with a coworker we showcased the game by playing it.
Player feedback was mixed – some loved the authentic AI interactions, others found the technology still rough around the edges. But the technical proof-of-concept succeeded: we showed that LLM-based gaming was possible.
Technical insights
This project revealed how quickly the machine learning landscape was evolving. Working with TTS models less than a week old, fine-tuning LLMs for specific use cases, and deploying all of it locally rather than via API required moving fast and solving novel problems daily. The security challenges were particularly interesting – prompt injection prevention in ways the industry was just beginning to understand.