This article is part of our coverage of the latest in AI research.
Researchers at Google and the University of North Carolina have introduced Unbounded, a game engine powered by generative models. Unbounded is based on the concept of a “generative infinite game,” a new paradigm that does not need hard-coded game engines and mechanics.
Unbounded takes inspiration from life simulation games and digital pet games such as The Sims and Tamagotchi. It also incorporates elements from tabletop roleplaying games such as Dungeons & Dragons to provide storytelling experiences.
The researchers believe this work could “lay the groundwork for future research and development in the field of AI-driven interactive experiences.”
How it works
Basically, Unbounded should allow you to create your own game and adventure on the fly. You start by providing it with some initial parameters, such as the character, story, and type of game tasks.
Based on these inputs, Unbounded sets up the game environment and generates character actions. Players can insert their characters into the game and explore the world, interact with objects, and engage in conversations. Users interact with the game through natural language, so the environment is open-ended. The game responds to users’ actions by generating new environments, challenges, and stories (it’s like playing Dungeons & Dragons in your basement with a creative Dungeon Master full of surprises).
Nothing in the game is scripted. All the images, conversations, and actions are created on the fly with large language models (LLMs) and image generators.
The researchers have given special attention to keeping the user experience fluid. LLM responses and image generations have been optimized to be both fast and consistent.
Innovations
Creating an interactive experience with generative models faces an important challenge: consistency. Getting image generators to generate the same character and environment is very difficult. At the same time, having an LLM that responds consistently to the same game rules and mechanics over a very long conversation can be tricky.
The researchers used several techniques to overcome these challenges. On the image generation side, they used low-rank adapters (LoRA) over Stable Diffusion XL to speed up image generation and achieve near real-time speed. They also used DreamBooth LoRA, a technique specially designed to maintain character consistency across image generations. Finally, to maintain consistency in both character and environment, they introduce “regional IP-Adapter,” a new technique that conditions the image generation on the game environment and character appearance. Regional IP-Adapter prevents interference between the character and environment.
For the game mechanics, they developed an LLM-based engine that maintains consistent game mechanics, generates coherent narratives, and produces contextual character responses in real time.
The LLM is built on Gemma-2B. They distilled the LLM using 5,000 user-simulator interaction samples collected from GPT-4o. They first collect diverse topic and character data. They then use a World LLM and User LLM that interact to generate user-simulation data through multi-round exchanges without the need for human data. This data is used to fine-tune the distilled model.
“Our diverse user-simulator interaction data effectively distills Gemma-2B into a capable game engine,” the researchers write.
Would you play an AI-generated game?
This is a very interesting concept and can possibly be the next step for generative AI experiences. A while ago, Google announced GemNGen, a generative model that could simulate the popular game DOOM.
There is certainly value to be gained from real-time interactive games powered by generative models. However, one of the key elements of video games is their deterministic nature. We expect the game to be consistent across time and similar conditions. Generative models are probabilistic in nature. Getting them to have consistent results goes against their very nature and requires a lot of technical backflips that are not guaranteed to work.
Even for tabletop games such as D&D, which are very flexible, we still expect consistency from the storyteller. A good Dungeon Master knows when to add twists or bend the rules to make the experience more entertaining. They know how to weave stories into the experience and add side plots to make things interesting. They can change narratives and fill gaps in backstories as the game progresses. There are a lot of intricacies that have yet to be captured by LLMs.
Moreover, a lot of the fun in games comes from the shared experiences between players. Would the game be as fun if every player got a custom-made version, or would it be better to have repeatable and shared experiences?
But generative AI has surprised us many times before, and usually, the unintended use cases turn out to be the most interesting ones. It will be interesting to see what kind of experiences we will get from interactive systems built on generative models.