Open Source Experiments in Generative Gaming

For game developers, the area gaining the most attention today is the potential acceleration of the production of games. But another is incorporating generative AI directly into the experience of games.

This article is about these experiences, and some of the business and technology challenges that currently exist—along with some open source projects in Unity that you can download to continue some of the experiments begun at Beamable.

Game Development

First, it is certain that generative AI—at both the coding and artistic levels—can significantly speed-up production. Today, it is particularly helpful on game prototyping where one simply wants to get an idea off-the-ground. For the most part, the code and images generated aren’t quite “shippable” yet, but that seems likely to change.

One of the most important parts of game development is getting through prototyping and “finding the fun” as quickly as possible, and that’s where teams will find the most benefit right now:

A simple shooter-game project I made in GPT-4 in a few minutes of work. Generative coding assistants can significantly speed up the scaffolding of game projects.

Generative AI “In the Loop”

While generative AI in the production process may offer numerous benefits—and could lead to greater experimentation, and bigger games made by smaller teams—the more transformative aspects for gaming could come from putting generative AI into the game itself.

However, there are two challenges I want to flag: the cloud-native infrastructure requirements of a proper implementation; and the right business model to support it.

You Need Secure Cloud Infrastructure

When you use the capabilities of an API like Scenario or ChatGPT (two of the generative AIs we’ve experimented with) you can’t simply invoke the generative APIs from within the client code. If you did so, you’d leak your the secret keys for the APIs. This will result in getting your key disabled (the game will now be down until you update the client!) or someone nefarious could abuse your API account—running up a huge bill.

A solution is to create a server-based proxy to securely perform the API requests on behalf of your game clients. Within a server environment, you may also find it helpful to adopt methods like queuing generative content in advance (so that you can provide real-time responsiveness) and/or identifying cases where you can serve the same generated content for multiple users (e.g., if there are the equivalent of key-value combinations that would make generated content non-unique).

All of this can result in a lot of added complexity—the exact kind you don’t want to slow you down during ideation of a game, nor during live operations when you start to scale-up.

How we addressed this: we used the Beamable microservices architecture within Unity to stand up server capabilities to contain all of this, which let the team return to focusing on the game itself. You can see the approach we used in the source code projects shared below.

Business Models Are Complicated

The other reality of the current API-based technologies is that they are usually fee-based. The more your player engages, the more it’ll cost you. This isn’t entirely unique: if you run a live services game, then you’re already paying for cloud-based infrastructure. But the cost of operating these APIs are likely to be significantly higher than you’re used to.

Currently, the approaches to solving this include:

• Forcing people to provide their own secret key: this is actually what we’ve done in our sample projects, because these were made for developers (not “real” players). But this isn’t a practical solution: first, that’s going to lead to a lot of security issues and I don’t think players should trust the games they download with keys. And it would be a UX nightmare for an ordinary player to go pick up a bunch of keys for the APIs you use.

• Throttle the content generation to a maximum threshold, limited to a portion of the game’s purchase price: this could work for some games, if you have a strong handle on the amount of content a player will consume before they are done playing. But capping the content won’t work for most live services games that are built for long-term engagement…

• Build an in-app purchase economy: this could be as simple as needing to purchase refills for generated content, or could be a more sophisticated item-based economy where the generative content is considered a cost of doing business (assuming it is a relatively small portion of the sale price).

On-Device Inference?

The last possibility—not quite practical yet—is to shift the inference from the servers all the way back to the player. That would mean players actually have the models running locally (so the “cost” is buried in their electric bill). This is actually possible (if not entirely feasible) for diffusion-based image models (users run versions of Stable Diffusion on their local machines). It may also be possible for smaller language models (definitely not things approaching the number of parameters in ChatGPT). However, this will carry a great deal of added complexity in terms of ensuring that the target market for a game has adequate hardware to run the models—not to mention new quality assurance complications.

As more neural compute makes it to end-user devices, much of this could change: keep an eye on the feasibility of on-device inference for generative games.

Open Source Projects

Here are the projects the team at Beamable worked on during a recent GameJam. All of them are readily available on Github:

AI-Forge

This is an example of a new type of crafting system that utilizes generative AI. Instead of a typical hierarchical crafting system with predetermined inputs and outputs, this demo uses ChatGPT to generate item descriptions, and then the descriptions are fed to Scenario to create the item art. Items are stored in Beamable and you can sell them for currency.

Github: https://github.com/beamable/ai-forge

Genamon

This example demonstrates a concept for a creature collection game. Like AI-Forge, it uses ChatGPT to create the descriptions of the creatures; and then the description is fed to Scenario. Beamable is used to manage the player’s inventory of creatures; and then it mints these creatures on the Polygon blockchain (using the Beamable live services plug-in for Polygon). Integration with a trading marketplace is one possible approach to the “business model” problem outlined above—in this case, an open marketplace on Polygon made it easy to add this capability. You could also see how this might work for different business models, like a traditional in-app purchase economy or a built-in auction house.

Github: https://github.com/beamable/genamon-polygon

Spoons

In this game, you play a traveling salesmen who is trying to sell spoons door-to-door. It uses Scenario to generate the house images, and the davinci API (OpenAI) to make the conversations with characters work. I like this use case because it has “real gameplay” with win/lose conditions as you interact with characters, and each character has their own belief systems you need to figure out.

Github: https://github.com/cdhanna/spoons

Are you experimenting with generative AI in a game?

Further Reading

• I previously wrote about ideas on how to use generative AI to create fun in Game Designs for Generative AI.

• Five Levels of Generative AI for Games provides a way of thinking about the different applications of generative AI to virtual worlds and games—and establishes a method for tracking progress, similar to methods used in autonomous vehicles.