Game Designs for Generative AI
42 design ideas to apply real-time genAI to games
Much of the discussion on games and virtual worlds has surrounded the use of generative AI (genAI) in the production process—to save money, accelerate development, and ideate more rapidly. Those are good reasons to incorporate genAI into the development process, but an even more exciting opportunity is to aspire to Level 5 Generative AI, which means building it into the game loop itself.
Back when I wrote my book Game On, I wrote a list of 42 fun things. Of course, there are a lot more than this number of things that are fun. Still, these can be a helpful start to a brainstorming session. I used it this a list (with a slight change to the order) to see what kind of in-game generative AI I could identity. I hope some of you feel inspired to use what’s here—or perhaps use the ideas as a jumping-off point for your own iteration.
Exploring a World (#1)
A vast, open world is continuously generated for you. Unlike simpler procedural systems in the past, this world contains layers of “visual storytelling” in which the past challenges and struggles of the people that have lived there are communicated through their artifacts, design of their homes, and evidence of their deeds. Stories can lay beneath the exploration of a space.
Collecting (#2)
Today's collection systems tend to be about static items, but what if they were about collecting inputs that the player used to prompt interesting content into existence? For example, could you go beyond items entirely and generate personalized player music, player sound effects, animations, and dance moves?
Finding Unexpected Treasure (#3)
Generative loot drop tables where the items have hidden attributes used to compose prompts that generate completely unique-looking items (some prompt components could be rare!) You could use a similar item-generator to populate the content of Collecting (#2)
Achieving a Sense of Completion (#4)
When you hit key milestones in the game, you'd be invited to make generative art prompted by your unique experience of the story. Sort of like those vignettes that appear in endgame sequences, except now they're far more memorable and shareable (consider the viral potential of epic vignettes!).
Gaining Recognition for Achievements (#5)
While playing the game in a virtual world, it may become adorned with statues and other monuments of your achievements, spreading tales of your heroic exploits to other players and NPCs.
Creating Order Out of Chaos (#6)
Play a game that generates a city in a state of general disorder. The metropolis could be unique down to the level of locals, politics, and architecture. Your mission would be to fix everything (correct supply lines, traffic, etc.) and make it shine.
Customizing Virtual Worlds (#7)
Think of all the sandbox worlds that already exist--imagine using generative AI to completely refurbish them with new biomes, housing, and even puzzles. Then, as the players experience the game, AI analyzes their experiences and chats to intelligently scale up or down the difficulty of challenges and opportunities, thus making sure the game is the most fun for the most desirable players.
Gathering Knowledge (#8)
Some games have a "lore book" for tracking discoveries as you play the game. What if the lore book was more personalized and reflected your discoveries and experiences in a constantly-evolving world? In this way, game lore not be carved in stone but instead created on the fly from the actions of all the players in an MMO.
Organizing Groups of People (#9)
A game in which you need to conduct negotiations with characters with a wide range of incentives. Some of your counter parties may use deception. Use time pressure (like a countdown clock) to see if you can reach a positive outcome in time.
Noting Insider References (#10)
Generative AI monitors popular culture, memes, and references, then inserts riffs as the lore evolves (see #8). By hinting at things that only insiders know, the gameplay becomes more relevant to "real life."
Being the Center of Attention (#11)
Invite a player's avatar to participate in generative scenes and content displayed in a social context so that the player gains a sense of importance in the environment. As a reward for accomplishing certain milestones, a player could apply additional stylization filters (like those used in RunwayML).
Experiencing Beauty and Culture (#12)
Envision a game about renovating homes where you can apply generative textures, furnishings, and even bric-a-brac to interiors and exteriors. Keywords used in the prompts for the generative content would be “purchased” as part of a budget. Gameplay resolves with the sale of the property: your profit margin is determined by how often virtual buyers like your work.
Romance (#13)
Several RPGs feature romantic storylines that depend on building relationships with a character over time. A humanlike AI chat interface would allow you to gain "relationship credit" by discovering things about the character through natural conversations rather than the typical prewritten dialog trees.
Exchanging Gifts (#14)
A generative AI game could learn about what you like to do via your interactions and send you in-game presents that are special just for you. They'd likely get a lot of attention and generate positive feelings for the game if the gifts were helpful, unique, and interesting-looking too.
Being a Hero (#15)
Characters in the world might start to speak of your heroic deeds with awe and wonder--instead of the canned lines, it would become based on your actual experiences in the game.
Being a Villain (#16)
As you perform evil acts, characters of the world will begin to perceive you with fear and contempt, and you might hear them whisper of the dastardly deeds you've done.
Being a Wise Old Being (#17)
Characters in the world will begin to speak of how you wisely solved certain puzzles or resolved a diplomatic difference between other characters.
Being a rebel (#18)
Those in positions of authority will begin to grumble about how you always seem to get away with everything... and they'll be specific about it.
Being the ruler (#19)
You could issue commands to your vassals, prompting them to carry out behaviors without needing to micromanage them.
Pretending to Live in a Magical Place (#20)
You can imagine a game mechanic that uses prompts as input to create a near-infinite combination of magical effects, accompanied by unique visuals prompted into existence by the spells. The game might be about unlocking the power to use specific prompt elements in your personal spellcraft.
Listening to a Story (#21)
Virtual worlds could feature quests that have fully-generated backstories, prompted into existence from models trained on the world's lore--yet adding new facets as you discover content. As a result, you may get a surprisingly different experience upon replay.
Telling Stories (#22)
Entertain the AI with a new game mechanic in which you improv while the game understands and rates your performance. People may find performing for AI less stressful and more fun than in front of humans. However, AI could simulate an audience with multiple generated individuals for a more immersive experience.
Predicting the Future (#23)
An in-game foreshadower that does generative "fortune cookies" with text and imagery, dropping clues to something important the player needs to do in an upcoming quest (which would also be generated).
Competition (#24)
Players compete in a new form of fashion game where clothing elements, patterns, accessories, make-up, hairstyles, and even runway walks are created from prompts (humans would judge the result). [interesting digression: use outputs from this competitive-prompting games such as this adjust a Reinforcement Learning from Human Feedback model.]
Psychoanalyzing (#25)
Tables turn in this game wherein you have the intellectual superpower: empathy. As you provide "talk therapy," the mood of the AI will subtly change and be expressed in music. You'll have to interact carefully to get the desired result based on AI chat and interpreting the generative melodic clues.
Mystery (#26)
Murder mystery game where you need to uncover clues about a crime by interviewing generative AI characters.
Mastering a Skill (#27)
Imagine a game about collecting keywords which are then used as prompts to generate some of the game’s content: you could have progression meters specific to subcategories of words. For example, you could increase your level in the “Color Mastery” skill progression by discovering color words in play and using them frequently in prompts.
Exacting Justice and Revenge (#28)
A generative quest engine could offer revenge-worthy events during the player's experience, note the results in a persistent store, and ensure that the player has satisfying opportunities for revenge later.
Nurturing (#29)
Use generative AI to cultivate gardens that involve breeding and gene-editing fruit and flowering plants in a variety beyond what procedural systems could accomplish. Players then trade the resulting new crops in a virtual farmer's market to create even more possibilities.
Excitement (#30)
The crowd goes wild in a sports simulation game featuring generative AI "live commentary." The sportscaster and crowd reactions stay new and relevant as AI reacts emotionally to the events as they occur.
Triumph Over Conflict (#31)
A science-fiction 4X game on a galactic scale where each generated alien culture and planet develops uniquely. These virtual systems then interact with each other and the player to create a dynamic and complex universe never experienced and played in before.
Relaxing (#32)
A breeding game in which you can raise social beings from a virtually unlimited palette of "galactic DNA." This results in various traits that make each entity unique. You earn access to more DNA by raising your creature to adulthood and releasing them into serene ecosystems where they interact realistically with its environs.
Experiencing the Freakish or Bizarre (#33)
Ancient horrors lurk in a survival game in which unsettling generative creatures are never consistent except that they are always scary and out to get you!
Being Silly (#34)
A goofy rap battle rhythm game in which players decide who the contestants are (such as cheese vs. wall paint). Players supply the beat with a rhythm mechanic to support their side as AI performers rap in time. The better the player does, the better their rapper freestyles!
Laughing (#35)
Laugh out loud during a whimsical party game where players are issued various absurd challenges and mini-games. Players never know what to expect, and part of the fun is seeing what the generative challenge system comes up with!
Being Scared (#36)
It will be impossible to stay in your seat while playing an endless horror-themed platformer that generates unique artistically spooky levels and fills them with jump scares that are always unpredictable.
Strengthening a Family Relationship (#37)
A social game that takes your old family photos and generates altered versions in new scenes filled with bric-a-brac, turning them into a hidden object game. As you play with other family members, you’ll find opportunities to connect and reminisce about what the old pics mean and laugh about the added objects. (Naturally, AI should tell you what the added hidden object are!)
Improving One’s Health (#38)
A generative AI augmented reality game that creates quests completed by taking actual walks and hikes in an area. The "quests" are generated from the local maps, weather, and past experiences of other players in the same place while incorporating information about local history and wildlife into the experience--like having unlimited personalized geocaching adventures.
Imagining a Connection with the Past (#39)
Players solve generative puzzles relating to the past in a historical adventure game set in a museum or an actual geographical region. The player interacts with exhibits, locations, and attractions that come to life through AI dialog and augmented displays.
Recognizing Patterns (#40)
How about a “merge” game with a generative twist? The player combines newly generated objects to merge based on appearance but are personalized, infinitely varied, and create a new depth to the level system, inducing a “flow” state for the player.
Improving Society (#41)
In the beginning, there was the prompt. You are a godlike being, able to morph the states around you; as you gain dominion, you learn new control words that alter the environment. Next, reorganize the terrain and structure of the world as you exercise your will, becoming increasingly combinatorial as you unlock new verbs and nouns. Finally, you gain points for helping the creatures in these realms become successful in their changing environments.
Enlightenment (#42)
A level-5 idea: participate with a virtual game master who learns about you and designs a series of adventures and puzzles for leading you down a path towards self-understanding and "grokking" your place in the cosmos. Afterward, you know the universe in a way you had not before.
Further Reading
I wrote the Five Levels of Generative AI for Games about tracking progress with AI in games and virtual worlds. Some of the ideas in this article span the level 3-5 range.
The true Level 5 would be when you can speak anything into existence, as discussed in The Direct From Imagination Era Has Begun.
This has been eye opening for me. For months, I've been contemplating on creating a #text-to-mod genAI software, where you can use an existing game and kind of "speak a mod into existence". It has its challenges as it needs to adhere in style, logic, design to the original game in some cases, but I believe it would open up tons of opportunities for the modding community (my initial deploy idea was for Minecraft or our GTA RP server).
Especially with web3 games, where you could build different games around the same NFTs, keeping their origins but working in an entire different setting. We'll see where that goes and pieces like this really gives me tons to think about, great stuff Jon! 💪🎮🔥