Anatomy of an MMORPG
What Building an MMO looked like in 2007
The following is an article I wrote a while back, called “Anatomy of an MMORPG.” It was originally published on March 27, 2007 in PlayerVox, an online publication intended to be a player-written supplement to the socially-driven, user-generated content on GamerDNA (which was then called GuildCafe). It was an idea that didn’t get much traction (perhaps a bit ahead of its time) but this article was liked by a lot of people, so I wanted to save it and republish it here.
An MMORPG is like a large piece of enterprise software: one consists of databases, specialized servers, client software and a huge amount of content. To begin our journey through the world of an MMORPG, let’s begin with the part that seems most familiar: the content.
Content and 3D Progamming
When people use the term “content” in an MMORPG, they are referring to all of the places, creatures and things you can visit within the virtual world. It includes the actual landscapes, buildings and dungeons you explore; the monsters and players you can see; the graphics for items. Even things like music, sound effects and the special effects used for things like explosions and spells are a type of content.
Very few individuals possess all of the skills necessary to create the large variety of visual effects that are present in an MMORPG. Just as in the film business, where there are graphics programmers, costume designers, makeup artists and miniature modelers (to name just a few), the game business employs a large number of specialized artists who are capable of working on the various sensory components of the game. By the time you see an animated monster inside a game, it is likely that it is the product of several peoples’ work.
Concept Art
The first step in creating a 3D image is to develop what is called the “concept art.” Depending on the game company, this could be as simple as a pencil sketch of the intended creature or building—or could be a sophisticated piece of art, crafted with either traditional media or inside a paint program (Photoshop is popular). The purpose of the concept art is to give the 3D artists a blueprint to follow; it also provides the art director with an opportunity to establish consistent themes across the various art.
3D Modeling
Once the concept art is created, the job of creating a 3D animation begins. The first step is to create a 3D model of the images. There are a large number of 3D modeling programs on the market, but most game companies use either Maya or 3D Studio Max. There are also some specialized products, such as Poser, for designing things such as human figures.
Creating a new model from scratch requires one to build a model from a set of “primitives,” which include objects such as spheres, cylinders, cubes, and so forth. The designer has the ability to attach these objects together, or perform a variety of transformations such as stretching vertices, twisting objects, and so forth. In many cases, a designer will tap into a library of existing models and modify them as appropriate, since creating new models entirely from scratch would take far too long for a project that may easily include hundreds of similar images. Finally, the designer might use another type of product called a “3D sculpting” program (such as Zbrush) to add natural looking features to the model, such as wrinkles or cracks.
The final surface of a 3D image is a set of polygons. The total number polygons in a model is called the “polygon count.” Models with larger number of polygons take longer for your computer to render, because each individual surface has its own texturing and shading. Because a computer game needs to render these models in real-time, the number of polygons is limited to what current computers are able to handle. As of 2007, some of the more advanced computer games in development have 10,000 or more polygons, whereas a model for a Hollywood movie (which has the luxury of rendering all of the images in advance) might have hundreds of thousands or even over a million polygons. Thus, game designers not only have to make models that look good—they have to make models that look good within fairly strict technical limitations.
Texture Mapping
The 3D model might have the shape of the image, but it is still a far cry from the vision laid out by the concept artist. The 3D model needs to be painted, or else it would look like little more than a sculpture.
Often, the people who do the painting of a 3D model are separate from the 3D modelers. These “texture artists” are responsible for creating images that are wrapped onto the 3D model, making them look like they’re supposed to.
In theory, a texture map is just a 2D image that is wrapped onto the geometry of a 3D model—they could be created in any paint program, be it Photoshop or even the paint program that ships with Windows. For textures of reasonably flat surfaces (such as the side of a building, or the bark on a tree), that’s usually adequate. When you need to create a texture map for a more complex object—like an orc or a racecar—artists these days will turn again to products like Zbrush, which allows them to paint directly onto the surface of a 3D solid, rotating and turning the object as if it were a figurine.
Bump Maps
There’s one problem with traditional texture maps: they can look “flat,” especially when applied to flat surfaces. If you remember older video games (or poorly funded current ones), you might have noticed that the cracks on walls look flat, more like decals pasted onto the sides of the surface. That’s because your brain expects things like cracks and creases to be lit differently depending on where you stand, and simply coloring a surface isn’t enough. Modern games use a technology called bump maps.
A bump map is a special type of texture that assigns a height to each pixel, which impacts the way light is received by the surface. For example, a pixel which has a negative height could act like a depression in the surface, swallowing up some of the light that is cast on it. Using bump maps, it is possible to create realistic-looking cracks or bumps in the surface, allowing much greater detail to a surface. This is just one of the ways that modern MMORPGs (and videogames in general) can add detail to an image without resorting to prohibitively complex 3D models. Thus far, very few MMORPG titles are using bump maps, although they’ve become common in modern FPS titles.
Physics and Kinematics
In addition to the visual depiction of a model, the designer also needs to define how the model interacts physically—the kinematics of a model. This includes things like how an arm might move through a sword swing, how a particular weapon is gripped, how an unconscious body might flop onto the ground, how different parts of a skeleton move when you tug on a particular bone, and so forth. The process of defining the skeletal connections in a model and how different objects attach and move each other is called “rigging” the model. Usually, the model designers are the same people who develop the kinematics—but often, the job of animating the complete models within a scene is done by other content creators.
Beyond the individual models, 3D worlds are simulations of physical reality: objects have acceleration and movement. Have you ever had a butterflies-in-the-stomach sensation while watching your character drop off a cliff? That’s because the velocity is accelerating based on Newton’s Law of Gravity, which our brains have evolved to interpret even when we only have visual cues. Many MMORPGs use basic physical rules such as gravity and friction through a fluid (for example, reduced movement while swimming). Some games may offer physics for special environments: Vanguard includes directional water currents in rivers, and Guild Wars has floors with varying coefficients of friction, such as icy surfaces that are hard to gain or lose momentum on. Few MMORPGs as of 2007 have implemented some of the complex physics models that have begun to appear in FPS games: destructible environments, parabolic artillery trajectories, debris that clatters around realistically and objects with realistic weight and density—but these are likely become part of a future generation of MMORPGs.
Animation & Composition
At some point, the content designers will have a rich set of models, objects, buildings, textures and environments to choose from. There’s still quite a bit of work to be done: someone needs to composite it into complete scenes. This means placing objects on the map, creating terrain elevation maps, defining scripts to determine how creatures and players interact with them, etc. In the world of FPS games, this is referred to as “level design,” (there’s no consistent term within MMORPG companies, so I’ll call them the “world designers.”) These people are usually a separate team from the model and texture artists. They’ll work closely with those who have designed the lore and concepts for the world (often, they’ll do much of this definition as well) and integrate the content created by individual artists into the world that players will explore.
A certain amount of animation can be done “by hand,” but nowadays most of the animation that you see characters performing comes from motion capture libraries (called “Mocap” for short). During Mocap, a person wears a suit that is covered with various sensors, lights or markers; they then perform real actions, such as fencing movements, dancing, or everyday activities such as sitting down in a chair. The location of these sensors is tracked, and recorded as a set of coordinates within a computer. These coordinates can then be mapped to the rigging of a model, and used to create a realistic 3D model of a person. Using various animations, combined with transition poses (for example, standing up and doing nothing; or an aggressive combat stance) allows a character to express a wide range of behaviors. Content designers can string these together as part of scenes, define patrol patterns for a creature, or respond emotionally to a conversation. Today, motion capture libraries are available from a number of sources that allows game designers to avoid recreating the same movements over and over—but certain motion capture (for example, a distinctive dance within a game) may still need to be recorded anew.
A model plays for the camera in this demo of Moven, a motion capture suit made by Xsens
Particle Effects
Another important area of animation in an MMORPG is called “particle effects.” This includes all the explosions, flashes of light, swirls of color and magic when a spell is cast, smoke plumes, fires, and so forth. These are called “particle effects” because the animation consists of a large number of discrete particles, all of which follow predefined functions that define their trajectory, propagation, noise, color transformations, etc. An MMORPG will need particle effects for many combat actions and spells. Most 3D modeling programs include a particle system, and there are now specialized plug-ins available for things such as fire and liquid effects.
Pixel Art
Beyond all of the 3D art, an MMORPG includes a large amount of 2D art: this runs the gamut from the user interface itself, to things such as all the inventory icons for every item in the game. In the industry, the people who create the 2D art are usually called “pixel artists” because they paint these objects pixel-by-pixel, using a zoom-in tool within a paint program. Sometimes, a 3D program might be used to do an initial rendering of these icons—and some artists may even draw or paint them using traditional media prior to creating a finished version within something like Photoshop. Getting icons and miniature objects to look right is a challenge in that they not only have to look good—but also look memorable and recognizable at a small size. It is a specialized skill, and good pixel artists are usually a different set of people from the model designers and texture artists.
Shaders
Another area of modern graphics programming is what’s called a shader. Video cards now include specialized, programmable hardware that allows a graphics programmer to write custom programs that alter how lighting and color transformations occur on various objects. Originally, these were written in specialized assembly language—but now, programmers can utilize High Level Shader Language (HLSL) to define these operations. What can game developers do with shader programming? One example would be a scintillating color that is mapped to the surface of a magical weapon, complete with a lighting effect that causes the color to radiate onto other nearby objects. Special effects like this are made possible because it is a lot faster for your video card to compute these effects in hardware (within the GPU itself) rather than in the software running in your PC.
When developers talk about shaders, there are generally two types: vertex and pixel shaders. Pixel shaders can do things such as modify the lighting or color properties of an individual pixel, whereas a vertex shader can modify the geometry of an object. Vertex shaders can do effects, such as deforming an object in strange ways, or giving new ways of viewing a scene such as haze or rippling-water effects.
Differences Between FPS and MMORPG 3D
A lot of the what goes into designing a 3D world is the same between MMORPG and FPS games. However, one big difference is the huge number of models and texture maps that exist within an MMORPG as compared to an FPS game. Although many game computers today have more than a gigabyte of RAM, it’s still a challenge to cache the various models and texture maps that a player sees on their screen. In a given fantasy-themed MMORPG, there might be thousands of weapons and armor pieces that a character might wear, and it needs to be possible to quickly render the various appearances of characters regardless of who you might come across.
Client
All of the content ultimately needs to be displayed, and that is one of the jobs of what’s called the client. This includes a 3D engine for rendering the game’s environment, as well as a user interface that allows you to do things like cast spells, shoot guns or manipulate inventory items.
The client talks to the server through a network protocol that consists of “packets” of information such as the location of objects, chat messages, and so forth. The client is responsible for interpreting these packets and displaying the world to the player. Client programs themselves do little or no interpretation of game mechanics in an MMORPG, because doing so opens the door to hacking. Instead, clients are the hands, eyes and ears you use to interact with the servers.
Publishers now invest some effort in making it difficult to modify client programs, to prevent users from hacking them in such a way as to automate activities or grant unfair advantages over other players. However, many modern MMORPGs contain “modding features” that allow users to hook into the client and add their own alterations or extensions in an approved way. For example, World of Warcraft uses a lightweight programming language called Lua that allows players to build their own user interface extensions, but does not offer any methods for fully automating your character.
User Interfaces
The simplest aspect of the client—but often the hardest for developers to get “just right” is the windowed interfaces within a game. This is all of the dialog boxes you interact with for common activities such as moving items around your inventory, assigning hotkeys to actions, etc.
There are two aspects to the creation of the user interface. The first is programming all of the user interface components themselves, and the second is organizing the components into a framework that is easy and fun to use. Although Windows computers come with a library of software that makes it possible to created windowed, mouse-driven interfaces, many games end up writing their own user interfaces from scratch due to the fact that these libraries (a) don’t look enough like a game and (b) don’t work well when combined with a 3D game engine.
It is fairly inefficient, but many games end up rewriting similar things (including basic components, such as slider controls) over and over again. Microsoft has attempted to solve some of this by creating the Windows Presentation Foundation (a feature of the Vista operating system), which makes it possible to use all of the 3D functions available in Direct3D side-by-side with 2D windowing features—however, games that require Vista are still quite far off as of 2007. In the meantime, developing the plumbing that makes possible a game’s unique look-and-feel is an area that can consume considerable development resources.
3D Engine
With the exception of a few 2D or isomorphic MMORPGs, almost every modern game now includes what is called a 3D engine. It is the job of the engine to take the mathematical description of objects, communicate them to your video card, and provide an animated world for you to experience. Most MMORPG companies license a third-party engine, but even then they will frequently need to perform a lot of low-level modifications that can only be done by advanced graphics programmers.
Servers
The servers are the heart of any MMORPG: it is here that all of the rules are interpreted, and all the data on the location and movement of objects is maintained. All of this work must be done within the servers, and not the client, or else unethical players could hack their client software to give them advantages unavailable to others. Most MMORPGs consist of a variety of different servers for providing different parts of the game experience.
Login servers record everyone’s payment information, and act as gateways that allow players to connect to the game servers. After you give your username and password to a login server, your client is given an electronic “admission ticket” that lets your client access all of the other servers that are required during the course of your game.
Game servers control all of the actions within a particular area of the game. These servers enforce all the game rules, activate skills and attacks, and record the location and movement of objects.
The movement of objects is usually recorded as a vector, meaning that motion is recorded as a combination of magnitude and direction—in other words, the server transmits information to all the clients in an area that various objects are moving in a certain direction at a certain velocity. This works better than a system that transmits absolute coordinates (which would look choppy, and require a large increase in bandwidth to update regularly) but it does introduce other issues such as strange animations due to latency: for example, on a laggy connection, you might see a creature walk through a wall and then warp back to its original position. The reason this happens is that the server had told your client that an object would continue on that path, but lag prevented you from getting the further update about the same object stopping or changing direction.
Another job of the server is what’s called “collision detection.” In other words, you need to detect if two objects have intersected with each other: for example, you need to stop when you hit a wall (and some games might not allow creatures or players to pass through each other). The job of collision detection sounds simple in concept, but in implementation it requires a bunch of relatively complex matrix math. The reason is that these are 3D worlds, and determining whether one object in 3D space has intersected with another requires a bit of math to figure out. Precise algorithms are computationally expensive, so most games use an approximation—and even then, only detect a collision after the fact, and move objects back to compensate. Poor collision detection can result in strange problems like the ability to jump through objects that should be impassable, or getting stuck inside terrain.
Because MMORPGs are large worlds, and because it would consume massive amounts of bandwidth to report to every client what happens with every object and event within the world, techniques have been created to divide the world into discrete units (often called “zones”). In games like WoW, this is done by allocating different servers to different areas of the game (such as continents) as well as separate servers for dungeon “instances.” Vanguard, which features a “seamless” world, actually divides the continents into what are called “chunks.” When different servers are used to maintain these different zones, they are often referred to as “zone servers.”
Chat servers provide communication between players. Although players might be connected to different game servers depending on where they are in the virtual world, things such as a guild-chat or instant messaging needs to work across the entire game. To handle this, most MMORPGs are designed so that you connect to a worldwide chat server regardless of where you travel. From a technical standpoint, after you connect to a login server you are then connected to a chat server in the background, and then connected to a game server specific to your location.
Databases record all the information on your character’s statistics and items, and save them in a database so that you can come back to the game at any point in the future. This data is stored in what’s called a relational database management system (RDBMS), the same sort of technology that is used to maintain things like banking information, electronic commerce systems, retail inventory systems, and so forth. Today, software designers can choose between commercial products such as Oracle, Microsoft SQL Server, or open-source options such as MySQL. Implementing a persistence system for a game involves defining a database architecture and then creating software to “wrap” the database for doing things such as saving a character or spawning a new item. It’s critical to design this system well, or the MMORPG will be plagued by item duplication problems, vanishing items or laggy item spawning.
Website integration:
It is becoming increasingly common for MMORPGs to expose some of their content to the World Wide Web. This can be used to provide ladder and player ranking information, character profiles, guild information, and so forth. Adding these capabilities requires a whole other area of expertise. Designers will write server-side code to access the game databases (often using popular Web development frameworks such as php or Java); other developers will create the HTML and graphics used to present a crisp user interface. In addition, many games are now packaging their data so that it can be reused on other people’s websites as well, which means defining data in a language called XML (for extensible markup language).
Patchers
MMORPGs are dynamic, changing games—and software updates happen regularly. As a result, there are patching servers that contain updates to the game code. Depending on the size of a game, this might need to update hundreds of thousands or even millions of clients in a month. Patching servers are simple in concept: they contain all the code needed for a client, and are capable of delivering any of the individual files that have changed since you last updated. However, implementing them to handle the massive demand that occurs around the time of a new patch requires considerable preplanning as well as a significant investment in server infrastructure.
Server Farms
How many computers does an MMORPG require? In 2006, Vivendi reported in SEC disclosures that it maintained over 1,900 servers for WoW. Aside from the software infrastructure, the task of designing and then operating a scalable server farm—with all the security, network and maintenance issues it entails—is a huge job in itself.
Game Design
Even if you have a great 3D environment, fantastic content and animation, a robust and friendly user interface and a server infrastructure that’s ready to support all of it—you still don’t have a game. Creating a good game is well beyond the scope of this article, but to give you a sense of some of the central issues within an MMORPG, the designers need to think about things such as:
What are the objectives? What will give individual players as well as guild organizations goals to keep them interested in playing, month after month?
What are the rewards and challenges? How do you reward the player quickly enough when they first start playing so that they fall in love, but don’t overload them so that they become complacent and bored?
What about the economy? How do you make the economy for money, items or whatever else the game is organized around vital and dynamic? How do you prevent runaway inflation, adequate outlets for spending, marketplaces for exchange?
Lore: how can you make the background of the world, the characters, the plots compelling enough that people will find it interesting? Even if a person is not a traditional roleplayer, there is a certain amount of context, storytelling and emotional content that is vital to an MMORPG. Without it, even the most jaded of players would find the world hollow and difficult to escape to.
Tactics and gameplay: are there enough options so that the player can have a satisfying experience, regardless of how they approach the game? Is it simple enough to learn, but complex enough that it cannot be mastered quickly? Is there enough there that players will be able to have active discussions about strategies and approaches to play?
If you’ve read the above, you’ll be able to think about just a few of the questions that game designers ask themselves when designing an MMORPG (or almost any game).
Quality Assurance (QA) & Testing
MMORPGs are living, breathing entities. Most of them have frequent updates, be it new content or new features. At the same time, MMORPGs are complex pieces of software that rival many of the most advanced enterprise software technologies—and as such, they need significant testing to make sure that updates are ready for use.
Often, players are invited to play on “test servers” before game updates are made live. This is a good final stamp of approval, but there is quite a bit of testing that goes on even before test-server builds are released. One of the major areas of testing that occurs before a game update is released is called regression testing, which is the process that confirms that previously working code still works the way it is supposed to. Regression testing is usually performed by a combination of human testers (people who go down a checklist and confirm that specific actions work as expected) as well as automated testing software (robot-like programs that invoke different operations, and systematically verify various outputs). If you’ve ever gotten a game update, and something that worked fine is suddenly broken, it is a sure sign that the company isn’t investing in good regression-testing. QA Engineers are the people at game companies who design test suites and write test scripts—but don’t automatically blame them just because broken code gets released. These are complex programs with substantial testing needs, and it is often the game company’s decision to underfund QA (an all-too-common choice at many game studios).
Conclusion
MMORPGs are staggeringly complex ventures, involving skills that include a large variety of game designers, storytellers, 3D modelers, 2D artists, animators, user interface designers, graphics programmers, client/server experts, database architects, network gurus, QA engineers and testers.
What does the future hold? As MMORPGs get even more complex, we might start seeing even more exotic jobs. Perhaps we’ll start to see MMORPG companies recruit for “virtual economists” or “physics engine programmers” in the near future—and in an industry as complex and fast-growing as this one, we can expect new innovations to drive even more advanced requirements into the virtual worlds we’ve grown to love. It’s an exciting time to be either a player or a designer!
References
Interplay’s disclosure of MMORPG funding