Every videogame creates some kind of world, be it one that somewhat resembles the real world or an abstract puzzle world. The definition of this world consists of the kinds of objects that can inhabit it and how those objects can interact with one another.

The first part of this definition can be thought of as a taxonomy, or a way of organizing objects into categories and subcategories. For example, an object taxonomy for Pac-Man might look something like this:

There are five major classes of objects: mazes, ghosts, dots, fruit, and Pac-Man himself. Each class can give rise to many different objects with the same basic properties, as in mazes with different configurations and fruits with different bonus values.

Likewise, a class of objects can have subclasses that inherit the properties of the parent class while adding some of their own. The four ghosts all interact with Pac-Man and the maze in the same way, but their movement follows different rules (or so Wikipedia tells me – I’m no Pac-Man expert).

In general, there is more analysis of videogames centered around the rules and potential actions of the game world, but looking at how objects relate to one another can also tell you a lot about a game.

Take a minor frustration from Final Fantasy VII. When you fight the ninja girl Yuffie, she has all these neat wave attacks that immediately disappear when she later joins your party. This suggests that, in this game, “enemies” and “allies” are separate classes that don’t lend themselves to easy conversion. This is not the case in some other Japanese RPGs, such as Chrono Cross; here, when former enemies join your party, they behave pretty much the same. “Enemies” and “allies” are only slightly distinct subclasses of “combatants.”

This difference has some major implications for gameplay. For one thing, Chrono Cross has a doppelganger character who can take on the abilities of an opponent, which wouldn’t be possible in FF7. More importantly, it affects your ability to read your opponents. Knowing an enemy is the same kind of thing as you helps you to predict what it might do more intelligently. In games like FF7, you have no idea what the enemy is capable of, so battles are more unpredictable.

Not that one way or the other is better, but it is a meaningful difference, and a sign that looking at how a game world organizes objects can be as revealing as examining its rules.

Form and content in the visual field of videogames

There is a fairly solid consensus now in cognitive science that visual information (and perhaps information from our other senses) takes two distinct pathways through the brain. The ventral pathway runs along the lower part of the brain, and carries information about things. This pathway focuses on what objects are: that is, what they’re called, what categories they fall under, and what they remind us of. It connects up with the language areas of the brain and those most closely associated with memory. It’s commonly called the what pathway.

The dorsal pathway, on the other hand, runs along the upper part of the brain and carries information about space. This pathway keeps track of where things are and the general spatial configuration of the scene we’re looking at. More recently it has been suggested that the ventral pathway is also involved in understanding the functional aspects of a scene. So instead of seeing what an object is, this pathway sees how an object can be approached, grasped, used, etc. It is traditionally referred to as the where pathway, or the how pathway if its functional side is being brought to attention.

Now, any fan of art criticism may find this all highly reminiscent of the classic form vs. content debate. Throughout history, art movements have concerned themselves with the balance between the content of a work (what it’s about) and its form (how it presents that content). In the past century alone, modernism first declared victory for the primacy of form, then postmodernism (inasmuch as postmodernism was liable to declare anything) declared the distinction irrelevant altogether.

It seems reasonable to suppose that this ancient debate has its origins in our cognitive architecture. Semantic knowledge, recognition, and attention to objects and events seem like a fair description of what goes on in understanding the content of a piece. Likewise, form can be thought of in many arts as the spatial arrangement of content and, more generally, the structure of a work, which might be more readily understood by the part of our brains interested in function.

But if indeed the distinction is so ingrained in the way we think, why should there be a debate? There is little evidence from neuroscience that it makes sense to think of one pathway as naturally dominant over the other. One possible reason is just that criticism is written, and – since the what pathway links up more directly with the language center – content is neurologically easier to write about. This may allow content-driven criticism to dominate discourse when criticism is left to its own devices, requiring an occasional corrective from strong formalism.

Which brings us to the videogame, an art form with a serious content problem. No medium that’s cropped up in fairly recent human history (that I can think of) so steadfastly favors the how pathway over the what. Here’s a small example of what I mean, in two screenshots.

The first is from The Legend of Zelda: Twilight Princess and the second is from The Elder Scrolls: Oblivion. In their current static form, the two scenes look roughly similar, and contain pretty much the same objects: paths, rocks, trees, a dashing rogue with some sweet weaponry. Since the objects in the two scenes correspond, one is hard-pressed to find major differences in their content.

But when you’re actually playing the games, you see the two scenes very differently. Specifically, I’m concerned with the raised rock formation to the center-right of each schene. In Twilight Princess, with an avatar who can’t jump upwards, this area is a barrier and only meant to divide the path. In Oblivion, where avatar movement is less restricted, those rocks can be scaled and used to, say, avoid an enemy or scan your surroundings. And my intuition is that this functional difference overrides the semantic similarity of the two objects. When you’re in the middle of Twilight Princess, you don’t see the rock as a rock, but as a barrier. And in Oblivion, you see it as a platform – provided, of course, that your Acrobatics skill is sufficiently high.

This may seem like a trivial point, but I think it’s worth considering. For all the superficial similarities between the visual appearance of a movie and that of a game, the viewer sees one as a collection of objects and the other as a field of affordances. In watching a movie, it’s the content that most immediately leaps out at us; what people naturally remember about a movie are the characters and the story (what happened). In playing a game, it’s the form that stands out, and what people remember are the mechanics (how the game worked). Perhaps this is why game designers trying to convey content have traditionally resorted to stopping everything and showing you a movie for a while.

This all may prove to be a lasting problem for game criticism and the acceptance of videogames as an art form in general. Critical language has always struggled to convey form in a way that’s both meaningful and coherent. How do you talk about a medium where form prevails? Of course, there are other arts where function takes precedence over content – architecture has been mentioned in relation to games before, but fashion, cuisine, and design could all be seen this way as well. It’s possible that these disciplines may provide more appropriate concepts and terms.

One significant difference, though, is that all of these more traditional functional arts are built on a core of tending to fairly basic human needs, which may provide an essential critical foundation that perfectly useless videogames lack. No matter how brilliant your theoretical foundations, if a dish tastes bad the food critic will send it back, and if you send a model down the runway naked you have performed an art prank, not fashion design. In the case of games, does fun provide a similar foundation? This is a question in need of further study, I think.

1. I’m on the theme of empathy because I’m interested in how emotional engagement works in games as opposed to non-participatory media, and empathy with the emotions expressed is what comes to mind when talking about emotional engagement with art. But is that too limiting a view of emotional engagement? What other kinds of emotional responses come up when we’re perceiving art?

2. When it comes to empathy specifically, are we talking about empathizing with the emotions of the characters, if there are any, or with the emotions of the artist or designer?

3. If we are talking about characters, what about a character makes us able or unable to empathize with it? Is our ability to engage with game characters different from our ability to engage with literary characters, and if so, why?

4. Does empathy with an avatar character work differently than empathy with an NPC?

5. If differences like this do exist, that may imply that we experience sudden shifts in style of emotional engagement when switching from gameplay to cutscene. If this is true, how does that affect our emotional experience of a game as a whole?

“(2) The “Heads in the Sand” Objection

‘The consequences of machines thinking would be too dreadful. Let us hope and believe that they cannot do so.’

This argument is seldom expressed quite so openly as in the form above. But it affects most of us who think about it at all. We like to believe that Man is in some subtle way superior to the rest of creation. It is best if he can be shown to be necessarily superior, for then there is no danger of him losing his commanding position. The popularity of the theological argument is clearly connected with this feeling. It is likely to be quite strong in intellectual people, since they value the power of thinking more highly than others, and are more inclined to base their belief in the superiority of Man on this power.

I do not think that this argument is sufficiently substantial to require refutation. Consolation would be more appropriate: perhaps this should be sought in the transmigration of souls.”

– Alan M. Turing, On Computing Machinery and Intelligence, 1950

“Preliminary Evaluation of the Interactive Drama Façade“, by Rachel Knickmeyer and Michael Mateas.

“Empathy: Its Ultimate and Proximate Bases”, by Stephanie D. Preston and Frans B. M. de Waal.

At the beginning of any medium’s lifespan is a period of rampant, unstructured experimentation. Creators work by trial and error, and in the absence of strategies that reliably work, they follow their own whims and interests. Over time, ideas about what does and does not work build up and are taught to future generations as rules or guidelines. For each medium, this process plays out differently, but in each case it is motivated by the need for some relatively reliable system to predict the profitability of a given venture. Once some novel venture starts making enough money, the accountants are going to start taking over.

The hope is that whatever system gets produced will be flexible enough to allow for innovation. American comic books provide a good example of how badly this can all go; in the beginning there were crime comics, horror comics, detective comics, romance comics, adventure comics, etc., all of which eventually gave way to the domination of superhero comics above all else. Innovation in mainstream comics has been an incestuous mess of deconstructing, reconstructing, recontextualizing, and rethinking superheroes, then repeating the process on whatever new ideas about superheroes came out of the previous cycle. This makes for an output which appeals to a fairly narrow segment of the audience, and has left the entire medium in the undignified state of mainly providing characters and ideas for other media in order to eke out a living at all.

On the other side of the spectrum are Hollywood movies. In 1927, just as filmmakers were starting to figure things out, sound came along and forced a violent shift in many practices over the course of only a few years. Many stars became unusable, shooting and blocking procedures had to be reconfigured to keep actors in the range of the microphone, and someone had to figure out what kind of dialogue and music people wanted in movies. The findings of the twenty-year process of figuring out what people wanted were suddenly near-useless, and quickly coming up with a new system was a matter of survival.

According to film historian Rick Altman¹, the reaction of the movie studios was to put out as many talkies as possible, observe which ones were successful, and put out even more talkies with similar thematic and structural elements in an attempt to isolate the variables that made that movie popular. The end result of this process was a remarkably well-defined system of genres. By repeating a successful movie’s elements in various configurations, the studios could analyze which of those elements were essential and which were incidental to a movie’s appeal, and to build up a series of fairly modular definitions of types of movies. This is perhaps the ideal of a reliable yet flexible system for commodifying an artistic medium; later critics would look back on Hollywood’s genre filmmakers as harboring much of the innovation and artistic success of the studio period.

Videogames have faced no such major paradigm shift in their history, but nonetheless are approaching the point at which the accountants are going to want some guarantee of results. Unfortunately, it looks like they’ve hit upon usability engineering as a good template to work from. Videogames are software, so why not evaluate game design like other types of software are evaluated?

The problem is that traditional usability engineering isn’t even working that well for other types of software anymore. The field was originally built up on the assumption that users have goals and tasks, so the best way to design software is to minimize obstacles between the user and those goals. This is fine when you’re talking about simple business software where the tasks are obvious. But more and more situations where this doesn’t apply are showing up – exploratory data mining and visualization applications where the goal is unknown, online stores that want to support idle browsing, collaborative software where multiple users may have conflicting goals or intersecting tasks, and so forth. We’re using computers for a lot more complicated and unstructured things these days. As a result, research in human factors and usability is moving away from the older task-centric point of view².

Nonetheless, the videogame designers who are showing an interest in applying usability to increase the reliability of their output seem to be absorbing the older task-centric style, if the Halo 3 team is any indication:

“Here’s the problem,” Pagulayan mutters, motioning to a computer monitor that shows us the game from the player’s perspective. He points to a bunch of grenades lying on the ground. She ought to be picking those up and using them, he says, but the grenades aren’t visible enough. “There’s a million of them, but she just missed them, dammit. She charged right in.” He shakes his head. “That’s not acceptable.”

The player has failed in the subtask of getting the grenades, which she will need to accomplish the task of finishing the level, etc. This focus on tasks isn’t surprising; after all, the new movements in human-computer interaction aren’t fully developed yet, and if you want something time-tested, task-based usability evaluation is all there really is, even if it makes an awkward fit with games.

Of course, this is only one game, and not necessarily an indicator of larger trends by any means. However, it is liable to be an enormous success, and a very high profile one at that, given the extent of its marketing. Which means that anything it does is probably going to get copied by a lot of people. If those people are looking for a good systematic way to make reliably successful games, the usability-style evaluation used by Bungie may seem pretty attractive.

And this would be a terrible idea. What usability researchers are finding now is that the task-based evaluation paradigm only works for a surprisingly narrow set of programs; likewise, task-based evaluation would produce a narrow set of games. The only obvious task in games is to finish the game. But of course, many simulation games have no end, and even in games with endings, the fun isn’t only, or even primarily, in finishing – the fun is in the process. Streamlining the process so that the gamer finishes it more smoothly is not necessarily what every gamer wants, or what any gamer wants all of the time.

The strength of Hollywood’s genre system is in its flexibility to aim at different audiences and to allow innovation. As Chris Bateman might put it, it’s tactical rather than logistical. It acknowledges the difficulty of creating a gold standard of “entertainment” that will appeal to the entire audience, and instead creates numerous kinds of entertainment that appeal to many smaller segments of the audience. And because of its modularity and its ability to get at the essence of what makes a movie enjoyable, a good genre system is also more capable of reacting when the tastes of the audience change. Musicals, for example, were hugely popular from the coming of sound to the mid-’60s. As more cynical ’70s audiences went for movies with the veneer of realism, Hollywood slowed its production of musicals, and by the ’80s and ’90s was focusing anew on big-budget screwball romantic comedies – basically musicals with the songs taken out. Now that a new generation has grown up with ubiquitous music videos, Hollywood is trying out musicals again, often aimed at teenagers and incorporating traditional elements of high school movies. Meanwhile, trendspotting critics identify brand new genres like torture porn and mumblecore, and studios obligingly release a bunch of movies under the new labels to see what makes them tick. Reaction times are pretty fast and changes are easily incorporated.

The idea of a “science of play” based on usability, though, suggests a much less flexible system. The Halo 3 article describes a process designed to create a smooth, homogenous user experience, exactly the kind of please-everyone ambition that gets you in trouble as your audience changes. It’s the equivalent of those crackpots who swear they’ve found a formula for predicting the success of a movie. Just plug in the script and go! The idea is ridiculous on the face of it, knowing just how many factors go into the box office performance of any given movie. The same applies to games, and attempts to homogenize the player experience would obscure a lot of those more subtle factors while developers should be learning from them.

It’s been nice living through the period of gaming when even the most mainstream games are filled with weird experiments and innovations, but there’s no way for that to last much longer than it already has. Already a lot of that experimentation is moving to the fringes, to indie games and small developers, which is normal. But it would be unfortunate if mainstream games were to end up like mainstream comic books, atrophied and weakened by their adherence to a single standard of entertainment. The industry needs to develop its tactics, not just one-size-fits-all standards of gameplay.

1. ”Cinema and Genre,” The Oxford History of World Cinema 
2.  A 2003 CHI panel discussed some of the new paradigms in HCI.  Additionally, experience-focused HCI seems to hold some potential for thinking about games.

Earlier this week I linked to an article from Wired entitled “Halo 3: How Microsoft Labs Invented a New Science of Play.” For all the hyperbole of the title, the gist seems to be “they’re play-testing it a whole lot.” The new science of play would appear to be the reasonably well-established science of human-computer interaction (HCI). What makes this endeavor remarkable is probably the fact that usability principles are being applied to a field which is not often thought of by either its supporters or detractors as being especially useful.

While HCI researchers have been trying to steal ideas from game design for years, under the reasoning that videogames are the class of software that people seem to enjoy interacting with the most, it seems that the interest has been rather one-way until more recently. Some researchers have been working to develop heuristics of gameplay analogous to Jakob Nielsen’s well-known usability heuristics, but being outside the industry I don’t know if these efforts have had any effect on actual practice yet. The techniques described in the Wired article, though, sound unmistakably like classic HCI evaluation methods: having the user think out loud while interacting with the software, administering subjective surveys of satisfaction and enjoyment, and recording keystrokes.

And whether or not usability is being explicitly considered in the process of game design, it’s clear from a player’s perspective that games have been getting substantially more usable since their inception. As John Harris points out in his piece on difficult games for Gamasutra, videogames have gradually transitioned from being seen as tests of skill to being seen as providers of an experience. When a game’s purpose is to put you through hell to see if you can survive it, ease of interaction is obviously the last thing on the designer’s mind. When its purpose is to immerse you in a world or storyline, there should be no barriers to that immersion in the form of excessive difficulty or awkward interaction.

The big question all this raises is this: what are we talking about when we talk about interaction? It isn’t just a matter of the interface; the Halo 3 team, at least, is also testing gameplay, level design, and mechanics with their analysis. And the changes alluded to by Harris encompass the basic philosophy behind the design of a game, not just how that design is presented to the player. I’ve been using the terms HCI, interaction, and usability pretty interchangeably throughout this post, mainly because I’m uncomfortable applying any of them to what goes on between a player and a game.

In any case, these ideas come from the analysis of software which is designed to aid in some specific set of tasks, and task orientation has been built into the fiber of HCI from the word go. The implicit assumption in the application of HCI to gaming seems to be that the task is to finish the game, or some subset of the game. After all, what else does the user do with the software that you can actually quantify? And if enjoyable finishability starts becoming the gold standard of game design, what kind of games are and are not going to get produced?

“Filmmakers proceeded blindly, with little to guide them in the way of either precedent or theory.  They did not exactly know what effects they wanted, nor, to the extent that they knew, did they all want exactly the same effects.  As a result there were many experiments – in technology, in dramaturgy, in narrative, in set design – some of which proved to have no sequel.”

Geoffrey Nowell-Smith, ”The Heyday of the Silents,” in The Oxford History of World Cinema

Halo 3: How Microsoft Invented a New Science of Play, by Clive Thompson for Wired.

Game Design Essentials: 20 Difficult Games, by John Harris for Gamasutra.

Man, talk about videogame logic!

Earlier this week I linked to an article from The New York Times, written by the sort of Times writer who says things like “to use the vernacular, I almost lost it.” Seth Schiesel, the monocle-popping writer in question, feels that emotional involvement in videogames is limited compared to other media, and that this limitation may be inherent. He suggests that the most powerful affective reactions to art come from what he calls “emotional surprise,” and that in a medium where the audience’s choice determines the outcome, this surprise is impossible.

I agree with some parts of this thesis, and I think it is important to consider what the addition of agency does to the nature of emotional engagement. What hardly ever seems to be remembered in these sweeping considerations of videogames as a medium, however, is the extent to which all games restrict the player’s choices. Even in the broadest sandbox, the mechanics of the game determine what scenarios can possibly arise, and learning the possibilities and impossibilities of a given game can indeed be surprising. What’s more, plenty of games have rigid narratives that can startle the player in the same way as stories in other media; Aeris’s ever-shocking death in Final Fantasy VII, for example. So I don’t think emotional surprise is what is at issue here. Nor do I think it’s that artistically impressive in and of itself, for that matter.

I do agree that the range of emotions that arise from actions for which you are personally responsible is very different from that which arises from observation alone. I don’t know yet what the characteristics of those two ranges are, or how much overlap there is. But the suggestion that the distanced, observational emotions of the latter are by necessity deeper than the personal, responsibility-driven emotions of the former – well, I think that requires a whole hell of a lot more justification than Schiesel provides.

Still, there’s plenty of anecdotal evidence for people responding with greater emotional fervor to movies, books, music, and the like than to videogames. What I think is really interesting about this is the apparent paradox of the more involved medium resulting in calmer emotions, while the more distanced media result in stronger emotions. Schiesel writes as though it’s normal for a golf game to be a more detached experience than a symphony, but I think that sounds completely insane. Probably true, but insane. Why do we respond more strongly to the recreated, abstracted feelings of an artist than to spontaneous feelings about our own successes and failures?

“Emotionally involving the audience is easy. Anybody can do it blindfolded, get a little kitten and have some guy wring its neck.”
– George Lucas

Even Games That Have Everything Are Still Missing Something, by Seth Schiesel for The New York Times.

More on this later!

The theme of Corvus Elrod’s Round Table for this month (which I may as a tendency take as a general monthly theme) is the role of artificial intelligence in gaming. In this context, as I understand it, AI most often refers to intelligent agents specifically; that is, the decision-making processes behind enemies, allies, social NPCs, and any other game objects meant to project an air of sentience.

I’ve had some small professional experience related to this particular field. The major recurring theme I recall from my study of the disposition of these intelligent agents is this: their human counterparts are, on the one hand, enthusiastic beyond measure to attribute motivation to every random or accidental behavior the agent displays and, on the other hand, consistently unimpressed by any and all intended behaviors.

While frustrating to the designer, this is interesting in what it suggests about how people react to a program attempting to appear intelligent. We are eager to attribute sentience to that which we don’t understand, yet quick to reject this hypothesis when an underlying mechanism peeks through. This makes some sense from an evolutionary standpoint; it is probably safest to assume that anything you don’t understand may potentially wish you ill. But the more knowledge we have about the world, the less we see spirits in it.

The simplest implication for videogames is that greater randomness in the algorithm may contribute to a greater impression of sentience – as in, say, The Sims 2, where conflicting drives and constant random rolls lead to a relatively unpredictable, and vivid, stable of NPCs. Their personalities (such as they are) are just arbitrary numbers, but that may be why they make such an easy target for the feverish imaginations of the players.

But there’s more to it than this, of course. We are equally eager to attribute sentience to characters whose motivations are drawn in a traditionally literary fashion, bypassing algorithms, such as those in Knights of the Old Republic. But this seems intuitively to be a different kind of sentience, and I’m not sure whether it carries over once your exhaustively detailed conversation ends and the battle begins. What is the relationship between AI and character? I’ll leave that question for another night.

For more posts on this topic, please visit the Gaming Blog of the Round Table home page.