The Art of How

Form and content in the visual field of videogames

There is a fairly solid consensus now in cognitive science that visual information (and perhaps information from our other senses) takes two distinct pathways through the brain. The ventral pathway runs along the lower part of the brain, and carries information about things. This pathway focuses on what objects are: that is, what they’re called, what categories they fall under, and what they remind us of. It connects up with the language areas of the brain and those most closely associated with memory. It’s commonly called the what pathway.

The dorsal pathway, on the other hand, runs along the upper part of the brain and carries information about space. This pathway keeps track of where things are and the general spatial configuration of the scene we’re looking at. More recently it has been suggested that the ventral pathway is also involved in understanding the functional aspects of a scene. So instead of seeing what an object is, this pathway sees how an object can be approached, grasped, used, etc. It is traditionally referred to as the where pathway, or the how pathway if its functional side is being brought to attention.

Now, any fan of art criticism may find this all highly reminiscent of the classic form vs. content debate. Throughout history, art movements have concerned themselves with the balance between the content of a work (what it’s about) and its form (how it presents that content). In the past century alone, modernism first declared victory for the primacy of form, then postmodernism (inasmuch as postmodernism was liable to declare anything) declared the distinction irrelevant altogether.

It seems reasonable to suppose that this ancient debate has its origins in our cognitive architecture. Semantic knowledge, recognition, and attention to objects and events seem like a fair description of what goes on in understanding the content of a piece. Likewise, form can be thought of in many arts as the spatial arrangement of content and, more generally, the structure of a work, which might be more readily understood by the part of our brains interested in function.

But if indeed the distinction is so ingrained in the way we think, why should there be a debate? There is little evidence from neuroscience that it makes sense to think of one pathway as naturally dominant over the other. One possible reason is just that criticism is written, and – since the what pathway links up more directly with the language center – content is neurologically easier to write about. This may allow content-driven criticism to dominate discourse when criticism is left to its own devices, requiring an occasional corrective from strong formalism.

Which brings us to the videogame, an art form with a serious content problem. No medium that’s cropped up in fairly recent human history (that I can think of) so steadfastly favors the how pathway over the what. Here’s a small example of what I mean, in two screenshots.

The first is from The Legend of Zelda: Twilight Princess and the second is from The Elder Scrolls: Oblivion. In their current static form, the two scenes look roughly similar, and contain pretty much the same objects: paths, rocks, trees, a dashing rogue with some sweet weaponry. Since the objects in the two scenes correspond, one is hard-pressed to find major differences in their content.

But when you’re actually playing the games, you see the two scenes very differently. Specifically, I’m concerned with the raised rock formation to the center-right of each schene. In Twilight Princess, with an avatar who can’t jump upwards, this area is a barrier and only meant to divide the path. In Oblivion, where avatar movement is less restricted, those rocks can be scaled and used to, say, avoid an enemy or scan your surroundings. And my intuition is that this functional difference overrides the semantic similarity of the two objects. When you’re in the middle of Twilight Princess, you don’t see the rock as a rock, but as a barrier. And in Oblivion, you see it as a platform – provided, of course, that your Acrobatics skill is sufficiently high.

This may seem like a trivial point, but I think it’s worth considering. For all the superficial similarities between the visual appearance of a movie and that of a game, the viewer sees one as a collection of objects and the other as a field of affordances. In watching a movie, it’s the content that most immediately leaps out at us; what people naturally remember about a movie are the characters and the story (what happened). In playing a game, it’s the form that stands out, and what people remember are the mechanics (how the game worked). Perhaps this is why game designers trying to convey content have traditionally resorted to stopping everything and showing you a movie for a while.

This all may prove to be a lasting problem for game criticism and the acceptance of videogames as an art form in general. Critical language has always struggled to convey form in a way that’s both meaningful and coherent. How do you talk about a medium where form prevails? Of course, there are other arts where function takes precedence over content – architecture has been mentioned in relation to games before, but fashion, cuisine, and design could all be seen this way as well. It’s possible that these disciplines may provide more appropriate concepts and terms.

One significant difference, though, is that all of these more traditional functional arts are built on a core of tending to fairly basic human needs, which may provide an essential critical foundation that perfectly useless videogames lack. No matter how brilliant your theoretical foundations, if a dish tastes bad the food critic will send it back, and if you send a model down the runway naked you have performed an art prank, not fashion design. In the case of games, does fun provide a similar foundation? This is a question in need of further study, I think.