Learning for Game Designers

by Clayton

1. Declarative and procedural knowledge

Note: The treatment of the psychology of learning that we will base our methods on is adapted from the work of John Anderson and colleagues at Carnegie-Mellon University. While almost nothing about psychology is uncontroversial, in my opinion Anderson's work provides the best-supported foundation for analyzing learning processes. It is used by Anderson and others as the design basis for the most successful interactive tutoring systems, and for analyzing some gamelike learning experiences. The treatment I'm giving here is gross simplification of the ideas of Anderson and colleagues.

It appears that knowledge of facts and ideas and possession of skills are embodied in different mental mechanisms. Psychologists call facts, like "the product of 2 and 3 is 6", declarative knowledge. Skills, like being able to recognize a multiplication problem and retrieve or calculate the answer, are called procedural knowledge. For simplicity, we will just refer to the two types of knowledge as facts (or ideas) and skills.

Here is what we will need to understand about facts, skills, and the differences between them, as educational game designers. Some of these things are familiar, summarized by the proverb, "Practice makes perfect."

Some are less familiar, perhaps, but not very surprising: Finally, some things are downright unintuitive, and for that reason especially important to grasp. We'll give them names: Lack of wide knowledge of PiE is responsible for widespread academic underachievement. Many people think that if they "understand" something, they have learned it. Sadly, they often have learned it only temporarily.

SoS is why it is important to distinguish facts and ideas from skills. Intuitively, most of believe that if we can do one thing, we can probably do something similar. But a lot of evidence demonstrates that that is simply not true. Here are some examples. Data from standardized tests show that when addition problems are presented horizontally performance is worse than for the same problems presented vertically. The same tests show that many kids who can add two columns of figures cannot add three. Such data contradict the common idea that there is a skill of "adding numbers", and make us replace it with the idea that there is a bundle of very specific forms of this skill. Researchers in Brazil located schoolchildren who had jobs selling produce in a market. By making particular purchases they verified that the children could calculate certain prices (5 melons at such and such a price) and make change for certain amounts. They then arranged for these children to be given the very same arithmetic problems on tests at school. Performance on the school tests was poor: the skills of working with money in the market place did not apply in the school situation.

The situation is better for facts. If you learn the fact that 11 is greater than 9, you can apply that fact in any setting in which you need it. But notice that a specific fact like this is only useful in some situations.

How about the more general idea, two digit numbers are bigger than one digit numbers, or even, numbers with more digits are bigger? These ideas are applicable to more situations, but how are going to get them? You may think that these general ideas would be learned just by working with more specific facts. But NaG says this won't happen.

Like Sos, NaG violates many people's theory of knowledge. For example, many people think that if you solve a lot of puzzles, you will get smarter. The evidence is that just solving a lot of puzzles does not help you, except that you get better at the particular kinds of puzzles your work on. To develop your general ability to solve puzzles you have to actively try to compare the puzzles and think actively about the methods you try and their effectiveness.

2. How learning can occur in playing a game

Let's consider how these ideas about learning can help us understand learning from playing a game. We'll use the board game Monopoly as the setting for this discussion. If you haven't played Monopoly, discuss this section of the notes with a classmate who has. It will be interesting for both of you.

Playing the game 0: interpreting the instructions

When you first played the game, you may have done so by working through step by step instructions that told you how to play, or a friend may have told you, step by step, what to do (roll the dice, count off that number of spaces clockwise around the board, find out who owns that property, etc. etc.) What's happening here is that skills you have as a reader or speaker of English enable you to interpret verbal instructions in the same way, conceptually, as an interpreter for BASIC executes BASIC code. You don't start out with a skill of playing Monopoly, but rather the skill of following instructions in English.

Learning the game instructions I: building facts that represent the instructions

Early in learning you may have to refer back to the printed instructions, or ask your friend to remind you what to do. But a side effect of the execution of your interpreter skill is that it builds facts in your head, and over time these form a representation of how to play the game. As this develops, you no longer have to refer to the printed instructions or ask your friend. But you still don't have a skill that can play Monopoly: you are still playing by interpreting the instructions, with your interpreter skill running off information in your head rather than in the outside world.

Learning the game instructions II: building skill

Once your interpreter skills are feeding off facts in your head, you start building a skill that can play Monopoly directly, without referring to the facts.This skill is very weak at first, but if you keep playing it will become strong enough to compete with your interpreter skills. Once that happens, it will enable you to play more quickly: you won't be thinking "what do I do now" but rather just rolling the dice when it's your turn, and so on.

Learning game facts I: facts in your head

If you land on St James Place, and it's unowned, you have to find out its price. You do this by finding the deed and reading the price on the deed. If you play a lot, you might build up a fact in your head of what St James costs. In reality, I would say that the statistics of the game, how often you need that particular fact, are such that very few people actually do build a usable representation of that fact. The ACT-R theory would allow one to make quantitative predictions about that, and would say that more people will know the price of the railroads (they are all the same) than of any of the single properties.

Note also that certain prices may be learned not because you land on the property more often, but because something makes them especially important, so that you think about them a lot. I think it is for this reason that I remember that Boardwalk costs $400 and Park Place $350. The process of thinking about something forces retrieval of facts and strengthens them.

Learning game facts II: skills

Conceivably someone who plays a huge amount (a tournament player, if there are such people) would actually develop skills that implicitly embed the price of St James, so that they would decide to buy St James or not based only on a check on their money supply, with no reference, even mentally, to the price.

Learning decision rules I: ideas

Playing Monopoly involves making decisions: Do I buy this? Do I mortgage that? Do I buy a house or hotel now? Initially you have to make these decisions by consulting whatever general decision policies your preexisting skills can find. For example, you might have something stashed in your head about the desirability of debt ("Neither a borrower or a lender be", say). Your reasoning skills might use that idea as the basis for a decision not to buy St James, because you don't have enough money to buy it without mortgaging something. Note that a good deal of knowledge, ideas and presumably skills, about money and how it works, is brought into your thinking about decisions like this. For young children, who lack this background knowledge, Monopoly is presumably an occasion to develop knowledge about money and how to reason about it, that they do not already have. For people like you who know about money, playing the game means building up ideas specifically about Monopoly ("the cheap properties aren't worth developing", or "the cheap properties are great because you can build hotels on them early".) As the example indicates, it's not clear how much of this "knowledge" is really true, but you're going to build it up as best you can anyway.

Learning decision rules II: skills

The aforementioned real or hypothetical tournament player will eventually convert some of their fact and idea knowledge about Monopoly decisions into specialized skill. They may buy Connecticut Ave when they land on it early in the game without thinking about it all, so to speak. A visible symptom of this will be the sheer speed of their play, as contrasted with the play of somebody who is still trying to work out whether buying the cheap properties is a good idea or not, and in the process is trying to remember what happened in the last game when Pat bought Connecticut Ave.

3. The educational value of game learning

Now let's consider the value of these forms of learning. As we've just seen, if you have the goal of playing Monopoly fast, it's all valuable. All along the way, your mental machinery is replacing slow processes by faster ones: don't read the instructions, that's slow: just remember them; don't look at the deed, that's slow: just remember the price; don't think about rolling the dice: just do it when it's time; and eventually, don't think about whether or not to buy Connecticut Ave: just do it (or not). Also, the speed and reliability with which you can access relevant facts and ideas increases.

If you want to play Monopoly well, not just fast, the theory has much less to say about that, and I think a little reflection will convince us that the theory is on target there. I think it happens often that people play something over and over again without actually becoming very good at it, meaning that our mental machinery, while we can rely on it to make us faster, offers no guarantee that we will get better. The machinery does favor skills that "work" over those that don't, but the real problem often is, they don't work to produce the right thing, because we don't know what the right thing is.

I'd venture that some people build up a lot of factual knowledge and ideas, somewhat analogous to the knowledge that allows us to read and interpret instructions in English, that allows some people to do a better job of analyzing games and game situations than others. Presumably some of this even becomes converted to skills for people who play a lot of games. But NaG says this only happens if you work at it, not as an automatic result of playing a lot.

So much for Monopoly, fast or slow, successful player or not.

But notice that, educationally, we don't care about any of this!

Playing Monopoly has no life value (well, except for those tournament players, if there are any.) More generally, for none of our games will we care whether players get good or not at the games themselves. Rather, we're hoping that some of what they learn will have value in some other setting. How can this happen?

Mostly, it won't.

We need to break this down and consider facts and skills separately, and here it's going to be simpler to consider skills first.

As SoS says, the basic, sad, fact about skills is that they are highly specific, meaning that they rarely transfer between different situations. "Transfer" is a technical term in the theory of learning: suppose you learn something A, and then you learn something else B, while somebody else just learns B. If it's easier for you to learn B than the other person (other things being equal) we say there is positive transfer from A to B. Sometimes you are actually worse off than the other person; then we say there is negative transfer from A to B. The robust finding about skill is that it only transfers when it can be used without change: in terms of the theory, learning A helps you to learn B only when parts of the skill needed for A are exactly the same as some of those needed for B. (You get negative transfer when your A skill applies when you're trying to do B, but it does the wrong things.)

It's not hard to imagine situations in which transfer of skill can occur from a game to some real life situation. Considering Monopoly, it's plausible that playing it helps young children develop and strengthen some skills that do transfer to many other situations. Examples may include counting squares on the board, counting money, and making change.

But it's also easy to be misled, by false intuitions about the nature of knowledge, into imagining that transfer of skill will occur from a game when in fact it will not. Many people suppose that it must be valuable, in some intellectual way, to play a game like Myst, that involves solving various puzzles, because the game must develop puzzle solving skill that will help in solving other problems you encounter in the future. The evidence is that this does not happen, because the skills used in solving different puzzles are different. Another example: many people think chess players must be "smart", that the skills developed in playing chess are somehow valuable in other settings. The evidence is that this is just not so; being good at chess means being good at chess, period.

Happily, the transfer picture, and hence the likelihood of educational value, for facts and ideas is very different than for procedural knowledge. Your factual knowledge is not linked to any particular task setting: it's available in any situation in which you think of it.

As educational game designers, we still have to worry. The question becomes, is the factual knowledge I obtain relevant to some other situation? Thinking about Monopoly, for most of the declarative knowledge we discussed the answer is, obviously not. Learning the price of St James Place is of no possible use outside the game. The same must be said of your knowledge of the instructions and rules of the game.

So, is there any educational value in Monopoly? If there is, it lies in the factual knowledge and ideas we may develop in the course of developing decision policies in the game. We may learn some consequences of borrowing, for example, in thinking about mortgage decisions in Monopoly. Another place these considerations may be reinforced is in connection with the price penalties associated with selling off houses to raise funds. I would assume that some players, at least, develop some ideas from Monopoly about keeping cash reserves.

We have to ask, though, is what I learn about cash reserves in Monopoly really true, in the sense that it is applicable and valid in other situations? What do you think? This is a serious issue in using simulations in education!

There's one more issue we have to consider about factual knowledge: level of generality. Do I learn, "Don't spend money, if you can avoid it, if you have less than $500," or do I learn, "Don't spend money, if you can avoid it, if it increases the likelihood that you'll need to borrow"? Presumably, the latter idea is more valuable in other situations than the former, which can hardly make sense outside Monopoly.

As NaG tells us, generalizations develop when (a) people are presented with comparable but differing situations, AND (b) they form the goal of forming a generalization from them. Both (a) and (b) have to be there: you need the raw material from which to form the generalization, AND you have to be trying to do it. Generalizations don't form spontaneously.

A challenge for us as game designers is, how can we induce generalization goals in players?

One way, not very satisfying to me, but perhaps often the best we can do, is to rely on structure outside the game to do this. If a game is embedded in a school lesson, the school lesson can require students to think directly about the relationship between situations encountered in the game and situations somewhere "in real life". Notice that we have to follow this up with activities in which the new generalizations are repeatedly retrieved. Because they weren't formed in the game, they won't be practiced in the game. And (as PiE tells us) if they aren't practiced, they won't be strengthened enough to be retrieved in the future. So you have to come up with some motivation, outside the game, for practicing with the generalization. Sounds just like work.

What can we do in a standalone game? Here's one approach; perhaps you can devise others. In the game, the player confronts a sequence of situations in which the same body of factual knowledge is useful, PROVIDED that the knowledge is stated at the right level of generality. This brings the generalization goal into the game itself, not leaving it for an outside lesson. It's a bit labored, but one could illustrate the idea in a variant of Monopoly in which the price and income levels vary from round to round. You'd have to frame your policy about cash reserves in more general terms than "keep $500 on hand." The motivation for developing the generalization comes from the game, not from duty or coercion.

One last point about factual knowledge. One of our knocks on Monopoly as an educational game was that there's no value in knowing the price of St James Place. (Incidentally, nobody thinks of Monopoly as an educational game, but it was actually derived from— some say ripped off from— an educational game called the Landlord's Game; see http://inventors.about.com/library/weekly/aa121997.htm .) But what if the properties in Monopoly were actual, and their prices were actual, and you were training real estate appraisers? Then there could be some value in the facts you needed to play the game. I remember playing a game called "Game of the States" as a kid, in which you had to learn what commodities originated in what states. I suppose I learned something from it. So you might possibly want to design a game in which some body of actual fact is needed to play the game (Latin and Greek roots of words, say). If the body of fact was important and hard to learn some other way, you could have a winner. The Carmen San Diego game was invented by (now Boulderite) Gary Carlston following just this line of thinking.

4. Analyzing a game

Let's pull this discussion together in a template that we can apply to analyzing (and eventually designing) a games. The purpose of the template is to help us identify what knowledge, ideas and skills, someone will learn from playing it. We'll focus on what they will learn that has value outside the game ("real world" facts, ideas, and skills. While we are at it, we'll note what prerequisite skills, facts, and ideas are needed. These are important in fitting a game to an audience (for example, few sixth graders can take derivatives, and a game that requires that as a prerequisite isn't a good choice for them.)

Template

Prequisite skills: To play this game, you already have to have the skill(s) of ____________________.

Prerequisite facts and ideas: To play this game, you already have to know __________________________.

Skills: Playing this game involves the skill of _____________________, which is a real-world skill. Playing the game provides extensive practice with the skill (yes or no.) The game includes resources, such as instructions or demonstrations, from which a factual representation of this skill can be formed (yes or no). These resources are_____________________________.

(repeat for other skills)

Specific factual knowledge: Playing this game involves knowing these important real-world facts:________________________. Within the game, access to these facts or ideas is provided by ________________. The facts will be used frequently in the game (yes or no).

(repeat for other facts)

General ideas: Playing this game involves knowing these important generalizations:_________________________________________. Forming these generalizations is supported in the game by presentation of the comparable but contrasting situations or entities, in this way_________________________________. These ideas will be used frequently in the game (yes or no.)

(repeat for other ideas)

Efficiency: Processing the above knowledge represents a (choose one) negligible, minor, major, dominant proportion of the time players spend on the game.

(We haven't discussed this, but obviously using Monopoly as a way of giving students practice calculating percentages is dumb. Playing Parcheesi as a way of giving kids practice counting is not so dumb: there's not much time going into anything else in that game.)

5. Analysis Example

Here's my attempt to fill in the template for the Leap Fractal Game you looked at for our Fun discussion. Prequisite skills: To play this game, you already have to have the skill(s) of ---recognizing buttons, clicking buttons, interpreting the score feedback---.

Prerequisite facts and ideas: To play this game, you already have to know ---nothing, really---.

Skills: Playing this game involves the skill of estimating the midpoint between two points, which is a real-world skill. Playing the game provides extensive practice with the skill ([YES] or no.) The game includes resources, such as instructions or demonstrations, from which a factual representation of this skill can be formed (yes or[NO]). These resources are_____________________________.

(repeat for other skills) NA

Specific factual knowledge: Playing this game involves knowing these important real-world facts:______NA____________. Within the game, access to these facts or ideas is provided by ________________. The facts will be used frequently in the game (yes or no).

(repeat for other facts)

General ideas: Playing this game involves knowing these important generalizations:___you can chain backwards from the goal to form subgoals____________. Forming these generalizations is supported in the game by presentation of the comparable but contrasting situations or entities, in this way__similar but different starting situations encourage this over a rote pattern__. These ideas will be used frequently in the game (yes or no.)

(repeat for other ideas)

Efficiency: Processing the above knowledge represents a (choose one) negligible, minor, [MAJOR], dominant proportion of the time players spend on the game.

I'm not entirely happy with this analyis, though I think it does capture the main fact about Leap Fractal: Although it is supposed to be educational, I don't think one learns much from it, or at least, that I didn't learn anything, apart from getting a little practice at estimating midpoints, and starting to develop some strategy (that's the chaining back part). In class we can consider whether some of you got things from the game that I missed.

I didn't play very long, and I certainly didn't develop an 'algorithm' as the rules suggest is possible. Had I done so, my analysis would be different, and perhaps more optimistic. But as it stands, my experience is that one can play the game without learning much, and the analysis captures that.

There's another consideration that I think this game brings out. Suppose I had played long enough to find the algorithm for optimal play. Would I have learned anything about fractals? I would say, only if there were some discussion that linked the strategy (and the game) to fractals. I believe there are such links, but unless they are somehow pointed out, I don't believe even people who play a lot will get them.

I think this is not an uncommon situation. You can take an idea and map it into a game, and the game may be entertaining. But unless one can somehow map back from the game to idea, not much has been gained, educationally.

A real-life illustration of this is the game Tic-Tac-Twice, by CU CS prof Andrzej Ehrenfeucht. The game is based on some interesting theory, and there is an interesting analysis of best play (I've been told.) But as the game is sold by its commercial publisher, there's no hint of any of this. It's just a game (a very good one, actually, but not educational as packaged.)

6. Design Example

Here's my attempt to use the template to analyze the circuits game I am working on.

Recall that in outline there is a complex circuit with a lot of switches, and current can flow to either of two timers, the player's timer and that of a robot opponent. The player and the robot opponent switch switches in an effort to make their timer run and their opponent's timer stop. Prequisite skills: To play this game, you already have to have the skill(s) of __clicking on switches in the diagram to switch them__.

Prerequisite facts and ideas: To play this game, you already have to know __nothing_____.

Skills: Playing this game involves the skill of _tracing current flow in a circuit___, which is a real-world skill. Playing the game provides extensive practice with the skill ([YES] or no.) The game includes resources, such as instructions or demonstrations, from which a factual representation of this skill can be formed (yes or no). These resources are__not fully worked out, but I have a general idea that one can turn on a view that shows where the current is going. Maybe doing so makes you lose a turn___.

(repeat for other skills)

Specific factual knowledge: Playing this game involves knowing these important real-world facts:__NA, I think__. Within the game, access to these facts or ideas is provided by ________________. The facts will be used frequently in the game (yes or no).

(repeat for other facts)

General ideas: Playing this game involves knowing these important generalizations:__some things about series and parallel circtuits________________. Forming these generalizations is supported in the game by presentation of the comparable but contrasting situations or entities, in this way__I'll try to develop levels that highlight these, in that parallel circuits provide an opportunity to make a move that is hard to block, while series circuits are easy to block [thinking about this suggests a rule I'll probably need: it shouldn't be legal to switch the switch your opponent switched last (kind of like the ko rule in Go, for those you know that game]______. These ideas will be used frequently in the game (yes or no.)

(repeat for other ideas)

Efficiency: Processing the above knowledge represents a (choose one) negligible, minor, [MAJOR], dominant proportion of the time players spend on the game.

I feel pretty good about this analysis. Apart from suggesting a refinement in the rules, the analysis brings out the fact that I need to work more on what education people call "scaffolding", a mode of presentation in the game that helps player see how the circuits work.

Exercise

(You should spend about three hours reading these notes and working on the exercise.)

Here is a description of a gamelet called "Tic Tac Toe products" (we'll abbreviate it TTTP). I haven't been able to determine who invented it. I learned about it from Andee Rubin, who works on educational software and related issues at TERC, a thinktank in Cambridge, MA. It's a two-player game that works like this...

There's a 6x6 grid, labeled with numbers, like this, with the numbers from 1-9 written in a row underneath:

01 02 03 04 05 06

07 08 09 10 12 14

15 16 18 20 21 24

25 27 28 30 32 35

36 40 42 45 48 49

54 56 63 64 72 81

1 2 3 4 5 6 7 8 9

There are two markers that can be used to pick out two numbers in the row (paper clips are often used.)

The first player puts one of the markers on any desired number in the row. The second player then places the other marker on any desired number in the row, and then puts an O in the square of the grid that contains the product of the two marked numbers (note that both markers can be on the same number.)

The first player then must move one of the markers to any new number, and then puts an X in the square of the grid that contains the product of the two marked numbers.

The second player does the same thing, marking a square with O. Neither player can move the markers in such a way that the product formed has already been used.

The winner is the first player to get 4 of their markers in a row, vertically, horizontally, or diagonally. If a player cannot move the game is a draw.

Play this game a few times with someone. You'll soon see in playing the game yourself that there is a strategic aspect of play beyond the mechanics. See if you can form a picture of what ideas, at what generality, you build up in playing.

Q1. Does playing the game require factoring small numbers? Explain.

Q2. Fill out the learning analysis template for TTTP, based on your experience with it. (Note: be sure you turn in the complete template, not just the things you would put in the blanks. Much easier for us (and you) to read.)

Q3. Can you think of any ways to increase the learning in TTTP, either by changing the rules or changing the presentation?

Submision instructions:Email your response to clayton.lewis at colorado.edu by 11:59 PM on Wednesday, Feb. 8. Please include your name in the subject line of your message, and (if you attach a document) put your name somewhere in the filename of the document.