@andrewchen

Get the newsletter · 2018 essays (PDF) · Featured · Recent

Are people like lab rats? Using reward schedules to drive engagement

Why do users get addicted to games?
A couple months ago in BBC there was an article called Korean Man Dies After Games Session:

A South Korean man has died after reportedly playing an online computer game for 50 hours with few breaks. The 28-year-old man collapsed after playing the game
Starcraft at an internet cafe in the city of Taegu, according to South
Korean authorities.
The man had not slept properly, and had eaten very little during his marathon session, said police.

When you read this stuff, it’s not hard to see the parallels to B. F. Skinner’s work on the so-called “Skinner Box” which trained poor little lab rats based on different reward schedules. In general, the Wikipedia page on reinforcement is quite enlightening.

And in fact, the entire MMOG = Skinner Box meme is an old one. Just look at the Google results which mention them all over the place.

In fact, for anyone studying user engagement, you might say that:

The science of user engagement = The science of addiction

That is, if you can figure out what makes people get addicted to gambling, video games, and other “psychological” addictions (rather than physical ones), you might be able to port that over to your shiny new Web 2.0 website.

The Science of Addiction
First, let’s jump into “reward schedules” which describe how often (and how frequently) a user is made to feel “good.” You can make people feel good in a bunch of different ways, such as the ones listed below:

  • Currency rewards: the acquisition of a game resource that can be spent represents a fairly universal reward system… giving the player shops to spend currency rewards can be effective, provided there is plenty in the shops to choose from. (Note that the shop can be a ‘meta-shop’ – it need not be a literal shop in
    the game world).
  • Rank Rewards: like currency rewards, but ratcheted – the player gains benefits from acquiring points towards an eventual step up in rank. The classic example is level in a class and level RPG, although in video games, Elite’s (entirely cosmetic) Rank system demonstrates that a Rank reward can motivate even without mechanical
    benefits. A draw for Type 1 Conqueror and Type 2 Manager players if expressed in verbal terms, but if the ‘Rank up’ is accompanied by sufficient fanfare its appeal can be more universal.
  • Mechanical Rewards: such as increases in stats that the player can feel the effect of. Highly motivating for many players – but the mechanical increases must maintain relevance to the play. Effective for Type 2 Manager and Type 1 Conqueror players in particular.
  • Narrative rewards: a little narrative exposition is effective for certain players as a reward. A cut scene can be a bigger reward than dialogue – when used well. But overlong or irrelevant cut scenes quickly become devalued. Effective for Type 3 Wanderer and Type 4 Participant players in particular.
  • Emotional rewards: related to the above, but applicable when the player feels they have done something for someone in the game. Animal Crossing’s present giving, for instance. A draw for Type 4 Participant players.
  • New Toys: anything new that can be experimented with is a ‘new toy’. Although primarily a mimicry reward, there may be mechanical benefits of well – a new weapon in an FPS is a new toy with mechanical rewards, for instance. Especially of value to Type 3 Wanderer players.
  • New Places: like new toys, new places are a mimicry reward for players driven to explore (a common drive!). Especially of value to Type 3 Wanderer and Type 1 Conqueror players.
  • Completeness: perhaps only a drive for the Type 1 Conqueror player (or the Rational player), achieving completeness (chasing 100% for instance) can be a reward in itself.
  • Victory: defeating a challenging foe (or a boss) is purely agonistic reward, especially appealing to Type 1 Conqueror players.

(This is stolen/paraphrased from the blog Only a Game)

As you can see, the term “reward” is used very broadly. In the case of a site like HotOrNot, you might even argue that the post-rating screen that shows you someone’s score and thus validates/invalidates your thinking is a “reward.”

In the games industry, people have spent a lot of time talking about reward schedules. After all, too many rewards, and the experience might peak too quickly and seem boring. And not enough rewards, the site might seem boring.

This Wikipedia article already goes into a pretty deep discussion of the different reward schedules, but just to break down some examples quickly:

  • Fixed interval: Every X minutes, reward the user
  • Variable interval: Randomly reward the user, but with average interval X minutes
  • Fixed ratio: Every X reponses from the user, reward them
  • Variable ratio: Randomly reward the user, with average X responses to trigger it

Variable ratio schedules will pwn you
In the analysis, the article further discusses:

  • Ratio schedules produce higher rates of responding than interval schedules
  • Variable schedules produce higher rates and greater resistance to extinction than most fixed schedules
  • The variable ratio schedule produces both the highest rate of
    responding and the greatest resistance to extinction (an example would
    be the behavior of gamblers at slot machines)

As stated above, “variable ratio schedules” are the most effective in getting lab rats engaged. And of course, slot machines are invoked as a great example of highly addictive experiences. With a slot machine, you don’t always win (that’d be too predictable and boring) but you win enough of the time to keep you in the loop. Furthermore, you sometimes win big, but more frequently, you win small, which keeps you interest level.

So how do you apply this to web applications?
From this perspective, you can look at web applications in a completely new light. You end up with two categories to think about:

  • Actions: Viewing, commenting, rating, friending, messaging, etc.
  • Rewards: New features, feedback/responses, messages from friends, compliments, new content, etc.

So from a user experience standpoint, the “out of box” experience of trying a website for the first time should be one where you can try stuff out, get a reward randomly, try new things, and get more rewards. And you need to be able to get into the reward schedule from the FIRST SESSION.

The other important angle that games are great at developing is the idea of progression, where as you use the site more, your rewards change through leveling up. Similarly, in MySpace (arguably one of the most addictive sites created recently), expert users use the site very differently than novices and receive different kinds of rewards. For example, they might receive more compliments from friends over their awesomely pimped out profile instead of feeling good just from having someone message them back.

So to summarize, looking at addiction implies a couple interesting questions:

  • How do you think about your site as a series of action/responses that span over time, not a collection of features?
  • What are the actions on the site? And what are the rewards?
  • How do you think of your reward system, and how do you get the rewards to progress and expand over time?
  • How do you break down the “out of box” experience with users, and what action/reward loops immediately get them hooked?
  • Why are lab rats so cute?

Why functional apps are often boring
It’s also clear from this that it’s easy to do things the “wrong way” and make your site completely boring and unengaging. From this reward schedule context, you can do it by simply removing rewards from your system. Or if you have rewards, make the action/reward loop really long.

Here are some examples of bad things:

  • When the user executes an action, don’t reward them
  • Don’t tie rewards to actions, just tie them to some time interval
  • Make things predictable and reduce randomness

Sound bad?

Well, it turns out that if you think of your product as a collection of features that functionally accomplish something, it’s doesn’t sound so bad, because you are trying to ALWAYS deliver some solution to a problem, not introduce randomness into the equation. You’re also often letting the functional result (seeing a resume, reading a message, scanning through a search engine result page) be the end reward, and you might not try to stick intermediate rewards in the mix.

That’s why, when I look at something like LinkedIn, the site has a ton of potential but is completely not engaging at all. First off, when I do things on the site, there’s no possible way for me to get a reward except days later when people write me back or approve my request. It’s also completely functional and deterministic, and I’ve never been surprised on the site. Compare that to MySpace or Facebook, where at least I’ll be amused by some new layout or application or a funny joke someone wrote on The Wall.

Have fun!
Hopefully this was an interesting discussion – there’s a lot to be learned from psychology and game design in web apps, and I’m not the only person following this. I’d encourage you guys to read more from Daniel Cook, Amy Jo Kim and also Raph Koster.

PS. Get new updates/analysis on tech and startups

I write a high-quality, weekly newsletter covering what's happening in Silicon Valley, focused on startups, marketing, and mobile.

Views expressed in “content” (including posts, podcasts, videos) linked on this website or posted in social media and other platforms (collectively, “content distribution outlets”) are my own and are not the views of AH Capital Management, L.L.C. (“a16z”) or its respective affiliates. AH Capital Management is an investment adviser registered with the Securities and Exchange Commission. Registration as an investment adviser does not imply any special skill or training. The posts are not directed to any investors or potential investors, and do not constitute an offer to sell -- or a solicitation of an offer to buy -- any securities, and may not be used or relied upon in evaluating the merits of any investment.

The content should not be construed as or relied upon in any manner as investment, legal, tax, or other advice. You should consult your own advisers as to legal, business, tax, and other related matters concerning any investment. Any projections, estimates, forecasts, targets, prospects and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Any charts provided here are for informational purposes only, and should not be relied upon when making any investment decision. Certain information contained in here has been obtained from third-party sources. While taken from sources believed to be reliable, I have not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation. The content speaks only as of the date indicated.

Under no circumstances should any posts or other information provided on this website -- or on associated content distribution outlets -- be construed as an offer soliciting the purchase or sale of any security or interest in any pooled investment vehicle sponsored, discussed, or mentioned by a16z personnel. Nor should it be construed as an offer to provide investment advisory services; an offer to invest in an a16z-managed pooled investment vehicle will be made separately and only by means of the confidential offering documents of the specific pooled investment vehicles -- which should be read in their entirety, and only to those who, among other requirements, meet certain qualifications under federal securities laws. Such investors, defined as accredited investors and qualified purchasers, are generally deemed capable of evaluating the merits and risks of prospective investments and financial matters. There can be no assurances that a16z’s investment objectives will be achieved or investment strategies will be successful. Any investment in a vehicle managed by a16z involves a high degree of risk including the risk that the entire amount invested is lost. Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z and there can be no assurance that the investments will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by a16z is available at https://a16z.com/investments/. Excluded from this list are investments (and certain publicly traded cryptocurrencies/ digital assets) for which the issuer has not provided permission for a16z to disclose publicly. Past results of Andreessen Horowitz’s investments, pooled investment vehicles, or investment strategies are not necessarily indicative of future results. Please see https://a16z.com/disclosures for additional important information.