Sign up for my email newsletter

Get new updates, usually once a week – it features long-form essays on what’s going on here in Silicon Valley.

I’ve written 550+ essays which have been featured and quoted in The New York Times, Fortune, Wired, and WSJ. The topics range from mobile product design to fundraising to “growth hacking.”

Thanks for reading. -Andrew

Close

@andrewchen

Subscribe · Featured · Recent essays

Whenever ad networks talk about their “targeting” remember the Netflix prize

A quick rant:

Every time you talk to an ad network or leadgen network or whatever, if you ask what their differentiation is they will say “targeting.” That’s probably wrong, and let me tell you why, based on the recent announcement of the Netflix prize winners:

Netflix was able to wring three years of research to nudge its recommendation algorithm up 10.5 percent, at a cost of $1 million in prize money — a stunning feat on its own.

This means if you combine dozens of the best machine learning people in the world, some of the cleanest datasets, you get a measly 10.5% increase. Compare this to starting a new ad network where you end up with noisy datasets, lots of crappy traffic, and a small team looking at the problem – that’s not an easy path to disruptive change. In general, 10% is not a big enough number to counteract the other economic drivers in the ad market, which revolves around better deal terms, a larger selection of advertisers, better ad inventory, etc.

I would guess that you need a number closer to 50% lift or higher in order for an upstart to dramatically change the ad landscape and neutralize the weapons of the mass of ad network players.

I think disruptive change will come not from algorithms, but rather two other areas:

  • Better ad inventory: New websites and mechanics emerge all the time, and who knows what happens when you put ads on them? It was clear, until they tried it, that with the right ads search can be >30% clickthrough rates or more, which is unheard of.
  • Better data: The other big opportunity is in using specialized data to drive your algorithms – rather than basing everything off of domains, cookies, and ad impressions like everyone else, there may be ways to extend the targeting to unique datasets that no one has access to. This is what’s happening in the world of retargeting.

The Netflix prize also included people adding in additional data, and that’s factored into the 10.5% improvement. Anyway, the point is, increasing performance on stuff like this is very hard, so when an ad network tells you about their targeting, you should push them instead on their revenue split ;-)

Like this post?
Get new updates via newsletter..

  • jamesshamenski

    Hi Andrew,
    I agree that disruption is bound to happen around inventory and data.

    One more point I'd add is around online branding campaigns. Today, most ads online are valued on conversion. The other side of advertising is around branding campaigns. With the right approach, those branding dollars being spent in Magazines and TV will come online. This represents the largest portion of the advertising market which is yet to be disrupted.

  • mattmaroon

    This is a touch disingenuous. BellKor didn't get a 10.5% increase over random recommendations, or ones made by me (especially since my recommendation to anyone who liked Michael Bay movies would be to off themselves as painfully as possible). They got a 10.5% increase over a recommendation engine built by highly-paid and probably highly-talented Netflix engineers. I'm guessing that Netflix's Cinematch was so state-of-the-art that a 10.5% improvement over it is quite impressive. I'd be surprised if any ad-networks, which generally aren't as engineering-focused as NFLX, are starting from such a baseline.

  • http://andrewchen.typepad.com Andrew Chen

    There are about 300 ad networks, and people know it's a massive $500B
    industry (offline+online). So there's a lot of very smart PhDs and geeks
    chasing it, probably thousands of talented people total – I'd argue the ad
    network industry is far more efficient than Netflix, no matter how smart
    they are :-)

  • http://andrewchen.typepad.com Andrew Chen

    There are about 300 ad networks, and people know it's a massive $500B
    industry (offline+online). So there's a lot of very smart PhDs and geeks
    chasing it, probably thousands of talented people total – I'd argue the ad
    network industry is far more efficient than Netflix, no matter how smart
    they are :-)

  • http://andrewchen.typepad.com Andrew Chen

    There are about 300 ad networks, and people know it's a massive $500B
    industry (offline+online). So there's a lot of very smart PhDs and geeks
    chasing it, probably thousands of talented people total – I'd argue the ad
    network industry is far more efficient than Netflix, no matter how smart
    they are :-)

  • http://www.facebook.com/barrychu Barry Chu

    Hey Andrew – totally agree, as you no doubt remember from working in behavioral targeting, it's become readily apparent that quality of inventory trumps all targeting. The best targeting “algorithms” are the ones that can transfer value from a known high-value entity (such as site with high purchase intent) by identifying those users as they move from site to site. Even then, the value degrades as you shift the context of the user. I would caution anyone looking at investing in or joining an ad network to inquire about their quality and quantity of unique inventory first before worrying about their technology.

  • huang

    Frozen winter is coming,do you like to buy ugg to preventing cold and keeping warm.As a pursuer for the most stylish ,is there any ugg boots you have in mind?I think you have no excuse anymore if you don't have a pair of UGG Bailey Button Boots.The Bailey Button UGG can be worn as a UGG classic short Boots or UGG classic mini Boots . Hottest pop style.If you like the UGG tall boots,www.realugg.com can provide high quality of Ugg Tall Boots in low price,Are you straitened for Ugg Boots Insoles,don't worry ,we have the best Ugg Boots Insoles on sale ,many of which are hand made by craftspeople.RealUGG,Inc came up with some new-styles- Ugg Lo Pro Button , Ugg Women's Rainier, We offer a wide range of UGG Boots in different color and styles. Browse the website, we are sure you will find Real UGG to tempt you!

  • http://www.darrenherman.com dherman76

    Andrew, great posts as always. You nailed it on the head about the algorithms- it's all about the data that the algorithms are processing. Garbage In, Garbage Out.

  • adrianbye

    A lot of ad inventory is driven via relationships. Therefore some ad networks will have significant competitive advantage simply due to the relationships they have developed.

  • http://MeetInnovators.com Adrian Bye

    A lot of ad inventory is driven via relationships. Therefore some ad networks will have significant competitive advantage simply due to the relationships they have developed.

Want more? Featured essays and book recommendations