Sign up for my email newsletter

Get new updates, usually once a week – it features long-form essays on what’s going on here in Silicon Valley.

I’ve written 550+ essays which have been featured and quoted in The New York Times, Fortune, Wired, and WSJ. The topics range from mobile product design to fundraising to “growth hacking.”

Thanks for reading. -Andrew

Close

@andrewchen

Subscribe · Featured · Recent essays

How sheep-like behavior breeds innovation in Silicon Valley

Once you’ve been working in Silicon Valley for a bit, you’re often offered advice such as:

  • Are you launching at X conference?  … where X is whatever hot conference is coming up, like SXSW or Launch
  • Do you have an X app? … where X is whatever new platform just emerged, be it Open Social, iPhone, or whatever
  • Have you pitched X venture capitalist? … where X is a prolific headline-grabbing investor with a recent hot deal
  • You should do feature X that company Y does! … where X is some sexy (but possibly superficial feature) that a hot startup has done
  • Do you know what your X metric is? … where X is some metric a recent blog post was written about
  • Have you met X? … where X is some highly connected expert in the field
  • Maybe you should pivot into X space! … where X is a space with a hot company that just raised a ton of funding
  • Did you think about applying framework X to this? … where X is a new framework, be it gamification or viral loops or Lean

Sound familiar? I confess that I’ve both received and given much advice along the lines of the above. I call it “advice autopilot.”

The perils of “advice autopilot”
Advice autopilot is when you’re too lazy to think originally about a problem, instead regurgitate whatever smart thing you read on Quora or Hacker News. If you’re a bit more connected, instead you might parrot back what’s being spoken at during Silicon Valley events and boardrooms, yet the activity is still the same – everyone gets the same advice, regardless of situation. The problem is, the best advice rarely comes in this kind of format – instead, the advice will start out with “it depends…” and takes into account an infinite array of contextual and situational things that aren’t obvious. However, we are all lazy and so instead we go on autopilot, and do, read, say, and build, all the same things.

That’s not to say that sometimes generic advice isn’t good advice – sometimes it is, especially for noob teams who are working off an incomplete set of knowledge. Often you may not have the answers, but the questions can lead to interesting conversations. You may not be able to say “you should do an iPhone app” but it’s definitely useful to ask, “how does mobile fit into this?” This can help a lot.

The other manifestation of this advice autopilot is the dreaded use of “pattern matching” to recommend solutions and actions.

Pattern-matching in a world of low probability, exceptional outcomes
One of Silicon Valley’s biggest contradictions is the love of two diametrically opposed things:

  • The use of pattern-recognition to predict the future…
  • … and the obsession with a small number of exceptional successes.

Exceptional outcomes for startups are limited – let’s say it’s really only 5-10 companies per year. In this group, you’d include companies like Facebook and Google that have “made it” and hit $100B valuations. On the emerging side, this would include startups who might ultimately have a shot at this, like Dropbox, Square, Airbnb, Twitter, etc. This is an extraordinarily small set of companies, and it isn’t much data.

The problem is, we’re hairless apes that like to recognize patterns, even in random noise. So as a result, we make little rules for ourselves – Entrepreneurs who are Harvard dropouts are good, but dropping out of Stanford grad school is even better. It’s good if they start a company in their 20s unless they’re Jeff Bezos. Being an alum of Google is good, but being an alum of Paypal is even better. Hardcore engineers as founders is good, but the list of exceptions is long: Airbnb, Pinterest, Zynga, Fab, and many others. And whatever you do, don’t fund husband-wife teams, unless they start VMWare or Cisco, in which case forget that piece of advice.

As anyone who’s taken a little statistics knows, when you have a small dataset and lots of variables, you can’t predict shit. And yet we try!

The intense focus on a small set of companies also introduces a well-known logical fallacy called Survivorship Bias. Here’s the Wikipedia page, it’s interesting reading. Basically, the idea is that we draw our pattern-recognition from well-publicized successful companies while ignoring the negative data from companies that might have done many of the same things, but end up with unpublicized failures. We’re all so intimately familiar with stories like “two PhDs from Stanford start Google” that we ignore all the cases where two PhDs from Stanford try to start a company and fail. Or similarly, YCombinator has built a great rep on companies like Airbnb and Dropbox, and yet you’d think that if you invest in 600+ startups that you’d get a few hits. Because of factors like this, it might seem as though A predicts B when in fact, it does nothing of the sort – we’re just not taking the entire dataset into account.

Conformity leads to average outcomes when we seek exceptional outcomes
The problem with giving and taking so much of the same advice is that ultimately it breeds conformity, which is another way if saying it reduces the variance in the outcome. And if you conform enough, you end up creating the average outcome:

The average outcome for entrepreneurs is, your startup fails.

Lets not forget that. And so one part of Naval Ravikant’s talk on fundable startups that resonated with me is the idea of playing to your extremes. He says in the talk:

“Investors are trying to find the exceptional outcomes, so they are looking for something exceptional about the company. Instead of trying to do everything well (traction, team, product, social proof, pitch, etc), do one thing exceptional. As a startup you have to be exceptional in at least one regard.” -Naval Ravikant @naval

Be extremely good at something, and invest in it disproportionately relative to your competition – this gives you the opportunity to actually create an extreme outcome. Otherwise, the average outcome doesn’t seem so good.

The flipside of innovation
The funny thing with all of this, of course, is that this is what innovation looks like. The remarkable ability for practical knowledge to disseminate amongst the Bay Area tech community is what makes it so strong. Before something becomes autopilot advice for a wide variety of people, often a small number of hard-working teams who know what they’re doing leverage it to great success. Follow those people, and you might find yourself successful – just like them.

So the billion dollar question is – how do you separate out trendy/junk advice from what really matters?

… well, it depends!

Like this post?
Get new updates via newsletter..

  • http://twitter.com/rohit_x_ rohit sharma

    cannot ever pattern match – the past is rarely a good predictor of the future when it comes to startups. 
    if one manages to condense learning + modify it according to the (now) changed marketplace and customer behavior, the lessons are useful. faced with rapid pace of change of technology, customer requirements, and competition, the best advice may be to figure out something not done yet vs. do “X” for “Y” type advice.

  • http://twitter.com/kevindewalt Kevin Dewalt

    Well said, Andrew.  For these reasons I get some of my best perspective by being a Skeptic and trying (pointlessly to some degree) to rebel against these cognitive biases.  It is what makes applying Lean Startup principles so tough:  we’re wired to believe whatever patterns we want to see.  

    I actually think Confirmation Bias is the biggest boogeyman, it causes all of us to ignore the data which suggests our product ideas or investments won’t work.  Michael Shermer’s The Believing Brain is a great popular text for digging further into this subject, http://kevindewalt.com/blog/2011/09/13/why-lean-startups-are-hard-part-2/

  • FAKE GRIMLOCK

    YOU RIGHT!

    OR ELSE!

  • destraynor

    Fantastic post Andrew.
    Thanks. 

  • http://www.facebook.com/jaydrayer Jay Drayer

    Great post Andrew… gives me lots to ponder that’s timely for where we are now.

  • http://GrasshopperHerder.com Tristan Kromer

    Despite the obvious fallacy, I have yet to find a really good argument against pattern matching in startup land. Even very smart people weigh companies they’ve heard about above and beyond countless failed startups they’ve never heard of.

  • http://www.nirandfar.com/ Nir Eyal

    Fantastic post Andrew! I love how you turn the mirror on the many ways Silicon Valley is slightly nuts. Some of the best advice I ever received was to stop searching for the answers and start writing the answers. Bravo Andrew!  

  • http://twitter.com/ieslick Ian Eslick

    What is worse, the people who go out and talk about their successes are rarely able to be honest with themselves about what drove their success.  The factors that lead to success are never fully in our control, but we hairless apes so love the illusion of control that we assign credit to our behavior for the outcomes of our life  Of course we have to, how else do we learn?  This human learning heuristic is only useful for tight feedback systems (avoid damage, find food, etc) where we can test our models over and over to smooth over all the variation coming from the external world.  The many variables, small data set generalization problem is well articulated here.  It takes lots of samples to learn/train a model that can exercise probabilistic control of a stochastic system.

    Personally I think we need to focus on sharing not what it takes to succeed, but what is usually necessary to avoid failure – these things are more prosaic but much more likely to be true.  For example, if you don’t execute, you almost certainly will fail – if you do execute, then you have a shot at success.  Better to raise more money when you can than to try too hard to preserve equity.

  • http://twitter.com/30StreetStudio RV

    This post brought to mind two quotes which are apt:

    “Rarely do we find men who willingly engage in hard, solid thinking. There is an almost universal quest for easy answers and half-baked solutions. Nothing pains some people more than having to think”. - Martin Luther King, Jr

    “Follow the path of the unsafe, independent thinker. Expose your ideas to the danger of controversy. Speak your mind and fear less the label of “crackpot” than the stigma of conformity”. – Thomas John Watson, Sr.

  • http://www.jordi.pro/netbiz Jordi Robert-Ribes

    Great post!
    “remarkable ability for practical knowledge to disseminate amongst the Bay Area tech community” => Yes, indeed! And this is done via people’s networks and meetings and coffees and so on.

  • http://profile.yahoo.com/EQNZJBXHX2ITJ47IKGE66JU3AI yahoo-EQNZJBXHX2ITJ47IKGE66JU3AI

    Excellent take on the situation out there.  As an entrepeneur, being outside of silicon valley and being in my 40s, I do not fit the criteria for success.  Nonetheless, the idea I have for the company I am starting is based on something I am passionate about and not about the money or following the herd, and is really trying to meet a gap in the market and provide a service for those people.  Surprisingly, I am hitting a nerve with some powerful people in the industry and they see a potential exceptional outcome and want to be a part of the company.

    Other ideas I have had for companies before this one were really based on money and how I can be better than the herd – those got me nowhere.  Truly if you are passionate about making things easier, better or identifying something of value to help others is where you should focus – all of the other things will come to you in ways of success.  Companies built around money or power plays will only get you so far.

  • http://twitter.com/BiddRocket Brian Ley

    Gold. Thanks for sharing

  • http://twitter.com/cleakstar Cleakstar

    Nice post and interesting blog. I’ll keep visiting. 

  • chris b

    I agree with the article (I’ve never read something as good as it), but I find that the following quote is contradictory with it

    Instead of trying to do everything well (traction, team, product, social proof, pitch, etc), do one thing exceptional.

    I surely will not follow this advice (an exceptionnal team with a bad product, who cares ?) ;)So I agree with Ian : we should share the tips necessary to avoid failure, not the tips that sometimes lead to success

Want more? Featured essays and book recommendations