@andrewchen

Subscribe · Featured · Recent · The Cold Start Problem 📘

My Quora answer to: How do you find insights like Facebook’s “7 friends in 10 days” to grow your product faster?

I recently answered a question on Quora and am sharing it on my blog:

How do you find insights like Facebook’s “7 friends in 10 days” to grow your product faster?

Here’s my thoughts below:

Why make a rule like this?
It’s important to remember the goal of making a pithy goal like “7 friends in 10 days” – it’s to help your team drive towards a clear objective. I’m sure “10 friends in 12 days” works well too, as does “5 friends in 1 day” but you just pick something that makes sense and easily memorable.

Anyway, here’s some thoughts about how to make something useful:

Defining the success metric
First, you need a way to evaluate how “successful” a user is, based on their behaviors. You might define this based on something like:

  • days they were active in the last 28 days
  • revenue from purchases in the last 28 days
  • content uploaded in the last 28 days
  • … or whatever else you want to define.

How do you figure out the right evaluation function? You just have to pick one, based on what makes sense for your business. There’s no one-size-fits-all answer here- you need to tailor this based on what makes your product work. In Facebook and Twitter’s cases, since they are ad-based models, they care a lot about frequency and engagement.

Exploring the data
Once you have a way to evaluate the success of a user, then you want to grab a cohort of users (let’s say everyone who’s joined in the last X days) and start creating rows of data for that user. Include the success metric, but also include a bunch of other stats you are tracking- maybe how many friends they have, how much content they’ve created, whether they’ve downloaded the mobile app, maybe how many comments they’ve given, or received, or anything else.

Eventually you get a row like:
success metric, biz metric 1, biz metric 2, biz metric 3, etc…

Once you have a bunch of rows, you can run a couple correlations and just see which things tend to correlate with the success metric. And obviously the whole point of this is to formulate a hypothesis in your head about what drives the success metric. The famous idea here is that, fire engines correlate with house fires, but that doesn’t mean that fire engines CAUSE house fires.

Running the regression
In some cases, it might be obvious that a particular metric correlates more strongly with your success metric than anything else. That helps you along. But if you want to get more formal, then you can do the kind of regression that David Cook describes.

The usual problem I’ve seen for startups is that there’s often not enough data, and too many variables, to be able to generate a really strong statistically significant model. And you can’t really tell your growth team “OK guys, active days is driven by friends, posts, likes, and 20 other factors. Let’s increase them.” Not very inspiring. So instead you’re just looking for something simple that explains enough of variation in success to rally your team behind it.

Verifying your model
After you’ve found the model what works for you, then the next step is to try and A/B test it. Do something that prioritizes the input variable and increases it, possibly at the expense of something else. See if those users are more successful as a result. If you see a big difference in your success metric, then you’re on to something. If not, then maybe it’s not a very good model.

“Branding” your model
Finally, once you’ve explored the data, run some regressions, and verified that your model works- then you have to be able to explain it to other people. So make it dead simple to talk about, repeat it over and over, and generally simplify it to the point where a lot of your growth product roadmap is focused on moving the metric up.

PS. Get new updates/analysis on tech and startups

I write a high-quality, weekly newsletter covering what's happening in Silicon Valley, focused on startups, marketing, and mobile.

Views expressed in “content” (including posts, podcasts, videos) linked on this website or posted in social media and other platforms (collectively, “content distribution outlets”) are my own and are not the views of AH Capital Management, L.L.C. (“a16z”) or its respective affiliates. AH Capital Management is an investment adviser registered with the Securities and Exchange Commission. Registration as an investment adviser does not imply any special skill or training. The posts are not directed to any investors or potential investors, and do not constitute an offer to sell -- or a solicitation of an offer to buy -- any securities, and may not be used or relied upon in evaluating the merits of any investment.

The content should not be construed as or relied upon in any manner as investment, legal, tax, or other advice. You should consult your own advisers as to legal, business, tax, and other related matters concerning any investment. Any projections, estimates, forecasts, targets, prospects and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Any charts provided here are for informational purposes only, and should not be relied upon when making any investment decision. Certain information contained in here has been obtained from third-party sources. While taken from sources believed to be reliable, I have not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation. The content speaks only as of the date indicated.

Under no circumstances should any posts or other information provided on this website -- or on associated content distribution outlets -- be construed as an offer soliciting the purchase or sale of any security or interest in any pooled investment vehicle sponsored, discussed, or mentioned by a16z personnel. Nor should it be construed as an offer to provide investment advisory services; an offer to invest in an a16z-managed pooled investment vehicle will be made separately and only by means of the confidential offering documents of the specific pooled investment vehicles -- which should be read in their entirety, and only to those who, among other requirements, meet certain qualifications under federal securities laws. Such investors, defined as accredited investors and qualified purchasers, are generally deemed capable of evaluating the merits and risks of prospective investments and financial matters. There can be no assurances that a16z’s investment objectives will be achieved or investment strategies will be successful. Any investment in a vehicle managed by a16z involves a high degree of risk including the risk that the entire amount invested is lost. Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z and there can be no assurance that the investments will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by a16z is available at https://a16z.com/investments/. Excluded from this list are investments for which the issuer has not provided permission for a16z to disclose publicly as well as unannounced investments in publicly traded digital assets. Past results of Andreessen Horowitz’s investments, pooled investment vehicles, or investment strategies are not necessarily indicative of future results. Please see https://a16z.com/disclosures for additional important information.