@andrewchen

Get the newsletter · Featured · Recent

10 years in the Bay Area – what I’ve learned

January 2007
Ten years ago this week, I took a long, cloudy drive from Seattle to Silicon Valley on Highway 1 to start a new job and new life. I was barely 24 years old, in a hurry to change the world, and eager to begin my first day at MDV, a Silicon Valley venture firm, as their new Entrepreneur-in-Residence. It was 2007, and the iPhone hadn’t been released yet, YCombinator was just getting started, and MySpace was still bigger than Facebook. And who’s this Obama guy that folks are talking about?

That was a long time ago :)

A decade later, the world is incredibly different. And I’m different too, because the Bay Area has profoundly and fundamentally changed me. Along the way, there’s been good decisions and plenty of bad. I want to share some high-level observations/thoughts, focused on mostly career/professional stuff but a little bit of personal too.

Let’s dedicate this essay to all the new folks starting out in the Bay Area. Welcome!

People are the secret sauce
First, what makes the Bay Area special for tech is the people. I barely knew anyone when I first arrived here, so I had a simple goal in 2007:

Meet 5 new people per day, every day.

It helped that working at a venture firm is all about networking, so I picked aggressive goal! I started by emailing my tech friends to intro me to smart people working on cool products. Upon grabbing coffee with them, I followed up to ask for more intros, and more. I kept this insane pace for 6 months, which created an incredible introduction to the SF tech community.

Years later, while I’m nowhere near this volume anymore, I’m still going! This is one of the most fantastic ways to learn. Most importantly, while this started out as a work thing, many of the folks I met in 2007 are now close friends.

Think long-term
Everyone you meet here will likely still be here in 10 years. This changes the professional dynamic so that we can all help each other, build relationships, and give real time/effort, without feeling like things have to be transactional. It starts to make sense to invest in activities that pay off in years or decades, not months.

My writing has also been a microcosm of this, where in the first few months, there was a grand total of maybe a few dozen readers – mostly friends and family, forcibly subscribed! It’s been a slow/steady ramp that’s taken thousands of hours of effort and many years to grow into a real following. So for the folks who are struggling to build audiences for your newsletters or blogs, keep going! It’s worth it.

Vuja De
The more years of experience you accumulate in tech, the easier it gets to become negative and closed off to new ideas. It’s easy to say “No, that’s never going to work” because experience usually generalizes towards everything failing!

And yet every couple years, there’s a new cycle – social, smartphones, ridesharing – that’s counterintuitive and huge, and blows away prior assumptions. I’ve written about why I doubted Facebook could be a billion dollar business, and why I was wrong. In my years in the Bay Area, that’s one of that’s just one out of many wrong calls :)

It takes real effort to stay open-minded, even as you learn more and get comfortable in your own expertise. The IDEO folks sometimes talk of “vuja de,” a twist on the familiar term:

Deja vu is when you see something new, but feel like you’ve already seen it before.

Vuja de is when you’ve seen something a million times, but see it like it’s the first time.

It takes a lot of openness and humility to try and understand weird new companies/categories, especially when there’s bad historical datapoints. Like how search engines were a terrible business until Google. Same with social networks. Or how mobile was always the next new thing, but actually perpetual vaporware, until the iPhone.

Missed chances
The longer you live in the Bay Area, the more missed chances you’ll have. I met the Facebook founding team when they were 11 people, and thought for a split second that I should try to get a job there, before deciding it could never be big. Hilariously wrong. I have friends who could’ve invested in Uber’s seed round back when it was valued at $4M, but passed because it was “just a taxi app” – oops. An early Googler told me about a guy who joined as one of the first ten employees, but quit on his first day, forgoing $1B of stock, because the founders’ mannerisms annoyed him.

These missed chances will weirdly haunt you, even when you know better.

Startup romanticism
From the outside looking in, I thought that doing a startup was a magical, rare experience that you only got a few shots at in your life. Kind of a romantic notion that I held on to for many years. But once you’re in the Bay Area for a few years, what you quickly realize is that starting a company and getting investors funding you, actually isn’t rare at all. It’s commonplace, because actually mediocre startups and tech companies are plentiful! And it’s unfortunately very easy to start a mediocre startup of your own.

Bill Gossman, a long-time mentor who’s lived in both SF and Seattle, gave me some advice early on:

Don’t think that Silicon Valley has better entrepreneurs. They don’t. But they have more people trying. They have more crappy companies and mediocre entrepreneurs, but also they have more great companies and people too.

For me, this meant my first years in the Bay Area were spent on trying to get the “rare” chance of building a startup. Over time, I came to believe that the rare thing is actually building the Amazon, Google, Facebook, Uber-type companies that come around only every 5-10 years.

Last year, I decided to jump onto the rocketship of a great company rather than continuing with the mediocre. This is a core reason I’m at Uber these days – to have a special experience that I’ll remember, years from now.

To another ten years!
Finally, I want to thank everyone who’s been reading for the past few years. As I mention above, writing has been an enormously fulfilling thing. I’m hugely appreciative for you to have come on the journey – thank you for reading and for your comments/feedback over the years.

Here’s to a happy new year and a few more decades in the Bay Area for me :)

PS. Get new updates/analysis on tech and startups

I write a high-quality, weekly newsletter covering what's happening in Silicon Valley, focused on startups, marketing, and mobile.

How To (Actually) Calculate CAC

[Andrew: Paid marketing remains an integral part of many products’ acquisition channels, and one of the key metrics is Cost of Customer Acquisition, which is a nuanced calculation with lots of gotchas. My good friend Brian Balfour (ex-vp growth at Hubspot) put together this incredible essays with details on how to think about it.]

Brian Balfour (ex-VP Growth at Hubspot):

avatarIn everything from growth projections to company valuations, it’s common to use CAC and CPA interchangeably — but it’s wrong, and it can cost you. In this post, I break down growth’s most important jargon to demystify the true cost of acquisition, and dig into the most common mistakes leading growth teams off track.

Note: This is a big topic that’s best addressed with live examples and interactive frameworks. To that end, I’ve included a number of examples of real companies, plus interactive spreadsheets that you can access throughout the guide. To adapt any spreadsheet for your own calculations, click here, then go to File > Make a copy to create your own version.

Customer acquisition is not CPA – Three examples

To start off, let’s address a common myth. Customer acquisition cost (CAC) and cost per acquisition (CPA) are commonly conflated, and yet in reality they’re completely different metrics. Understanding the difference is the start to understanding CAC in depth.

CAC specifically measures the cost to acquire a customer. Conversely, CPA (Cost Per Acquisition) measures the cost to acquire something that is not a customer — for example, a registration, activated user, trial, or a lead. The two are related because CPA is usually used to measure the cost of things that are leading indicators to CAC.

Four examples of how CAC and CPA are different but related:

1. Dropbox

Since Dropbox is a freemium product, CAC would be the cost to acquire a paying user on either their pro or their team plan. CPA would be used for things such as the Cost Per Registration (of a free user), Cost Per Activated (free) User, and other important, but still non-paying, actions that signal that someone has moved from being a visitor to a user of the product.

image-1-dropbox-cac-768x576

image-2-dropbox-cpa-768x576

2. HubSpot

Since HubSpot is a B2B SaaS product, CAC would equal the cost to acquire a new customer on one of their Basic, Pro, or Enterprise plans. CPA would be used for leading indicators to CAC, such as Cost Per Lead, Cost Per Sales Qualified Lead, Cost Per Trial or other points in the marketing and sales funnel.

image-3-hubspot-cac-768x576

image-4-hubspot-cpa-768x576

3. Facebook

B2C companies supported by ad models are a little different. In Facebook’s case the paying customer are advertisers so CAC is the cost to acquire a new advertiser.  However if you just look at users, users/customers are the same. CPA is most likely used for things like Cost Per Registration, Cost Per Activated User, etc.

image-5-facebook-cac-and-cpa-768x576

The key point is the first thing you need to do to understand CAC is very simple:

Define who/what a customer is in your model.

Make that definition clear and simple and get the language between CAC and CPA consistent otherwise communication will be very confusing.

The basic calculation of CAC and why it’s wrong

If you google ‘how to calculate cost of customer acquisition” you will get the basic formula below:

CAC = Total Marketing + Sales Expenses / # of New Customers Acquired

image-7-cac-equation-1-768x576

On the surface this is correct, but it is missing a lot of details and definitions around each variable in the equation to get it right. Even the best basic calculation can be very misleading. For example, using this formula you might look at the below spreadsheet and assume the following:

To adapt this spreadsheet for your own calculations, click here, then go to File > Make a copy to create your own version.

But what if I told you the following things:

  1. It takes on average for most customers 60 days from lead to become a customer.
  2. Not all customers are new customers, but some of them are returning.
  3. This is a freemium product and there are costs to supporting users while they are free before they become a paying user (customer).

These three tidbits of additional information should change how we look at the basic formula above. Instead of taking the equation at face value, we need to evaluate a few questions first.

How to really calculate CAC

There are three key questions we need to ask to define a more accurate calculation of CAC for a business. All of them dig into a level deeper around the variables in the equation.

Key Question #1:  How long between your marketing/sales touch points and when someone becomes a customer?

The first issue with the basic calculation is that it doesn’t take into account the time period between when you spend the marketing/sales money and when you actually acquire a customer. Here are two examples:

Example 1:  Freemium Product

Let’s look at Dropbox as an example. When you sign up for Dropbox you start using their free tier. You use Dropbox free for some time period until you hit your storage space limit, which you then might upgrade. For a lot of users that time period is months (and in some cases over a year). The story is the same with other freemium products such as Evernote, Buffer, etc.

image-8-time-delay-1-768x576

Example 2:  SaaS Company w/ Inside Sales

Most SaaS companies with inside sales models someone might become a lead this month (due to our marketing efforts this month) but will take 60+ days before they become an actual customer because they need to go through the sales process.

image-9-time-delay-2-768x576

If you don’t take these time periods into account, you could be overestimating or underestimating CAC and as a result making some terrible operating decisions.

Here is an example of how you could overestimate.In the below example, CAC is being calculated by taking the month’s marketing costs and dividing it by new customers in the same month.

To adapt this spreadsheet for your own calculations, click here, then go to File > Make a copy to create your own version.

In March we try some new channels which causes a large increase in marketing costs.  Under the simple calculation our CAC is $148.  If our target CAC is $125, we would likely make the decision that March was unsuccessful and we would turn off those new channels.

But, let’s say it actually takes 2 months for someone to become a customer.  Below is the same data but changing the calculation to account for this 2 month period.

To adapt this spreadsheet for your own calculations, click here, then go to File > Make a copy to create your own version.

This change tells a completely different story. In March we have a CAC of $84, and in April a CAC of $111. Instead of turning off the new channels we tried in March, we would probably make the decision to scale them.

The key question about time between marketing/sales expenses and acquiring a customer does not matter if you fall in one of two scenarios:

1. The time between marketing touch point and someone becoming a customer is very short. This is true for a lot of B2C companies that have very short decision funnels for users: Snapchat, Instagram, and others.

2. Your marketing/sales expenses are so consistent that it normalizes itself out over time. But even in this case it is best to be more accurate.

You need to figure out how the timing of the expenses correlate with the timing someone actually becomes a customer. Marketing expenses could correlate differently than sales expenses.

The simplest way to account for this is figure out your average marketing/sales cycle. In other words what is the average amount of time from first marketing touch point, to acquiring the customer?

Here’s an example. Let’s say we are a SaaS company where the average time is 60 days from lead to customer and we believe the sales expenses are spread evenly over that two month time period.

image-10-cac-equation-2-768x576

The CAC equation would be as follows:

CAC = (Marketing Expenses (n-60) + 1/2 Sales (n-30) + ½ Sales (n)) / New Customers (n)

n= Current Month

Here is an example model with that calculation built in:

To adapt this spreadsheet for your own calculations, click here, then go to File > Make a copy to create your own version.

Key Question #2:  What expenses do you include in the Marketing + Sales?

The second question you need to answer to get an accurate CAC calculation is what expenses do you include in the numerator (marketing/sales)? Before we look at some examples of the answer to this question differs, here are the three most common mistakes I see:

Mistake #1:  Not Including Salaries

You need to include the salaries of all people working on marketing and sales. We see CAC numbers a lot where they aren’t included. This includes not just the individual contributors who are 100% dedicated to marketing/sales, but also those (often times managers) who spend part of their time on marketing/sales. A CAC number with salaries included is often referred to as “Fully Loaded CAC.” There are some scenarios where it is useful to separate Fully Loaded from Non-Loaded CAC which we’ll explain later.

Mistake #2:  Not Including Overhead

Similar to the mistake of not including salaries, you need to include the overhead (rent, equipment, etc) allocated to those employees working on marketing sales.

Mistake #3:  Not Including Money Spent On Tools

The marketing and sales tool space has exploded. Most teams are using 10+ tools to operate their marketing and sales machine. These tools can add up in cost and need to be included in the expenses  of your CAC calculation.

image-11-fully-loaded-cac-768x576

If you include those main things, the answer to this question starts to get a little more complicated and range from company to company. A couple of examples:

Spotify and the Case of Freemium – Does that Include Product/Engineering/Support?

image-12-cac-spotify-freemium-768x576

Spotify is a freemium business. They have millions of users using the free version of their product which helps them acquire new users through sharing music and other viral channels.

In most companies, product, engineering, and support are not included in CAC (typically part of R&D). But if the free product is your primary method of customer acquisition, shouldn’t the expenses that support that free product be included in the expenses portion of your CAC calculation? There are different opinions to this question, but we fall on the side of yes.

If you had engineers, PMs, and other roles on the marketing or sales team for marketing/ops you would include those salaries and expenses in the CAC calculation. The engineers, PMs, or other roles might not technically be on the “marketing” or “sales” teams, but they are still expenses that are required to support new customer acquisition.

HubSpot and the Case of SaaS –  Include Customer Success Costs?

Most SaaS companies like HubSpot have sizable Customer Success teams. The definition and roles of these teams can vary widely. Some Customer Success teams are purely devoted to churn prevention. Some are devoted to helping onboard a new customer. Some are devoted to winning previous lost customers back.

All this begs the question, should customer success costs be included in CAC? Some of these responsibilities touch acquiring new customers and therefore make the case that they should be allocated to CAC.

Dollar Shave Club and the Case of Subscription Ecommerce – Support, Shipping for Free Trial?

image-13-cac-dollar-shave-ecomm-768x576

Dollar Shave Club is a subscription ecommerce business. They famously have a one month trial for $1. That one month trial has a number of expenses outside of marketing, including shipment costs for the initial package, support during the trial, and other costs.  Should those costs be included in CAC? The answer to that depends how you define a new customer.

In Dollar Shave Club’s case, we would make an argument that someone on a $1 trial is not a customer yet. A new customer is someone that extends beyond the trial.  Therefore all of those costs associated with supporting the free trial should be included in CAC. Those costs are partially offset by the $1 trial payment, but not fully offset.

A customer is a customer, right? Not necessarily. When it comes to calculating CAC, we need to distinguish between new and returning customers. In most organizations there are marketing and sales efforts focused on new customers, and there are marketing and sales efforts focused on retaining or getting customers back.

The mistake is only including marketing and sales expenses in the numerator for new customers, but including all customers (including returning customers) in the denominator. This will make your CAC look artificially low. You can solve this in one of two ways:

1. Include all marketing/sales expenses (including those focused on retention) and all customers.

2. Separate expenses for new customers from reactivating old customers and separate out new from reactivated customers in the denominator.

Recap

From the examples we’ve covered today, I hope it’s more clear that calculating true CAC involves a lot more than a simple, one-size-fits-all equation.

Instead, an honest assessment of customer acquisition cost looks at the length of your sales cycle, how many customers are truly new customers (as opposed to returning customers), and the total costs and resources required to support marketing efforts that lead to new customer acquisition.

Brian Balfour is CEO of Reforge, previously VP Growth @ HubSpot, EIR @ Trinity Ventures, Co-Founder of Boundless, Co-Founder of Viximo, Co-Founder reCatalyze & PopSignal. 

PS. Get new updates/analysis on tech and startups

I write a high-quality, weekly newsletter covering what's happening in Silicon Valley, focused on startups, marketing, and mobile.

Growth Interview Questions from Atlassian, SurveyMonkey, Gusto and Hubspot (Guest Post)

[Andrew: Excited about today’s guest post! I was recently interviewed by the folks at Reforge, a new company started by my friends Brian Balfour and Susan Su focused on advanced professional education. They asked a great question – how do you interview for growth folks? I gave some my 2 cents based on my experience helping startups and growth folks. They pulled together a great essay below!]

Growth Interview Questions

Guest Essay by Susan Su, Reforge

Together, Shawn Clowes (Atlassian), Elena Verna (SurveyMonkey), Nick Soman (Gusto), Andrew Chen and Brian Balfour (Reforge, previously at Hubspot) have interviewed or screened over 1,000 individual candidates for growth roles – both for their current and previous companies, plus startups where they’ve invested/advised.

Growth is an emerging field, and there’s hardly a playbook on how to ace your growth interview (whichever side of the table you’re on), and yet hiring and team could be the most important “growth hack” of all. I recently asked a handful of folks from the Reforge Collective about the questions they ask when interviewing candidates for competitive positions in growth.

Here are some of the questions we’ll cover:

1. The Golden Gate Bridge
2. Growth Hacking a City
3. Are you an early adopter?
4. Live brainstorm an experiment backlog
5. What Are Your Setbacks?
6. Look for opportunity, not just risk
7. But… why?
8. The Bullshit Test
9. The Trajectory of Growth
10. Probe on Metrics
11. And on Culture
12. 6-Month Roadmap
13. Growth Resources
14. Low Hanging Fruit

If you’re looking for your next job in growth, or you’re trying to fill one at your company, consider this a peek into how a few other growth marketing experts are structuring their growth interviews.

This 1 Weird Interview Question
Growth is as much about showing creative problem solving as much as quick metrics-based iteration. If a candidate is out there googling answers to “trick” interview questions, then they’re missing out on an opportunity to showcase their own unique way of combining creativity with numbers.

“Weird” interview questions let you out to roam beyond the constraints of the role in front of you. The best weird questions are designed to highlight original problem-solving, and if that doesn’t excite you, then a position in growth probably won’t be fun or fruitful.

1. The Golden Gate Bridge

Elena Verna SurveyMonkey growth interview

When she was still a candidate interviewing at SurveyMonkey, Elena Verna fielded an unusual interview question that she continues to use in her own growth interviews to this day:

Let’s say you were to come in tomorrow, and you got a new project to work on the Golden Gate Bridge and its toll collecting. But, you’re told that they’ve recently lost all of the tracking of both traffic flow and revenue collection, and your boss has just told you that you need to come up with a revenue estimate in the next 15 minutes. You don’t know anything about how much has been collected historically. All you have is a blueprint of the bridge.

How would you estimate how much revenue or the bridge generates on a weekly basis?

Elena explains that the question is meant to demonstrate your problem solving skills (not to get to the “right” answer necessarily).

For me it’s a sign that you’re willing to tackle an open-ended question in a creative way — without all of the data and answers. How am I going to approximate with only broad strokes of knowledge? What are the key variables (Bridge length? Car length? Car weight? Traffic volume?), and how far can I go in 15 minutes?

Elena insists there’s no right answer. Instead, it’s a window into your process of thinking with uncertain variables — and also how excited you get when you’re tasked with reasoning through an unsolved problem.

2. Growth Hacking a City

Nick Soman Gusto growth interview

Ultimately, hiring for your growth role is both high risk and high opportunity. The teams with the most at stake (and offering the best opportunities) will need to explore every last corner of a person’s thinking and style, and that sometimes means taking a lateral approach.

Nick Soman poses an unusual question designed to get at the way candidates think about growth beyond the templates and playbooks that circulate in the growth community:

How would you growth hack a city?

It’s not an immediately technical or product-based experience, and yet it’s an interesting question that might actually become more and more relevant. How would you attract residents to it? How would you attract the other people and elements that that ecosystem requires? What mechanisms would you employ to grow your city? It’s very revealing to see how people approach growth when they have no templates, when they start from zero.”

3. Are you an early adopter?

Shaun Clowes Atlassian growth interview

Have you ever thought about your own relationship with Snapchat? Have you broken down your own psychology of engagement with Facebook? How about for lesser-known, non-consumer products that are in your life?

We are what we eat, and the products and apps we consume (and how we interact with them) can say a lot about who and how we are when it comes to creating and growing our own products and apps.

Shaun Clowes wants to know what you’re using at work:

If you just got a new computer at work, what apps would you immediately set up?

I’m looking for their take on a piece of software that they care about, something that gets them excited, and then how they explain it to me.

What are the most recent apps you been playing with on your phone?

That gives me insight into how in touch you are with the industry, how much you’re seeking out things that are different or somewhat common, and whether you’re an early adopter of things.

4. Live brainstorm an experiment backlog

It’s the default to come prepared to a growth interview. You’ve looked at their core growth loops, you’ve analyzed their funnels, and you have an (externally informed) idea of where the business is going. But what if you were put on the spot to dig even deeper into how the business can grow?

Nick Soman wants to see candidates live-brainstorm an experiment backlog:

How many ideas can you come up with in 3 minutes?  Maybe it’s a handful, let’s say 5. Then, I really want to push for a 6th or a 7th.

I want to see the candidate beyond their comfort zone and extend beyond their pre-planned ideas and analysis, especially in real-time. I resist the urge to ask follow-up questions — even when I’m really curious — because I want to see you go for breadth, and then I want you to be able to follow that up with an objective assessment of those ideas.

5. What Are Your Setbacks?

If none of your experiments are failures, then you’re probably not testing the right things. Running growth means swimming in setbacks — and it helps if you’ve had some life experience with that.

Andrew Chen says the level of your setbacks say more about you than their specific details:

One of the questions that I like a lot that doesn’t have to do with growth but tells you a lot about a person is to ask about someone’s biggest setback, either personally or professionally. You can get a sense of if that’s a real setback and how they reacted to it.

For example, someone who’s very junior is going to come up with a small setback. They’re going to say, ‘Oh well at work I did a project and it sucked.’ That’s very different than something like, ‘I was at a company and I convinced the board to do something and it was a long decision and the company failed. And everyone got fired.’ Or, ‘I moved from country A to country B, left my family and friends and started over again.’

Understanding setbacks helps you understand what a person’s priorities are, and how much resilience they have to bounce back.

What Most Growth Interviews Are Missing

6. Look for opportunity, not just risk

Elena Verna surveymonkey growth interview excellence question

A lot of people try to understand a candidate’s weaknesses, but Elena Verna wants to know your superpower. Early to mid-stage growth is as much about doubling down on unique advantages as it is about fixing leaks (ie, identifying and addressing weaknesses). Later, the truly stand-out orgs are the ones defined by their unfair advantages, particularly in growth and product.

Building a growth team is a microcosm of the trajectory you want to see that team take, according to Elena:

Very few growth managers will look at an opportunity in a person and say, “Ok, these are your strengths and I’m actually going to tailor a roll around you to make sure that I’m playing to your strengths.

It’s good to know what people aren’t good at — where they’ll be a liability. But I want to dig into to what they’re excellent at as well. That’s what you really need to focus on, and to make sure that that strength aligns with the position at hand or that it’s possible to mold the position around it.

Too often, we identify a problem or a hole in the business and start looking for the person that will fit it. The person you find could be effective very early on, but evaluating too tightly against specific role can be very short-sighted. Yes, they might be able to sort out that immediate issue for you but in the same stroke you may end up hiring the wrong person long term.

The real opportunity is finding the person who will be happy (and make your business happy) as the definition of growth itself expands, and the immediate problem becomes obsolete. Where do you want them to be in a year? Look for the opportunities, not just the “urgent” holes.”

7. But… why?

Most of us aren’t professional interviewers. We know our area, or we know growth marketing as a broader domain, and we stick to what we know. But, in building a growth team, you’re called to ask people about things that may not be your cup of tea.

As a result, many of us don’t dig into the details of a candidate’s experience, which is a bad idea for both the hiring organization and the candidate — the former misses out on opportunity and risk assessment (is this person a good fit?), and the latter may not get to tell their punchline.

Shaun Clowes asks a deeper layer of question where most interviews have called it a day.

Most interviewers will ask you a question about how in the past have you done X thing. You give them a surface-level answer like ‘We did Y and then achieved Z.’

A few ways to drill in more deeply would be to follow up with questions like:

  • When you say “we,” how much was you and how much was everybody else?
  • Were you really pivotally involved in this or was this really something that you just got carried along with?

Sometimes it feels like the answer has been rehearsed. It’s the correct answer, but when you drill into it, it’s clear that either the initiative isn’t all they said it was or it wasn’t as deep as they said it was or their involvement wasn’t as deep as they said it was. The best way to get through this is with a one-word question: ‘Why?’ ‘Why did you do that?’

You actually can reply with “Why did you do that?” to every subsequent answer, and it’s almost endlessly educational. This addresses the need for depth that growth roles need, but that many interviews often lack.”

8. The Bullshit Test

How do you know someone really knows growth and doesn’t just have a great handle on acronyms?

Growth is a critical role but not one that hiring teams can succinctly test for. That is, you can’t check out someone’s GitHub for growth or decouple what public evidence you find of their work from other situational factors, like a great team, a solid company, or a unique ecosystem opportunity whose bandwagon they jumped onto.

Andrew Chen runs a “bullshit test” to make sure that candidates aren’t merely fluent in blog posts and jargon.

I ask people to get on the whiteboard and draw out the whole thing. For example, I may ask someone “How does YouTube grow?

There, I want to watch you draw out the entire flow for how a user comes into YouTube and how you think it might all work — just from what you’ve observed on the outside. Then I’ll ask you where you might make improvements. I want you to do all of this in real-time with me in the room.

This exercise shows a level of detail and thinking that indicates that you’ve mastered what you’re doing, versus that you’ve read all the blogs. I want to get the sense that your capacity goes beyond knowing the concepts, and that you have a depth of process understanding that you can bring to anything you do next.”

What You Should Ask Your Interviewer (but Probably Aren’t)

“Do you have any questions for me?” is a common wrap-up to an interview session. It’s also its own covert test of your listening skills and the depth of your analytical abilities.

But aside from generalities about company culture, project overviews, and basic metrics, the top candidates for growth roles get under the surface of their opportunity with specific questions for their interviewers.

9. The Trajectory of Growth  

The definition of growth or growth team can differ significantly from one business to another. Some growth teams are focused only on driving acquisition into the business while others are making fundamental calls on product strategy and development.

Elena wants to see candidates who are taking a long, non-static view of growth.

Ask, ‘What does growth mean for this company, and what will it mean?’ You need to know whether they are responsible for driving metrics across the rest of the funnel or not, and how they may or may not evolve with the rest of the business.

Ask, ‘How does the growth team actually catch up with the structure of the business as it evolves?’ Many applicants simply want to understand where they’re going to be in a couple of months. This is very short-sighted. It’s not just about growth today, in this place and time; it’s about trajectory.”

10. Probe on Metrics

Brian Balfour growth interview

For growth roles you want to know are you coming in to fix the broken system, or are you coming in to make a good system great?

Understanding the answer to that fundamental question comes down to understanding both the metrics of the business and the culture of the team.

When it comes to metrics, most people simply don’t go far enough into the specifics. Brian Balfour says that’s where great growth candidates stand out.

What does retention look like? What does LTV look like? What are the biggest dropoffs?

If a growth candidate doesn’t ask me those basic question in an interview, I’m shocked.

But then you need to keep going. Keep asking questions about the metrics until your interviewer stops you and says, “It’s too detailed and we can’t give that out in an interview”.

You should get as much information as you possibly can. Not only will you know what type of situation you’re walking into, you’ll also show that you know how to think about growth for that company.”

11. And on Culture

The velocity of growth is determined by one part strategy, one part implementation. Great strategy and promising metrics can still be blocked by cultural issues within the organization; successful growth is technical, but it’s also fundamentally human.

Brian wants to see candidates who seek to understand the relationship between the growth team and other teams: core product, marketing, sales, executive.

Ask, ‘If I wanted to make X type of change in this part of the product, what would the process look like to make that happen?

Follow it with, ‘Would I or my team have our own resources and autonomy to be able to make that change?’

Changes can require negotiation, politics and navigating a number of other ‘people’ steps. You need to pose that same type of situational question to your interviewer that they’re probably asking you: ‘If I wanted to do this we thought if it was a good idea to do this, how would we get this done?’

That gives you a much better idea of how the team works, rather than simply asking them, ‘What’s the process around here?’

You’ll start to realize when they are describing how to get something done that there are certain points where they’ll show discomfort. Those are the areas where there’s friction within the company.

No company is perfect. But, it’s much better to know what the flaws are going into it, rather than being surprised after the fact. That way you can be better prepared and more effective from day one.”

12. 6-Month Roadmap

Being effective in flux starts with having a sense of what’s expected of you in the first six months in a new role. When you’re running weekly experiments or solving previously untackled problems, Andrew Chen says you should ask:

If I were to join, what would I be tasked with achieving in the first 6 months?’ You have to have a good sense for the 6 month roadmap — what you would actually do in terms of experiments and goals — so that you can come in with that from day 1.

13. Growth Resources

There are many different flavors of growth, and resourcing the growth function is a key variable from team to team. Growth has been interesting because there are different flavors of it.

Andrew wants to see candidates ask:

Are there are going to be dedicated engineers? Are there going to be dedicated designers? Or is this a situation where we need someone to kind of think about growth but they’re not part of the product?

14. Low Hanging Fruit, aka How Many Times Has the Homepage Been Optimized?

Andrew Chen uber growth interview

The best candidates want to understand the potential reaches of their own impact. How wide and deep are the outcomes that are under your purview? Andrew wants to know how much low-hanging fruit already been picked.

Have people been working on growth or really smart people thinking about it?

As a candidate, I would want to ask how many times has the homepage been optimized in the last 6 months, same with landing pages, same with everything. That gives you a sense of the kind of impact that you’ll be able to create.”

Susan Su leads marketing at Reforge providing training and connections for growth professionals, and is a venture partner at 500 Startups.  Special thanks to the Reforge Collective members who contributed to this post: Elena Verna, Nick Soman, Andrew Chen, Shaun Clowes, and Brian Balfour.

PS. Get new updates/analysis on tech and startups

I write a high-quality, weekly newsletter covering what's happening in Silicon Valley, focused on startups, marketing, and mobile.

What 671 million push notifications say about how people spend their day

The state of growth hacking (Guest post)

Uber Growth Happy Hour

Growth is a system, not a bag of tricks – Manifesto Conference Q&A video/transcript

A Practitioner’s Guide to Net Promoter Score

Quick announcement: The Backstory, a private discussion forum for tech, marketing, growth

Uber’s virtuous cycle. Geographic density, hyperlocal marketplaces, and why drivers are key

My top 2015 essays on Uber, online dating, push notifications, Apple Watch, and more

Personal update- I’m joining Uber! Here’s why

This is the Product Death Cycle. Why it happens, and how to break out of it

Quick update: Quoted in WSJ on dating apps, recent podcast interview, plus recent essays

New data shows losing 80% of mobile users is normal, and why the best apps do better

Photos of the women who programmed the ENIAC, wrote the code for Apollo 11, and designed the Mac

The Next Feature Fallacy: The fallacy that the next new feature will suddenly make people use your product

Why investors don’t fund dating

Why we should aim to build a forever company, not just a unicorn

Ten classic books that define tech

How I first met Eric Ries and also why I’ve ordered his new Kickstarter-exclusive book The Leader’s Guide

This is what free, ad-supported Uber rides might look like. Mockups, economics, and analysis.

Meet me and Eric Ries at a private event on March 21st – here’s how to attend

Personal update: I’ve moved to Oakland! Here’s why.

The most common mistake when forecasting growth for new products (and how to fix it)

The race for Apple Watch’s killer app

My top essays in 2014 about mobile, growth, and tech

Why messaging apps are so addictive (Guest Post)

IAC’s HowAboutWe co-founder: How to Avoid Delusional Thinking in Start-up Growth Strategy (Guest Post)

Mobile retention benchmarks for 2014 vs 2013 show a 50% drop in D1 retention (Guest post)

New data on push notifications show up to 40% CTRs, the best perform 4X better than the worst (Guest post)

Why Android desperately needs a billion dollar success story: The best new apps are all going iPhone-first

Early Traction: How to go from zero to 150,000 email subscribers (Guest Post)

New data shows up to 60% of users opt-out of push notifications (Guest Post)

Why aren’t App Constellations working? (Guest Post)

There’s only a few ways to scale user growth, and here’s the list

Lessons learned adding messaging to a notes app (Guest Post)

Retention is King (Guest Post)

Why consumer product metrics are all terrible

How to solve the cold-start problem for social products