Metrics Driven Design slides from SXSW, by Joshua Porter

Joshua’s slides on Metrics-Driven Design got tweeted out during SXSW and I wanted to share them.

In general, I think all of the various MVP/customer-development oriented startups out there are struggling with how to incorporate design into their product process. And at the same time, more designery teams are trying to figure out how to get more agile. It’s hard. As someone smarter than me has observed, the vast majority of the MVP-oriented companies end up with pretty uninspired, incoherent products- and they don’t seem to get any better over time. So I think it’s a great challenge for the whole community to get more informed about design and figure out how to really make it work.

Great Google color-testing followup
In particular, in the first few slides there’s a really funny followup to Doug Bowman’s complaints about Google testing shades of blue. These slides claim that in fact, the color choice really did matter, and quite a bit so, and quotes Bing search guy saying that the decision was actually worth $80M. I suppose in retrospect it’s not surprising, because the bluer something is, the more it looks like a link- so given the visual signal, it is meaningful for users over billions of searches.

Metrics-informed versus metrics-driven
All that said, I do have to say that I much prefer the term “metrics-INFORMED design” rather than “metrics-driven.” You should really be driven based on your vision of the product and where you want it to go, not the metrics that you use to validate or learn about your vision. (I first read the distinction of being data-informed over data-driven in a talk by some Facebook product folks, and have much preferred it ever since – this topic probably deserves an entire post by itself).

Finally, the slides
Anyway, the Joshua’s slides are excellent and I’d encourage you to flip through them. The official place to read the details around this presentation is here, on his site. His Twitter is here.

Metrics Driven Design by Joshua Porter

Like this post?
Get new updates via newsletter..

http://twitter.com/lauraklein Laura Klein

Very interesting post. And I love pretty much everything Josh Porter writes.

I agree that metrics should not drive actual design decisions. However, metrics can be extremely useful in driving overall product decisions.

In other words, metrics tell you what your biggest problems or opportunities are, and where you should probably concentrate your design efforts next. They do nothing to tell you why your users are having problems or how you should solve those problems. That’s what qualitative research and great design vision are for!

For example, your metrics might tell you that your biggest problem is that your retention sucks. It’s up to you to figure out why it sucks and then come up with a good way to fix that problem. The best part is, metrics tell you, at the end of the project, whether your design vision was right for both users and your business, so you can improve your knowledge of your user and make better design decisions in the future.
http://twitter.com/StartUpAndGoBiz Matthew Crossett

Thanks for this great information! Just posted something similar – would love to hear your thoughts! If you ever want to link up sometime over GoToMeeting to talk about a possible collaborative article, let me know!
http://takecareof.biz

@andrewchen