ataussig.com

The HEART framework

Kerry Rodden, a UX researcher at YouTube, wrote one of the best posts I’ve seen recently on managing product with meaningful metrics.

Her team has developed something she calls “the HEART framework.” It has similarities to Dave McClure’s Startup Metrics for Pirates, but it goes a level deeper to break out how one arrives at a given metric: first set Goals, then look for Signals you can measure, then develop Metrics based on those Signals.

She visualizes the approach in the following 5x3:

If you manage product, or work with people who do, I recommend reading the entire article and filling out your own matrix. Here’s an excerpt defining each term of HEART:

• Happiness: measures of user attitudes, often collected via survey. For example: satisfaction, perceived ease of use, and net-promoter score.
• Engagement: level of user involvement, typically measured via behavioral proxies such as frequency, intensity, or depth of interaction over some time period. Examples might include the number of visits per user per week or the number of photos uploaded per user per day.
• Adoption: new users of a product or feature. For example: the number of accounts created in the last seven days or the percentage of Gmail users who use labels.
• Retention: the rate at which existing users are returning. For example: how many of the active users from a given time period are still present in some later time period? You may be more interested in failure to retain, commonly known as “churn.”
• Task success: this includes traditional behavioral metrics of user experience, such as efficiency (e.g. time to complete a task), effectiveness (e.g. percent of tasks completed), and error rate. This category is most applicable to areas of your product that are very task-focused, such as search or an upload flow.

A metaphor for your product

Finding the direction and voice of a technology product early on can be one of the most challenging tasks an entrepreneur faces. Human beings are pattern recognition machines and, as such, products that make direct reference to objects or experiences we have in the physical world tend to resonate better with users.

Numerous examples of this phenomenon litter the landscape of tech:

• GUIs: Apple introduced to the general public a GUI with a directory structure based on a familiar organizational paradigm: folders, files, desktop, etc.To some extent, Dropbox has extended this metaphor to the cloud.
• Tabbed browsing: Amazon popularized the ubiquitous notion of tabbed browsing that we now see everywhere on the web. Even the browser itself now has tabs. The physical metaphor was obviously the file cabinet.
• Profile pages: When Zuckerberg created the Facebook profile page, Harvard still had its own physical facebook that was handed out to freshmen. The use case for the physical facebook I observed (finding the hottest classmate in your dorm) was the same one I saw online when I registered for Facebook my sophomore year.
• Journals and scrapbooks: Path and Pinterest have more in common than their first letter and nasty user growth. They are both digital manifestations of an offline activity: the chronicling of one’s own life through personal anecdote and memorabilia. Path is the journal you keep. Pinterest is that dusty pinboard in your bedroom. The only difference is that many more people now see these artifacts of your personality.
• Constant updates: Dick Costello of Twitter recently said he wanted to make checking Twitter “like checking your watch.” This is a physical metaphor of a compulsive behavior many have that translates well. Checking the time is one step removed from checking what’s happening right now.
• Doing things with friends: Apps like Turntable.fm or even Draw Something are direct metaphors of activities that friends used to do together offline: get together and spin records, or play a party game together.

This list is just the beginning. I’d love to hear some additional examples you guys can think of and will add them if you tweet me at @ataussig.

Why I’m leaving Pandora for Turntable.fm

I wrote a post for Mashable a little while ago, in which I claimed the three most important qualities in an internet product were its appeal to (1) ego, (2) serendipity, and (3) familiarity. Serendipity is especially important for applications where discovery is involved.

My mainstay for music discovery, in particular, has been Pandora, but over the last 6 months I’ve officially shifted from that service to startup Turntable.fm, which is nailing serendipity all over the place. This shift is kind’ve a big deal for me, if you know how serious I am about music. I’m also serious about not paying for things (read: cheap!), so while I use the free version of Spotify, I haven’t yet turned on its full feature set.

Turntable gets serendipity very right, mainly because its real-time human curation captures something that Pandora simply misses. Turntable is comfortable with large amounts of variability from song-to-song, while Pandora seems to pick songs which are sequentially more similar. By way of example, here’s what a sequence of songs may look like on Pandora:

• Tool, “Sober”
• Alice in Chains, “Would?”
• Nirvana, “Smells “Like Teen Spirit”
• …some other 4-chord, power-chord-laden grunge tune with soaring vocals

Starting with the same song, here’s what a sequence may look like on Turntable in the 90’s Alt Rock room:

• Tool, “Sober”
• Temple of the Dog, “Call me a Dog”
• Hole, “Violet”
• Blind Melon, “No Rain”
• Third Eye Blind, “Crystal Baller”
• Bush, “Machinehead”
• Dog’s Eye View, “Everything Falls Apart”

What drives the difference, and makes Turntable a better discovery experience, is that the DJs in Turntable pick songs that appeal to the room, but also challenge its norms. Going from “Sober” to “No Rain” would be a huge leap for a computer algo, but anyone alive in the early 90’s who listened to one probably liked the other, or at least was willing to check it out. In the event Turntable’s serendipity machine gets it wrong, of course, the room can “LAME” the song out of the playlist, so DJs do get strong feedback when they push the envelope too far. Generally, though, experimentation by humans is welcomed in Turntable, and some of the best DJs earn their stripes with awesome non-sequitor choices. 

I think we’ll see more adoption of human curation in coming years, especially in this era of social media where humans expect to be connected to other humans, not computers. I’d love to see Turntable’s approach in other media formats that I consume on a regular basis.