product-metrics

Frameworks for feature metrics
There are two frameworks that I have known to be useful to track feature success:

Dave McClure’s Pirate Metrics

Acquisition

How do users become aware of you?
- SEO, SEM (Search Engine Marketing) , widgets, email, PR (Public Relation), campaigns, blogs…
Function
- Generate attention through a variety of means, both organic and inorganic
Relevant metrics
- Traffic, mentions, cost per click, search results, cost of acquisition, open rate
  Activation
Do drive-by visitors subscribe, use, stc.?
- Features, design, tone, compensation, affirmation…
Functions: Turn the resulting drive-by visitors into users who are somehow enrolled
Relevant metrics: Enrollments, sign ups, completed onboarding process, used the service at least once, subscriptions
Retention
Does one-time user become engaged?
- Notifications, alerts, reminders, emails, updates,…
Functions: convince users to come back repeatedly, exhibiting sticky behavior
Relevant metrics: engagement, time since last visit, daily and monthly active use, churns
Revenue
Do you make money from user activity?
- Transactions, clicks, subscriptions, DLC, analytics…
Functions: business outcomes (which vary by your business model: purchases, ad clicks, content creation, subscriptions, etc.)
Relevant metrics: customer lifetime value, conversion rate, shopping cart size, click-through revenue
referral
Do users promote your product?
- email, widgets, campaigns, likes, RTs, affiliates
Functions: viral and word-of-mouth invitations to other potential users
Relevant metrics: invites sent, viral coefficient, viral cycle time

Google’s Heart Framework:

Happiness

satisfaction, likelihood of recommendation via user study User Survey
Engagement
how much on average user is using your product (by time, sessions…) Analytics
Adoption
the percentage of users that adopt your product after signing up (user onboarding), and/or the percentage of users that adopt a specific feature of your product Analytics
Retention
how many users are still present later Analytics
Task Success
time to complete a task, error rate User tests

(Choose one or two categories in the HEART framework that are the focus of your product or project.)

Metrics by Products

Shareride App (Uber)

(http://www.businessofapps.com/data/uber-statistics/)

two-sided market place
real world component (driver take the courage to drive the cart to pick up the first rider)
paid acquisition
App:
booked rides : weekly trips is the north store metrics for Uber
engagement
Trips/hr per driver
Driver churn (Active since become Uber driver, Uber driver continuation rates half over 12 months)
Customer churn
Average cross fare per trip
total number of active users/drivers this week = last week active number + new user this week - churn user this week + re-engaged user this week
buyer/seller growth (rider, driver)
search/match effectiveness
conversion funnels
rating and signs of fraud
pricing metrics
( - inventory growth (driver growth, region coverage))

Dimension:

Different Products
City
Time of day (rush hours)
Compare to taxi, car rental
gender
age
User:
Earning per hour
Driver online time

Table Design

CREATE TABLE driver (
    user_id INT NOT NULL,
    date_of_birth DATE,
    start_at DATE NOT NULL,
    last_use TIMESTAMP NOT NULL,
    verified BOOLEAN NOT NULL,
    rating FLOAT,
    city VARCHAR(255) NOT NULL,
    country VARCHAR(255) NOT NULL,
    total_trips INT,
    active_status VARCHAR(25),
    total_earning FLOAT,
    acquisition_cost FLOAT,
    PRIMARY KEY (user_id)
)
CREATE TABLE rider (
    user_id INT NOT NULL,
    car_id INT NOT NULL,
    date_of_birth DATE,
    start_at DATE NOT NULL,
    last_use TIMESTAMP NOT NULL,
    verified BOOLEAN NOT NULL,
    rating FLOAT,
    city VARCHAR(255) NOT NULL,
    country VARCHAR(255) NOT NULL,
    total_trips INT,
    active_status VARCHAR(25),
    total_spend FLOAT,
    acquisition_cost FLOAT,
    PRIMARY KEY (user_id)
)
CREATE TABLE ride
    ride_id INT NOT NULL,
    driver_id INT NOT NULL,
    rider_id INT NOT NULL,
    start_at TIMESTAMP,
    end_at TIMESTAMP,
    price FLOAT,
    start_location VARCHAR(255),
    end_location VARCHAR(255)
;
CREATE TABLE car
;

Music App (Spoify)

Common query types:
- KPIs by specified and hoc parameters
- FB audience list for social targeting for AU campaign
- Top tracks by age/ gender by market
- How many free users age 18-30 listen to hip hop in the US?
- What platforms do those hip hop fans like to use?
- How do those platform habits differ from all free 18-30 year olds in the US?
- What cause user not stream after impression?

fact
- MAU / DAU
- Retention
- session length (avg)
- % of total impression (An impression is the number of times one of your visitors had viewed a page where a video or audio was included)
- conversion (after impression, …)
- engagement
dimension
- services (streaming, downloads, radio, discovery…)
- genre
- platform/device (mobile, web, desktop,…)
- user tiers (free, premium)
- age
- gender
- country/city

Messaging App (Messenger)

Messaging is the one thing people do more than social networking like posting on a wall, and the content exchanged over chat is richer, more intimate, and prolific.

Interactions on messenger would be an important source of data, which could then be leveraged to target more relevant content and ads.

It also has the potential to be a powerful platform for developers, designers to sell games, premium stickers and other things that make communicating more delightful or easier, or for businesses to offer customer support, content, services, etc.

Messenger was broken out from the main Facebook app, so I imagine that acquisition and adoption — driving app downloads and retention — is a major goal for this product team. After that, it would be engagement levels for existing users like average response time…

Mobile Game

MAU
MUP
Mobile %

Instagram /Youtube/ Snapshot

Facts
- 3s 10s view… (3s mark) -> attention
- reach
- total minute view
- average watch time and engagement
- total audience
- post engagement
- top location
- completion rate
- view through rate
- audience attention
- unique viewers
Dimension
- traffic
- thumbnail / full video? (outside sources do not have thumbnail like retweets, and other shared links/embeds)

Twitter

Photo upload feature

metrics
- DAU, MAU
- number of sessions
- number of photos per session
- activation
- churn/retention
- session length, feature up time
- happiness (complaints, supports, bugs…)
dimension
- device
- time
- user groups
- age/gender/location
- service (premium, free)

table design

CREATE TABLE user (
    user_id SERIAL,
    age INT,
    gender CHAR(1),
    city VARCHAR(25),
    country VARCHAR(50),
    joined_date TIMESTAMP,
    last_login_date TIMESTAMP,
    PRIMARY KEY (user_id)
)
CREATE TABLE upload(
    upload_id SERIAL,
    user_id INT REFERENCES user (user_id),
    total_photos INT NOT NULL,
    completed_photos INT NOT NULL,
    start_time TIMESTAMP NOT NULL,
    end_time TIMESTAMP
    PRIMARY KEY (upload_id)
)
CREATE TABLE photo(
    photo_id SERIAL,
    user_id INT REFERENCES user (user_id),
    upload_id INT REFERENCES upload (upload_id),
    link VARCHAR(255) NOT NULL,
    upvote INT,
    share INT,
    del BOOLEAN,
    PRIMARY KEY (photo_id)
)

query

-- DAU, MAU
with dau_table as (
      select date(start_time) AS date, count(distinct user_id) as dau
      from upload
      group by date(start_time)
     )
select date, dau,
       (select count(distinct user_id)
        from upload u
        where u.start_time between date - 29 * interval '1 day' and date
       ) as mau
from dau_table;

Collections of common metrics

Click Through Rate

clicks / impressions
CTR > 1:
- You can have multiple clicks during the same session
Invalid traffic (related):
- Accidental clicks that provide no value, such as the second click of a double-click
- Manual clicks intended to increase someone’s advertising costs
- Manual clicks intended to increase profits for website owners hosting your ads
- Clicks and impressions by automated tools, robots, or other deceptive software
- Impressions intended to artificially lower an advertiser’s clickthrough rate (CTR)

Bounce Rate

Total number of visitors viewing one page only/ Total entries to page
It is typically used as a measurement of a website’s overall engagement
potential reasons of bouncing:
- Clicking on a link to a page on a different web site
- Closing an open window or tab
- Typing a new URL
- Clicking the “Back” button to leave the site
- Session timeout

Churn Rate

Customer Churn Rate = (Customers beginning of month - Customers end of month) / Customers beginning of month
Revenue Churn Rate = [(MRR beginning of month - MRR end of month) - MRR in upgrades during month] / MRR beginning of month
- MRR = Monthly recurring revenue

DAU/MAU Ratio:

(#) Daily active users / (#) Monthly active users = (%) DAU/MAU Ratio
measure stickiness of your product

Customer Lifetime Value

Average Revenue Per Account (ARPU) (month) / Customer Churn Rate (month)= Customer Lifetime Value (LTV)
It is an estimate of the average gross revenue that a customer will generate before they churn.

Conversion Rate

number of conversions / number of visitors.
e.g. Paying Subscribers / Total Signups = Signup to Subscriber Conversion Rate

Retention Rate

Aggregate Retention Rate = (#) Monthly active users / (#) Installs
Retention Rate for Cohort = (#) Users retained at the end of time period / (#) installs at start of time period
leading indicator of growth trends and potential revenue
stats: mobile apps on average have a 30-day Retention Rate of 42% and a 90-day Retention Rate of 25%.

Average Revenue Per User

($) Total revenue generated during specific time period / (#) Active users during the same period = ($) Average Revenue Per User

Cost Per Install

($) Ad spend / (#) new installs from ad = ($) Cost Per Install

Cost Per Acquisition

($) total media spend / (#) new customers acquired via media = ($) Cost Per Acquisition

Viral Coefficient

(#) invitations sent per user X (%) conversion rate = (#) Viral Coefficient
a good indicator of your company’s growth trajectory

Marketing ROI

[ ($)Gross profit - ($) Marketing Investment ] / ($) Marketing Investment = (%) Marketing ROI

Session Length

Time user becomes inactive - time app was launched = (#) Session Length
baseline for user engagement
For some apps, long session length leads to frustration (Uber)
SEO
Estimated monthly value = Total value created in the month for this keyword, in dollars

Total monthly searches = Total search volume for this particular keyword each month (estimated)

CTR[Position] = Estimated click-through rate for this keyword, based on current or target position

Value per visit = Estimated value of each visit

Impression

Reach is the total number of people who see your content. Impressions are the number of times your content is displayed, no matter if it was clicked or not.
(To continue, say tomorrow you publish two Tweets to the same 100 followers. Your reach is still 100 users because your follower count didn’t change. However, now you have 200 impressions. Why? Because every single one of your 100 followers saw both Tweets you published.)

Facebook Reach vs. Impressions

On Facebook, reach falls into three different categories:

Metrics to consider

Numbers of Downloads and Installation
Number of Users
Lifetime Value
Retention Rate
Active Users
Session Length
Average Revenue per User
Cohort Analysis
Number of Uses During the Initial Day or Week
Permissions Users Grant to an App
Virality
Crash Analytics

Organic: This represents the number of unique people who saw your content—for free—in the News Feed.
Paid: This is the number of unique people who saw your paid content, such as a Facebook Ad.
Viral: This is the number of unique people who saw your post or Page mentioned in a story published by a friend. These stories include actions such as Liking, sharing or commenting.

–
Just like reach, Facebook impressions are also broken down into three categories, which are viewable in Sprout’s Facebook Pages report:

Organic: The number of times your content was displayed—for free—in News Feed or on your Page.
Paid: The number of times your paid content—such as a Facebook Ad—was displayed.
Viral: The number of times content associated with your Page was displayed in a story published by a friend. These stories include Liking, sharing or commenting.