It’s time to stop… vanity metrics

I’m writing this during the South Carolina Republican primary.  The votes haven’t started being counted yet, but I know who is going to win.  Because I know that Ben Carson has 35% of the Facebook likes among GOP contenders in the state; Trump is second at 25%.  Thus, Carson will get approximately 35% of the vote.

What?  Doesn’t it work that way?  Facebook likes aren’t a reliable indicator of support, donations, interest, or almost anything else?

The bitter truth: Facebook likes are a vanity metric.  They have little to do with your ultimate goal of constituent acquisition, donor conversion, and world domination, yet people will still ask what that number is.  And when they hear it, they will nod, say that that’s a good number, and ask what we can do to increase it.

That’s when a tiny little part of you dies.

So, in our Things To Stop Doing, we have vanity metrics.  These metrics may make you feel good.  They may be easy to measure.  And some of them may feel like a victory.  But they bring you little closer to your goals.  We are creatures of finite capacity and time, so the act of measuring them, talking about them, or (worst of all) striving for them drains from things that actually matter.

Facebook likes and Twitter followers are probably some of the better-known vanity metrics.  But they are far from the only ones.  And while some of these are partly useful (e.g., Facebook likes is an indicator of a warm lead repository for marketing on the platform), there’s almost always a better measure.

Because it always comes back to what your goals are.  Usually, that goal is to get people to take an action. Your metrics should be close to that action or the action itself.

Without further ado, some metrics to stop measuring.

Web site visits.  Yes, really.  This is for a couple of reasons:

  1. Not all visitors are quality visitors.  If you’ve been using Web site visits as a useful metric, and wish to depress yourself, go to Google Analytics (or your comparable platform) and see how long visitors spend on your site.  Generally, you’ll find that half or more of your users are on your site for more than 30 seconds.  Are 30 seconds long enough for people to take the action you want them to take on your site?  Not usually (except for email subscribes).

  2. Not all visitors are created equal.  Let’s say you find that people coming to your site looking for a particular advocacy action sign up for emails 10% of the time; those who come looking for information about a disease sign up 5% of the time; those who look for top-line statistics sign up 1% of the time.  Which of these is the most valuable visitor?

    This isn’t a trick question.  You would rather have one person looking for advocacy actions than nine people looking for stats.  Except that the metric of Web site visits lumps them all into one big, not-very-useful bucket.

These are both symptoms of the larger problem, which is that if you had to choose between two million visitors, of whom 1% convert, and one million visitors, of whom 3% convert, you’d choose the latter.  Thus, potential replacements for this metrics are visits to particular pages on the Website where you have a good idea of the conversion rates, weighted Web traffic, and (most simply) conversions.

Mail acquisition volume.  You get the question a lot – how many pieces are we sending in acquisition?  Is it more or less than last year?  And it’s not a bad estimate as to a few different things about a mail program: are they committed to investing in mail donors?  Is the program growing or shrinking?  What are their acquisition costs?

But from a practical perspective, all of these things could be better answered by the number of donors acquired (and even better by a weighted average of newly acquired donors’ projected lifetime values, estimated from initiation amount and historical second gift and longer-term amounts, but that’s tougher).  A good rule of thumb is:

Never measure a metric that someone could easily game with a counterproductive action.

And you can do that with mail acquisition volume by going on a spending spree.  Of course, you can also do that with donors acquired, but it will spike your cost per donor acquired, which you are hopefully pairing with the number of donors acquired like we recommend in our pairing metrics post.

Time on site.  You notice that people are only spending an average of 1:30 on your Web site, so you do a redesign to make your site and content stickier.  Congratulations – you got your time on site up to 2:00!

Someone else notices that people are spending 2:00 on your Web site.  They work to streamline content, make it faster loading, and give people bite-sized information rather than downloading PDFs and such.  Congratulations – you got your time on site down to 1:30!

Therein lies the problem with time on site – whatever movement it makes is framed as positive when it could be random noise.  Or worse.  Your sticky site may just be slower loading and your bite-sized content may just be decreasing conversion.

So another rule of good metrics:

Only measure metrics where movement in a direction can be viewed as good or bad, not either/both.

Here again, conversions are the thing to measure.  You want people to spend the right amount of time on your site, able to get what they want and get on with their lives.  That Goldilocks zone is probably different for different people.

Email list size.  While you totally want to promote this in social proof (like we talked about with McDonalds trying to get cows to surrender), you actually likely want to be measuring a better metrics of active email subscribers, along the lines of people who have opened an email from you in the past six months.  These are the people you are really trying to reach with your messaging.

When you remove metrics like these from your reporting or, at least, downplay them, you will have fewer conversions with your bosses that ask you to focus on things that don’t matter.  That’s a win for them and a win for you.

I should mention that I am trying to build my active weekly newsletter subscribers.  Right now, we have an open rate of 70% and click-through rates of 20%+, so it seems (so far) to be content that people are enjoying (or morbidly curious about).  So I’m hoping you will join here and let me know what you think.


It’s time to stop… vanity metrics

One thought on “It’s time to stop… vanity metrics

Please share your thoughts.

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s