Setting KPI’s is a bitch
by asi
This is a problem I’ve been struggling with for a long time, something Iain and I had discussed a lot in the past but always reached some dead end. I’m surprise how thin is the blogversation around it (well not really as anything around digital measurement is still pretty rudimentary)
Here is the situation:
So the client signed off your proposed idea/solution for a new project and you need to set targets. There are two immediate challenges:
1. How to set up traffic and interactions volumes?
This is the most straightforward one and surprisingly the one that no one likes to talk about . What does success look like? 50K UV? 3million? 12967 fans? 37K plays? 3.4min avarage spent on site? (this is of course a very partial list)
2. How to set up the metrics for the VALUE of traffic and interactions
What does it mean that 34K people interacted with you? or that you had 582 blogs writing about your project etc?
At the moment, and correct me if i’m wrong in many cases both are founded purely on guesswork (and a bit of common sense). We know that unlike traditional media (TV etc) every digital project is different in shape, form and colour from the previous one so there is not much to compare to and benchmark against – a company blog is not a campagin microsite is not a facebook fan page is not a youtube video is not a game etc.
Now, i know that in North America it is a bit easier as traffic volumes are not guessed, they are usually bought i.e. you set up target and buy media to meet these targets, but in the UK as far as i can tell there are some unreasonable hopes and expectations from ‘digital’.
I’d like for now to ask your help on the first point as there is some thinking going on on the second. So let’s go back to the very basics – How the hell do you set up traffic and interactions volumes? How to you decide if success is 50K or 500K? What do you benchmark your numbers against?
How do you (and clients) come to a MUTUAL, REALISTIC and REASONABLE set of targets during planning?
I think that this issue deserve a proper meeting over coffee with some clever minds but for now – please share your wisdom in he comments.
Thanks in advance for your contribution
I think it depends on the brief. By talking about volume, it suggests that the objective laid down in the brief is about volume. In this case, I think its simply a matter of does ‘social media’ activity provide a more efficient way of driving traffic to the campaign microsite than PPC or paid display media? If not, then its not what you should be doing.
If its about generating interactions, then you need to ask the client very straight up – do you want a lot of low level interactions or a small number of high level interactions? For the budget you have – you can’t achieve the best of both worlds.
i.e. do you want a Facebook page where people might leave comments, or do you want to create an ARG where only 20 people will ‘get it’, but for the duration of the game they’ll devote massive portions of their lives to your campaign.
If its not as complex as an ARG but isn’t as simple as just a Facebook page – then a combination of made up metrics help
Assign a social value to actions to create a score card. A video view = 1, a video rating = 1, comment on a video = 2, creating a mashup = 20. Potentially base the values over the estimated time taken for the action.
Actually come to think of it… this doesn’t help you set up KPI’s at all… lol. I reckon just always make sure the brief is rock solid, and compare the value that the campaign provides against related media like display and PPC – making sure that you compare them on an equal footing using a scoring system. A display campaign will drive 100,000 website arrivals, but 95% will bounce. A social media campaign might only drive 10,000 arrivals, but with only 5% will bounce etc.
Oh well! I’ll go back to work now.
Hurrah! I’m really glad that you raised this Asi, it’s something I too have battled with for some time. I can’t wait to hear what intelligent and insightful comments people have to share.
I don’t claim to be an expert in such matters, but I suppose I’ve got a few thoughts.
First of all let us deal with ‘performance indicators’. I suppose it should be exactly that, a metric that gives an indication of performance or success. There is of course a natural relationship between the KPI and the communication objective, so if the objective is ‘sell more onions’, the KPI is ‘number of onions sold’. We know that in reality that objectives are often more complicated than that and finding the most appropriate metric(s) can often be very challenging.
Now to your actual question – what do you benchmark against? I don’t think you can. Benchmarking is very difficult because most of the time there are very few comparable projects (with the same objectives, audience, market, media spend etc).
From my experience, setting targets is (as you suggest) often based on common sense and guess work. I suppose that ideal clients would understand metrics better; the caveats that they come with, the difficulty and flaws of benchmarking, and (this is a personal ‘bug bear’), that they don’t always fit into their traditional ways of viewing success.
Great question. I think the KPI is often an issue becuase the pupose of the activity in not fully thought out (particularly in the social sphere), and the value is rarely a single thing. We also suffer from short-termism. A Facebook page should be a platform for interaction for as long a Facebook is the place to be, which could be another 10 years. KPI that, baby.
I think a Facebook page should be as much a customer service tool as a marketing one. Like a call centre. What is the KPI for a call centre? What is the measure of a happy customer? I think Orange have this cracked – their call centre people are lovely and very helpful. I asked one once why this was, and she said “Oh, I don’t know. They train us on the phones and that, but then they just kind of tell us to be ourselves”. Measure that…..
Also, digital suffers because everything works. Read revolution or new media age, look at the IAB website. It is literally impossible to fail at digital. I have never read about a strategy that didn’t work, and work brilliantly. I’ve worked on a few, but maybe I’m shit?
This probably isn’t helping, is it? I think the way to measure it is to try to put a value on the interaction, not on volume. This can be done through ugly but necessary tools like exit surveys, panels, telephone research etc. Define how you want the interaction to affect or influence behaviour, then measure for that. Then multiply that by your volumes. Our friends in DM should be all over this.
Great question.
Is it possible that one of the reasons you’re having problems assigning real values is that there is no real value? Bear with me here.
People who make money online (say roughly, catalogue retailers, financial services providers, travel agents, paid-content suppliers, software developers, SaaS providers, and publishers) know pretty much the value of their interactions.
Real value for many clients is still measured in hard currency (or sales). For other clients (our healthcare clients, for example) it’s measured in behavioural transactions — things like “number of people being screened for disease x”.
Ultimately, I think we have to be able to tie things to the bottom line. That seems like a pretty good benchmark.
Asi, I’m sure you’re already aware of Google’s benchmarking service, but if not….
http://tinyurl.com/bb4k8m
Hi Asi, good post, KPIs are frequently discussed in the same breath a “business case” and I guess the 2 are closely related.
One of the more successful strategies we’ve had is to align the softer social metrics with their existing ATL framework (e.g. COI = Artemis).
On a more direct note, Scoble made a great point on TWIT last week about twitter and site statistics (no. 181, about 1hour 29 mins in)
“There is a direct correlation, If they’re talking about you on tweetdeck your hits are Up, if not they are down”
clearest case for social that i’ve heard.
A great question. Foremost, as Razmi says, it depends on the brief. DR campaigns will (should?) have pretty realistic ideas of KPIs, whereas branding or awareness campaigns may not.
In my head, there is a sort of dial where reach and (average?) engagement are inversely proportional. They aren’t mutually exclusive, but I believe a campaign should focus on one and not both. Each campaign needs to decide where they want to be on that dial
Thanks so much for your insightful comments. I was hoping to get some more ideas on point one in relation to (mainly) brand activities.
Not necessarily social media but general digital activity – you create something for your consumers to interact/engage with – how do you know HOW MANY is success? (and OF COURSE that in many cases it’s the quality and depth of engagement rather than volume of traffic/interaction/fans/readers but still – if we want clients to shift more £££ to digital, we should be able to tell them what success look in the medium of permission etc.
I’m starting to believe that in that sense we should start working like traditional advertising. The client will set the targets – i.e “I want 25K people interacting with your work”. Our answer should be: fair enough – pay for media worth 25K visits – anything beyond that is a ‘viral bonus’ that is due to the strength of the creative and so the agency will be rewarded accordingly.
Anyone with me on this?
There’s only one true KPI: cashmoney.
And there’s plenty of opinion on how to compare brand measures to share value and longterm revenue.
I think the stumbling point is measuring the effect on brand resulting from digital campaigns. Reach of a single advert (or blog or Fb page or campaign thought etc) is rarely high enough to use traditional methods to measure this.
Digital branding activity always provokes a trackable reaction. What hasn’t been solved is how to attribute value to each different type of interaction (views, chatter, sales, ect.). I think the answer lies in buzz monitoring because it’s the broadest spectrum here. And once digital spend is high enough then there will always be some sort of chatter as a response. The key is not the level of buzz but in the sentiment of the chatter. The analysis on this could be awesome. Plenty of work by Onalytica on attributing buzz uplift to sales.
Nice post. Big topic. Time to us to sort it.
Hmmn I think I’m against you on that last part Asi. Clients already do that – and agencies like Go Viral serve their purpose for things like this.
Paying for hard metrics and encouraging that practise will only result in pay per post and rev share for video view type activity. I know that isn’t what you mean – but that is what would happen.
In this industry and economy, agencies can’t run if they’re having spending more resource than they’re being paid for on making something go naturally viral. The idea of creating something which is naturally viral is oxymoronic anyway as if it happens naturally, then it can’t be forced.
As Brad always says… “Money in… money out” – and guaranteeing results through practices which can’t be guaranteed would bring agencies to their knees if they tried it. As a business owner – I wouldn’t want to be in that position at all.
Also if the business objective is 25k visits and nothing further, then clients would be better informed to spend their money on PPC which will do that much more cost effectively.
Think you’ll have to re-think this my friend.
-tobeconfirmed-
ôîðåêñ áàíê âòá…
ôîðåêñ áàíê âòá …
Who thought up or invented the KPI?