The problem(s) with social media monitoring technologies
I’ve been meaning to write this post for a long time now and while I’m 2 days behind deadline to give one of my clients another monthly report I have to take a break and spit it off my chest.
Over the past 8 months I tried and tested practically all of the key players in ‘social media monitoring services’: Radian6, BuzzMetricsBrandwatch, Attentio, Techrigy (SM2) etc. And I have very few good words to say about them. The few good words will be in relation to the people who work at these companies – these are usually nice people. But the software and service is verging on epic fail – very problematic and clunky solutions wrapped in a big bold shiny (but rather hollow) promises. (OK maybe I’m being a bit too harsh….but that’s the result of two long nights sorting out monitoring dashboards)
I don’t mean and don’t have the time to give you a fully detailed review on each one of the tools I’ve tested. Honestly I can’t be bothered. so the following points i will make are rather generic and represent similar issues I had with ALL of them. I will bullet point so to avoid rambling…
1. The technology is fairly stupid. It either don’t do what it says on the tin or do it quite badly. It is supposed to be simple: based on keyword configuration, the software scrolls the social web and collect mentions of your keyword(s). It then supposes to analyse themes and influence ranking and sentiment etc, which it does with limited accuracy . For example, will pick up any header, ad sense or footer mentions of your keywords even if it’s in the totally irrelevant context. If your brand name is pretty generic you are in deep sh*t. Hours of configuration and exclusions awaiting you.
2. Unreliable data. The most important thing to understand is that the software simply provides you with piles of data. Before you can extract anything meaningful from this data you have to go through hours and hours of spam filtering which can be very tedious if you are dealing with 1000s of mentions every week/month. In some occasions I had over 50% irrelevant data coming through my dashboard. Additionally, the spiders cannot access all social spaces and sometimes the most important conversations are blocked.
3. Sentiment analysis is flawed. Again, this is part of the limitation of the technology. The software analyses keywords, not human emotions and, on average, the software gets it wrong 30% of the data because human emotions are subtle and complex and not easily categorised by software – we are not there yet.
4. Region specific data: for global brands, social media have very strong global element as well as clear regional bent (forums, blogs, networks etc). This is tricky especially if you are working with a regional client (e.g Huggies UK). Problem is for the software it’s not about where you are but which domain are you using. So reliable geo / regional analysis is, in many cases imposible to carry or not complete so need to be complemented with manual search.
5. Influence analysis is flawed. Well, the concept of influenced is flawed so of course technologies of measuring it are flawed as well. Similar to sentiment, the technology is just not as clever as they want you to believe. It is based either on bogus metrics or just irrelevant, obsolete ranks .
6. Unreliable data + partial data = Unreliable stats and visualisation. Because of the above, the one good thing for which you actually want to use this software, i.e. the funky visualisation tools can’t be used before you make quite a lot of amends and refinements to the data
7. Time consuming. Because all of the above, the reality is that while thess companies provide you with piles of data and funky visualisations the profound unreliability of the software means you have to sit for hours and days and configure the dashboard, refine the data, correct the scores, filter the spam, get rid of irrelevant data AND THEN, AND ONLY THEN you can start making some meaningful analysis.
8. Price. This varies significantly but the fact is that you pay just for the data and license fees to use the software. For the level of service you don’t get value for your money.
The overall feeling is of a serious problem of reliability of the results across the different functionalities of the software. In essence it’s like paying a market research firm for 10000 people’s survey results + 300 focus groups transcripts + 687 depth interviews and what you get is just the raw data with 30% of it irrelevant/spam you actually never asked for. Now you have to invest a lot of time in sorting out the data, analysing and interpreting everything you have. It’s doable but sometimes not really cost effective.
If you want to get the best out of these tools you almost have to plan for someone to sit 2-5 hours a day and sort the data (depends on the volume it’s a full time job.
At the moment I’m working with both SM2 and Brandwatch on different clients for different reasons and I can’t say I’m satisfied with either of them – it’s more of a default until better solutions will be developed. Until the software will be better at sorting through context and semantics it will remain very problematic and unreliable.
- Pay for them to do the analysis for you or always plan/cost for more man hours than you expect
- DIY dashboard with free services- search online there are quite a lot of how-to posts
- Wait for google to provide this service for free. This will either be the end of the bubble or they all will turn into market research 2.0 where you pay for the reports not just the data
naff said. back to my dashboard.