The problem(s) with social media monitoring technologies
by asi
I’ve been meaning to write this post for a long time now and while I’m 2 days behind deadline to give one of my clients another monthly report I have to take a break and spit it off my chest.
Over the past 8 months I tried and tested practically all of the key players in ‘social media monitoring services’: Radian6, BuzzMetricsBrandwatch, Attentio, Techrigy (SM2) etc. And I have very few good words to say about them. The few good words will be in relation to the people who work at these companies – these are usually nice people. But the software and service is verging on epic fail – very problematic and clunky solutions wrapped in a big bold shiny (but rather hollow) promises. (OK maybe I’m being a bit too harsh….but that’s the result of two long nights sorting out monitoring dashboards)
I don’t mean and don’t have the time to give you a fully detailed review on each one of the tools I’ve tested. Honestly I can’t be bothered. so the following points i will make are rather generic and represent similar issues I had with ALL of them. I will bullet point so to avoid rambling…
1. The technology is fairly stupid. It either don’t do what it says on the tin or do it quite badly. It is supposed to be simple: based on keyword configuration, the software scrolls the social web and collect mentions of your keyword(s). It then supposes to analyse themes and influence ranking and sentiment etc, which it does with limited accuracy . For example, will pick up any header, ad sense or footer mentions of your keywords even if it’s in the totally irrelevant context. If your brand name is pretty generic you are in deep sh*t. Hours of configuration and exclusions awaiting you.
2. Unreliable data. The most important thing to understand is that the software simply provides you with piles of data. Before you can extract anything meaningful from this data you have to go through hours and hours of spam filtering which can be very tedious if you are dealing with 1000s of mentions every week/month. In some occasions I had over 50% irrelevant data coming through my dashboard. Additionally, the spiders cannot access all social spaces and sometimes the most important conversations are blocked.
3. Sentiment analysis is flawed. Again, this is part of the limitation of the technology. The software analyses keywords, not human emotions and, on average, the software gets it wrong 30% of the data because human emotions are subtle and complex and not easily categorised by software – we are not there yet.
4. Region specific data: for global brands, social media have very strong global element as well as clear regional bent (forums, blogs, networks etc). This is tricky especially if you are working with a regional client (e.g Huggies UK). Problem is for the software it’s not about where you are but which domain are you using. So reliable geo / regional analysis is, in many cases imposible to carry or not complete so need to be complemented with manual search.
5. Influence analysis is flawed. Well, the concept of influenced is flawed so of course technologies of measuring it are flawed as well. Similar to sentiment, the technology is just not as clever as they want you to believe. It is based either on bogus metrics or just irrelevant, obsolete ranks .
6. Unreliable data + partial data = Unreliable stats and visualisation. Because of the above, the one good thing for which you actually want to use this software, i.e. the funky visualisation tools can’t be used before you make quite a lot of amends and refinements to the data
7. Time consuming. Because all of the above, the reality is that while thess companies provide you with piles of data and funky visualisations the profound unreliability of the software means you have to sit for hours and days and configure the dashboard, refine the data, correct the scores, filter the spam, get rid of irrelevant data AND THEN, AND ONLY THEN you can start making some meaningful analysis.
8. Price. This varies significantly but the fact is that you pay just for the data and license fees to use the software. For the level of service you don’t get value for your money.
The overall feeling is of a serious problem of reliability of the results across the different functionalities of the software. In essence it’s like paying a market research firm for 10000 people’s survey results + 300 focus groups transcripts + 687 depth interviews and what you get is just the raw data with 30% of it irrelevant/spam you actually never asked for. Now you have to invest a lot of time in sorting out the data, analysing and interpreting everything you have. It’s doable but sometimes not really cost effective.
If you want to get the best out of these tools you almost have to plan for someone to sit 2-5 hours a day and sort the data (depends on the volume it’s a full time job.
At the moment I’m working with both SM2 and Brandwatch on different clients for different reasons and I can’t say I’m satisfied with either of them – it’s more of a default until better solutions will be developed. Until the software will be better at sorting through context and semantics it will remain very problematic and unreliable.
Solutions:
- Pay for them to do the analysis for you or always plan/cost for more man hours than you expect
- DIY dashboard with free services- search online there are quite a lot of how-to posts
- Wait for google to provide this service for free. This will either be the end of the bubble or they all will turn into market research 2.0 where you pay for the reports not just the data
naff said. back to my dashboard.
[...] No Man’s Blog – The problem(s) with social media monitoring technologies A list of issues found with all social media analytics software. Are they worth what you pay? (tags: socialmedia monitoring analytics tools metrics analysis) [...]
Hi Asi,
Was my previous comment sent to you?
-Kelly
Interesting post, although I think you have to separate self-service solutions (relatively cheap but altogether over-simplistic as technology, your own words ‘simply is not there’) from solutions that involve a lot of manual analysis done by the provider.
The latter are of course much more expensive but they provide somewhat meaningful data and analysis.
They are not 100% perfect yet but then, as some other guy wrote, do we really need 100% accuracy?
Hey dude,
Read Write Web are big fans of yours. So much so they copied you… check this:
http://www.readwriteweb.com/archives/sentiment_analysis_is_ramping_up_in_2009.php#more
Ben
Hi Asi,
I appreciate the frustration you experienced. I have also tried out most of these tools, and our experience is not nearly half as bad. We have managed quite well and do realize that the data that we get is very relevant. It may not be perfect but it is not a waste by any means.
Not seeing any easy way for the data clean up to happen, we have done the next best thing. We have our own team of Social Media monitoring specialists who we use to read through all posts, remove spam, to trend spotting, do sentiment spotting and also to classify the conversations as per predefined tags. This makes the data very actionable.
We do this for ourselves and also offer the service to others, as our Social Media monitoring effort. More about this at our website at http://www.socialwavelength.com.
[...] have been several recent write-ups about social media monitoring tools. The majority of the reviews surround flaws and [...]
[...] There are a number of challenges when it comes to social media monitoring solutions for business and you can find out some things to watch out for in Asi Sharabi’s blog post. [...]
Plannersphere Top 20/20 – September 2009…
The top 20 planning blogs in September 2009 and the top 20 planning blog posts from August 2009….
[...] of these, entititled “The problems with Social Media Monitoring technologies“, seems to have decent web findability as it was sent to me a couple times by friends and [...]
Social Media Monitoring vendors are full of shite.
It is that simple.
Try ASOMO, http://www.asomo.net. There is a human involved in the service, so sentiment analysis is accurate. Results are great and I don´t have any of the above problems you mentioned. Good luck!
Sorry to arrive at this post so late in the day. Music to my ears. The problem with all of the black box solutions is they are about monitoring places – not spaces. In my experience you can only do this with a bespoke, DIY panel approach. See http://richardstacy.com/2009/07/16/is-much-of-social-media-monitoring-snake-oil-or-have-i-missed-something/ and http://richardstacy.com/2009/09/04/what-social-media-monitoring-and-the-english-channel-have-in-common/
[...] This rant has gained me few haterz in the industry and an invite to speak in a panel at the monitoring social media conference. Whether client or agency side, PR, market research or marketing if listening and monitoring ‘the conversations’ is something you should know about I highly recommend you to come along. [...]
You Can Try
Social Master Tool 2.0 – Social Media Management and Analytics Tool
[...] the dying days of the summer, Asi Sharabi wrote a late night rant about the state of social media monitoring. He directed his anger squarely at the many social media monitoring services that have emerged in [...]
[...] against such monitoring tools was also a point of discussion with Asi Sharabi’s controversial blog post inspiring a lively panel debate. No solution (right now anyway) is perfect and the current [...]
[...] followed on, what was wrong with social media monitoring , including Asi Sharabi, who’s blog post on the same topic, inspired the conference. Some of the main issues discussed [...]
[...] The problem(s) with social media monitoring technologies [...]
[...] No Man’s Blog – The problem(s) with social media monitoring technologies – On social media monitoring and failing services in that field. [...]
[...] //No Man’s Blog [...]
Hey Asi,
I can’t agree with you more. Spot on with your post.
Check out beRelevant (http://www.berelevant.com).
Really well thought out and thought provoking post. As an industry we have a lot of work to do, it’s clear. First of all, sentiment measurement is no trivial task. Especially in a medium like Twitter, sometimes I can’t even tell what the sentiment is, and I am human. We (Biz360) advise clients that this data is directional, and should be only part of the brand analysis. We, as well as our competitors, are constantly looking for ways to improve sentiment analysis. We actually allow to score sentiment on the topic / object or the article level, which helps a bit with accuracy.
One thing that is of value, in my opinion, (and we offer), is topic discovery, that helps you figure out most relevant phrases that occur around the search topic. To help uncover what’s going on with your product or brand, you can take a look at such a tag cloud, take a look at a trendline, and sentiment trendline and dig in deeper to see where engagement efforts should be applied. We allow our users to sort by impact and reach, so that they can focus in on the most relevant articles and mentions.
If you would like to give Biz360 Community a whirl, please feel free to ping me.
Maria Ogneva
Director of Social Media, Biz360
@themaria @biz360
[...] than the previous one. However the fundamental purpose is still not being served. No Mans blog puts better light on this matter and you should read his [...]
[...] rien et nombre d’experts soulignent les insuffisances de ces solutions (voir par exemple l’excellent billet d’Asi Sharabi qui a suscité une importante polémique dans le secteur). Parmi ces fonctionnalités que tout [...]
Its a pity clients don’t give more weight to people’s experience particuarly when the social media tools don’t provide the full answer. We are a numbers driven society and I’m not sure that is the best measure for social media.
[...] The Problem(s) With Social Media Monitoring Technologies [...]
The comments in the post can be applied to technologies and software in general. In this age of rapid Internet development, software (websites) are pushed out so quickly, it sometimes forgets about the important matters such as usefulness, not to mention bug-ridden software due to lack of sufficient development time and process.
There is doubt and dispute on the legitimacy of Alexa rankings. It is monitored through the Alexa toolbar without controls on abuse. And what if my site is not visited by people with the Alexa toolbar?
[...] “The reality is that while these companies provide you with piles of data and funky visualizations, the profound unreliability of the software means that you have to sit for hours and days and configure the dashboard, refine the data.” – Asi Sharabi [...]
[...] Social Media ROI, Wendy Troupe's Perspectives by Wendy Troupe on August 19th, 2009 Asi Sharabi’s Private Selections, “No Man’s Blog” recently posted his feelings about what is lacking in the field of social media [...]
[...] like to thank people like Asi Sharabi, Murray Newlands, Ken Burbary and the people at ThreeMinds Organic for inspiring and to some [...]
It sounds related to a data mining/analysis problem. They are typically non-trivial to solve. A couple of years ago, I looked at the Netflix Prize where they want to know the best recommendation for members based on a 5-point rating system provided by members. The key is to try to spot correlations that are significant in the data. Gathering the data is the simple thing that simple software can do; analysing it in a meaningful way is a difficult problem.
[...] The Problem(s) With Social Media Monitoring Technologies [...]
[...] The Problem(s) With Social Media Monitoring Technologies [...]
[...] The Problem(s) With Social Media Monitoring Technologies [...]
There are a few other challenges that are sure to arise from trying to tackle the technology problems such as budgeting, how much a technology solution cost to develop? Well, that depends on what analytics and data solutions you are currently using and if you already have any technology solutions in place that allow them to communicate with one another. As you can see, developing technology solutions is not an easy task, it definitely takes work to get done but it should be a very rewarding endeavor as this should also help your company understand the value of marketing investments as well as the value from social media.
Hello,
The best simple and the best Social Media Monitoring tool is here. Have a look and I am sure you will find the best.
[...] The Problem(s) With Social Media Monitoring Technologies posted under business support, marketing ideas [...]
What is wrong with the simple-minded analysis of measuring source of traffic to your website from social media sources? Google Analytics provides you with the ability to track visitor to your site and provides reports showing you what is coming from where.
Between poor software monitoring systems and Google Analytics, I would surely choose the latter.
[...] There are a number of challenges when it comes to social media monitoring solutions for business and you can find out some things to watch out for in Asi Sharabi’s blog post. [...]