Today, two seemingly unrelated but actually very similar discoveries: socialmention is offering sentiment analysis among other metrics; and SpinVox uses people to transcribe messages.
Humans as machines
First, the second. SpinVox.They offer voice-to-text conversion which is something of a holy grail for computing, and given my past interest in AI, I found the proposition fascinating. I haven’t used the service myself but I’ve followed their progress keenly over the past couple of years, having actually done some work for them. At the time, I met Daniel Doulton and Christina Domecq, and they were a powerhouse. You got the feeling that everything, and anything, was possible.
And it turns out that yes, everything was possible, both good and bad, because news is out that their systems aren’t purely tech. They use people to transcribe, in call centres dotted around the world. This is a revelation to me and kind of damages their core proposition. People on Twitter seem to think so too, as does Rory Cellan-Jones of the BBC who sees SpinVox not so much spinning as unravelling.
Quite apart from potentially being in trouble by having a call centre in Egypt, contrary to their claims of working within the European Economic Area, it implies to me that, far from having systems that scale, they have human beings that do not.
If their solution truly worked entirely with speech recognition then it would be gloriously easily – and a compelling business model – just to plug in server farms and data centres when load grew. But the ultimate corollary of human transcription is that you have half the world calling, and the other half transcribing. It doesn’t compute.
This would account for their other large headache: money. They’ve been asking staff to take share options instead of money, which was probably ok for Apple in 1960s, but times have changed since then. A while ago I heard Christina Domecq on Radio 4’s Bottom Line programme in which she implied the recession was a huge opportunity.I wonder whether she still thinks this?
She also said her systems ‘learned’. From what we now know, I guess this was the truth but maybe not the whole truth.
Machines as humans
Secondly, the first: the search engine socialmention which scans the social media space – blogs, forums, microblogs etc – for your search terms.
After reading about SpinVox I decided to use socialmention to see what people were saying about it. I noticed with interest that socialmention has some metrics I haven’t seen before (admittedly because I haven’t used it in a while). One of them I ‘get’: reach is calculated as the number of unique authors divided by the number of mentions. But the other three – strength, passion and particularly sentiment – I do not.
Strength is ‘phrase mentions within the last 24 hours divided by total possible mentions.’ Total possible mentions? What does this mean? Surely the total possible mentions is virtually infinite?
Passion is ‘the likelihood that people talking about your brand will do so repeatedly.’ This is maybe a bit clearer in that it probably uses frequency of mentions by unique authors. Or something. Again, it’s not particularly clear.
But sentiment is what truly gets me. It talks about ‘generally positive’ and ‘generally negative’ and, being free and openly available, it’s probably doing something similar to Waggener Edstrom’s twendz twitter sentimenting tool which, it seems to me, just uses fairly crude keyword proximity algorithms rather than anything rigorous.
That is, figuring out sentiment, but fairly badly. I used the tool as a test when Jade Goody died. I noticed it would class as ‘negative’ tweets that said “sad that Jade Goody died” – clearly figuring that the proximity of ‘sad’ to ‘Jade Goody’ implied negativity. Wrong.
I’ve done sentimenting myself in the past. I’ve been through search results for clients and figured out whether they’re positive or negative by actually reading them. But I can only do so much, often restricting myself to only a few pages of search results. Machines can do much more – they scale – but can they do it better?
I’ve recently been working a lot with PR measurement, and have had my eyes opened to the crudity of some measures out there. AVE for example, is only good for impressing people. That’s why some PR companies use it to impress their clients, and their clients, in turn, use it to impress their bosses. But it’s total bollocks.
So given the importance of accurate measurement, I would argue that tools like socialmention are actually dangerous. Some people out there might actually be using it to gain insight, and they will be doing so in a wholly unaccountable way. The conversation goes thus: “We’ve found that people are overwhelmingly positive about your brand.” “How do you know that?” “Socialmention says so.” “How does it know that?” “We don’t know.”
They’re not the same (not yet anyway)
On the one hand, perhaps it’s better that SpinVox is using humans because they understand language better than computers, at least for the time being (quite apart from also being naughty by posting their SpinVox grievances on Facebook). On the other, they have some explaining to do because they’ve kind of sort of perhaps maybe possibly led people into believing they were a tech solution, which would imply a much more effective business model if less effective transcription.
Meanwhile, socialmention is an unashamedly tech solution. But it’s claiming to do what humans do, and I just don’t believe that is the case. If they could, SpinVox would be using them, right?