Compare and contrast
Alex Evans at Global Dashboard:
The point about Twitter and other social networking technologies is that in our hyper-networked age, we just haven’t yet had the time to develop the collective mechanisms to make sure that this awesome power to aggregate, to build positive feedback loops, is channelled safely.
Erik Hersman at The Ushahidi Blog:
Since we don’t believe there will ever be one tool that everyone uses for gathering information on global crisis, we see a future where a tool like Swift River aggregates data from tools such as the aforementioned Twitter, Ushahidi, Flickr, YouTube, local mobile and web social networks. At this point what you have is a whole lot of noise and very little signal as to what the value is of the data you’re seeing.
Anyone who has access to a computer (and possibly just a mobile phone in the future), can then go and rate information as it comes in. This is classic “crowdsourcing”, where the more people you have weighing in on any specific data point raises the probability of the finding the right answer. The information with greater veracity is highlighted and bubbles to the top, weighted also by proximity, severity and category of the incident.
The question is, how viable is a tool like Swift River as one of the “collective mechanisms” that Alex correctly identifies the need for? I think the Ushahidi developers are on an interesting track, but I think that there are limitationss to what crowdsourcing can achieve – not a problem when it’s a forum like Digg (for example) where the weight of numbers has a levelling effect on any individual distortions, and where the ratings are trivial.1 However I’m still waiting to be convinced about the value of crowdsourcing in an emergency, because it’s crowdsourcing of the type which Alex describes in his example of a potentially dangerous Twitter meme.
![[del.icio.us]](http://images.del.icio.us/static/img/delicious.small.gif){kind=small}
- I don’t mean “trivial” in a derogatory way, only in the sense that nobody’s going to make potentially life or death decisions on the basis of a Dugg article. [↩]
When the crowdsource behemoths at Amazon or Lovefilm finally recommend to me a book or movie that meets my tastes, I’ll start trusting crowdsourced recommendations a bit more!
Perhaps there’s a very useful place in something like SwiftRiver for “old” sources of information and authority to contribute to rating or verification of the crowdsourced river. This is part of the Ushahidi model anyhow: the concept that there are trusted entities (like NGOs) with sufficient knowledge, resource and credibility to make judgements about information.
Tom Longley
6 Feb 09 at 8:19
[...] Currion once again cuts to the chase over at humanitarian.info. No TweetBacks yet. (Be the first to Tweet this post) Share and [...]
A few quick links… | Aid Worker Daily
11 Feb 09 at 0:18
Hey Paul. I think it’s a mixture, but that there needs to be a human element. I had a really good discussion with Sean Gourley on this, where he operates on the side of pure algorithms for figuring out the “mathematics of war” (an incomplete primer for which can be found here). The question arose as to whether we were talking about the opposite sides of the same coin when gathering data and filtering for veracity.
Neither way can be completely accurate, and neither way will be completely accurate when combined either. However, my hypothesis at this point is that one (or both) can lead to having a greater probability of telling “better” information from “worse” as we deal with a larger and larger firehose of incoming data.
It’s a beginning, and we’ll likely learn much along the way that proves many of our presuppositions wrong. Such is the beauty of trying new things, many times you end in failure, but with continued iteration, something good can/will finally come of it – even if it’s not from us.
Erik Hersman
11 Feb 09 at 2:27
the question is not whether a tool like swiftriver will capture everything (it won’t). Rather the question is instead will swiftriver be a useful tool for aggregating, sorting and presenting information in crisis and conflict situations. And it is the second point that is an easier question to answer.
1) We know now that there exists a lot of information out there on the web in the form of images from flickr, streaming updates from twitter, txt messages from Ushahidi and the traditional news from the various bureaus.
2) we believe that there exists valuable information within this stream of events and that we can extract the signal from the noise.
3) we believe that the current state of the art sorting algorithms and filters can be combined with human intelligence to create a tool that probabilistically assigns a likelihood to an event.
The science is not at the point where we know for sure whether points (2) and (3) will turn out to be true. The key problem is how to assign weight to an individual story, how to combine the various available metrics to determine reliability of the source. This can be a combination of crowdsourcing, algorithms, geographic details, past history of user, repeatability of the information. Not an easy set of variables to combine together and indeed a difficult problem, but one that if solved could be a very important tool. As with anything of this nature it is trying to determine where the bar of certainty lies – too low and you get noise, too high and you miss out on information.
As to the final point about the twitter meme – i.e. “someone might abuse the tool” – this is always a possibility. As engineers you create tools and they have the power to do good and do evil. What has to be controlled is the algorithms and community structure to ensure the probability of false events becoming ‘real’ is kept to a minimum.
sean gourley
11 Feb 09 at 4:07
Sean – great input, thanks. I suppose my question is not whether swiftriver will be able to aggregate, etc – I am sure that it will. My question is much more, if it’s a tool then what will it be useful for, and who will it be useful for? I realise that this isn’t the province of scientific enquiry per se, but if it’s more of an engineering question (i.e. building a tool, not just identifying an algorithm) then it is very relevant.
Erik – the firehose is the thing, isn’t it? From an operational point of view, I feel more like we suffer from data droughts rather than information floods (paper available here, now out of date), but once again it depends what you need the information for… information for informations sake is not our goal!
Paul Currion
18 Feb 09 at 22:49
The flipside of the effective crowdsourcing meme is something that fascinates me. I had a very positive experience of crowdsourcing in a non-emergency situation: http://is.gd/i0wy
And a more challenging one yesterday, in a locally-small, though genuine ‘emergency’ situation. Which led me to these thoughts http://is.gd/BGkw about the behavioural aspects that need to be instilled if we are to get the best out of memes, and avoid some of the time-wasting they can generate.
Paul Clarke
20 May 09 at 18:56