On Crowdsourcing, with a big sigh
Some people working on information management in Haiti have concerns about the role of crowdsourcing in humanitarian response, which is something I’ve written about previously. If you don’t want to read that whole blog post, the short version is this: I don’t think that crowdsourcing and humanitarian response are a good fit, for a number of reasons, some of which I will explain below.
Unfortunately Patrick Meier’s efforts to defend crowdsourcing are more likely to persuade people that crowdsourcing is yet another technology project that is going to eat their time and fail to deliver. It may be worth treating the views of other professionals – communicated in what appear to be private emails – with a little bit more respect if you want to persuade them of your case.
It would be fair to say that I disagree with pretty much everything that Patrick has written, so there’s little point in trying a point-by-point rebuttal. However many people – including Patrick – appear to mistake the extensive media coverage of Ushahidi (and therefore “crowdsourcing”) for meaningful impact1 and this needs to be corrected. Since Patrick has set the ball rolling, I’ll use his arguments as my starting point.
Not sure how you’d interpret these words but what they say to me is this: unless information comes from official field personnel, i.e., Muggles, it’s absolutely useless and should be dumped in the trash. I personally find that somewhat… is colonial too provocative?
Not provocative, but deliberately misrepresentative. In the literal sense humanitarian agencies aren’t colonial – check a dictionary if you don’t believe me – but “colonial” (or rather “imperial”) is an easy term to throw around if you want to disparage somebody. It’s a bit like calling somebody a fascist – unless they are actually a fascist, it’s just lazy grandstanding. In this instance it feeds into Patrick’s conception of the aid sector as elitist, which we’ll come to later.
And of course the way in which Patrick interprets those words bears little relation to what those words actually said, which is this: “Unless there are field personnel providing “ground truth” data, consumers will never have reliable information upon which to build decision support products.” This is trivially true, even for crowdsourcing initiatives – why do you think Wikipedia requires all its facts to be externally referenced? It’s their equivalent of ground truthing.
In the actual quote that Patrick supplies us with, his humanitarian critics are quite clearly not saying that crowdsourced data is absolutely useless, only that verifying that data is essential. The really strange thing is that Patrick agrees with this, since later he points out that this verification is “the whole point behind Swift River, to provide a free and open source platform that can help validate large quantities of information in near real time.”
So Patrick does think that verification is important, which leads us to the next strange argument that he presents, one which he’s presented before:
Crisis information that was crowdsourced using the distributed short code 4636 in Haiti helped save hundreds of lives according to the Marine Corps.
Given Patrick’s agreement on the importance of validating data, there seems to have been very little validation of this claim. The claim itself first appeared in this blog post and was expanded on in this blog post. Read both those blog posts first, especially the first one, and especially the correspondence from an unnamed military source, who makes two claims:
- That Ushahidi saved the lives of three women who were evacuated. The problem with this claim is that those women do not appear to have been in the Ushahidi data – the expeditionary unit happened to be in Grand Goave verifying more general data2 when they encountered them by chance. From my perspective, this is an extremely weak claim to success: given the situation in Port-au-Prince in the last week of January, the unit could have been anywhere in the city and would probably have found somebody who required medical evacuation.
- That there are “100s of these kind of stories” of Ushahidi having an impact. Only one such story is described by this single unnamed source, the one referenced above, and if we assume that this is the strongest example, then the case looks extremely weak indeed. However the lack of further cases makes it impossible to draw a reasonable conclusion about this claim – but that in itself is a problem, since it’s impossible for anybody to say one way or another whether Ushahidi has any impact at all.
It’s worth emphasizing the second point, because it’s true for (as far as I know) all Ushahidi deployments so far: there is no evidence at all for their impact. On the first blog post referenced above, there is a list demonstrating how the feedback loop was closed – but if you read the detail, Ushahidi was not instrumental in the actual delivery of aid, merely in the reporting of aid. Contrary to Patrick’s claims, this is not “disrupting” the humanitarian system.3
The same goes for claims about Mission 4636, which I agree was as extraordinary as the Ushahidi effort from the supply side but for which there is little evidence of success from the demand side. I look forward to seeing the results of the impact assessment being carried out4, but it would be useful to see the TORs, because as anybody in M&E knows, the scope of an evaluation determines the findings of that evaluation.
A forthcoming USIP report that reviews the deployment of the Ushahidi platform found that Haitian NGO’s and local civil society groups were physically barred from entering LogBase—the humanitarian community’s compound near the airport in Port-au-Prince. One Haitian NGO rep who was interviewed said he felt like a foreigner in his own country when he wasn’t allowed to enter LogBase and attend meetings where he could share vital information on urgent needs.
I’m sorry that a Haitian NGO rep felt like a foreigner in his own country but I’m pretty sure that even before the earthquake there were many places in Haiti where he couldn’t just walk in whenever he wanted to. In the UK, I can’t just walk into Oxfam’s offices on a good day, and if an earthquake had just destroyed half of Oxford I imagine that they’d be a bit too busy to have a meeting with everybody who walked in off the street. Simply put: no time to deal and filters apply.5
Now tell me, how is trashing Haitian text messages any different than physically excluding Haitians from having a voice at LogBase? Because the so-called “unwashed masses” don’t have the “right” credentials as defined by the Muggles? Either way, they are excluded from having a stake in the hierarchical system that is supposed help them.
There are two responses to this. First, nobody in his correspondence referred to Haitians as “unwashed masses” – those are words which he’s putting into their mouths. This is offensive, since those entrance policies are based on security rather than snobbery and, while I may disagree with those policies on occasion, they’re in place because people can die if they’re not.
Second, nobody in the correspondence appears to be “trashing” Haitian text messages, and it’s inaccurate to say that “Haitians” were physically excluded from having a voice at LogBase, since Haitians were in fact present at LogBase – they just weren’t the Haitians that Patrick wanted to have a voice. At this stage it’s difficult to explain Patrick’s persistent misrepresentation of his opponents.
Writing that “crowdsourcing is a technology” reveals how out of touch Muggles are. Crowdsourcing is a methodology, not a technology. Worse, to write that crowdsourcing should be used to disseminate information shows just how much confusion exists in the humanitarian space.
I don’t doubt that there is much confusion in the humanitarian space. However, it’s fair to assume that for most if not all aid workers in Haiti crowdsourcing was a novel idea and Ushahidi was their first exposure to it. So any confusion regarding what constitutes crowdsourcing must rest largely with those people who were introducing crowdsourcing – and surely that means the Ushahidi team?
Confession: I shudder when reading language like “according to recognized/accepted standards.” Not because standards are not important, but just because I’m weary of the exclusive and at times elitist attitude that tends to come with this language.
I said I’d return to Patrick’s accusations of elitism, but he presents no actual examples of elitism, unless having a discussion via email (on which Patrick was copied) is somehow elitist. What worries me is that the accusation of elitism seems to be cover for an argument that aid work shouldn’t be a field of professional endeavour6 but that anybody can try their hand at. Needless to say, this is misguided.
Despite what some Muggles may think, crowdsourcing is not actually magic. It’s just a methodology like any other, with advantages and disadvantages.
That’s exactly what “Muggles” think. If they’re like me, they think that the disadvantages may outweigh the advantages, and that crowdsourcing and disaster response are not a good fit. Even the fiercest advocates of crowdsourcing don’t claim that crowdsourcing is good for everything, and I think that humanitarian response is one of those things that it isn’t good for. There are a couple of reasons for this, but the main one is this:
Crowdsourcing should not form part of our disaster response plans because there are no guarantees that a crowd is going to show up. Crowdsourcing is no different from any other form of volunteer effort, and the reason why we have professional aid workers now is because, while volunteers are important, you can’t afford to make them the backbone of the operation. The technology is there and the support is welcome, but this is not the future of aid work.
I’m sorry to put it in these terms but if we listened to (and waited for) Muggles all the time, then perhaps several hundred more people would have needlessly lost their lives in Haiti.
Stay classy.
![[del.icio.us]](http://images.del.icio.us/static/img/delicious.small.gif){kind=small}
- Which explains why so many media types have been contacting me about it recently. [↩]
- It appears that the military also believe that Ushahidi data needs to be ground truthed by field personnel, but for some reason Patrick doesn’t attack them for this in the way that he attacks civilians – possibly because the military gave them all medals. [↩]
- Ushahidi is exciting for people working in HQ rather than people on the ground because internal reporting mechanisms are so thin – but this is a one-way street since information from HQ seldom goes back down the chain in any organisation. [↩]
- Apparently “by a team of three accomplished experts” – one wonders why they didn’t crowdsource the evaluation? [↩]
- Having said that, this is a perennial problem for which there is no good answer. We need more accessibility for local organisations, but not at the expense of security and effectivenesss. The solution is to create more neutral spaces and honest brokers who can mediate between the international and the national, and that’s where coordination bodies should play a critical role, but often fail to. [↩]
- This would not be surprising, since this argument underlies a lot of the discourse about crowdsourcing in general – as per the title of Andrew Keen’s book, the Cult of the Amateur. [↩]
Dude. I could almost kiss you right on the lips for this post. Don’t worry – I won’t. But am adding you to my blogroll.
I confess that I am rather weary of the accusations of “elitism” coming at the aid industry of late. It’s not elitist, of course, although the plain and uber-unsexy fact of the matter is the not everyone can do aid work.
While I was in Haiti, attending meetings on the UN logs base with some regularity, I very frequently interacted with Haitian people representing local NGOs and CBOs. Somehow they’d managed to get in. I can’t remember meeting Patrick there, though….
J.
30 Jul 10 at 16:53
Maybe it was crowdsorcerors they were physically excluding, rather than Haitians?
Paul Currion
31 Jul 10 at 12:00
well put Sir!
Craig
31 Jul 10 at 12:49
[...] On Crowdsourcing, with a big sigh – humanitarian.info – Takes a critical look at Patrick Meier’s defense of crowdsourcing the Haiti earthquake relief. [...]
Good Intentions Are Not Enough » Blog Archive » Interesting Articles and Posts – Late July
31 Jul 10 at 17:22
[...] aid – this time humanitarian response in specific – seem to be polarizing slowly into pro and anti-crowdsourcing camps. I linked you to the calmest posts there. There are positions staked out all the [...]
Blood and Milk » Blog Archive » Change Hurts
31 Jul 10 at 19:40
Double, double toil and trouble; SMS’s churn in disaster bubble. With apologies to the Bard.
Sanjana Hattotuwa
1 Aug 10 at 9:52
You may have missed my comment on Patrick’s post. The Mayor of Cite Soleil was prevented from entering the Log Base in March, until some white folks intervened. The Inter-Agency Standing Committee’s six-month review of the relief effort acknowledged that access to the Log Base is a problem, impeding totally legitimate and Haitian groups and individuals from making important contributions to decision-making. It’s not a made up problem. I have witnessed it myself several times at the gate of the base.
I don’t see how Ushahidi is very different from 911 in the States. The government here was incapacitated and other actors led the humanitarian response. Ushahidi is a feedback flow from the people on the ground to the people with access to humanitarian resources. When 911 receives a call, does there have to be a known professional at the site of the call to verify that the data is accurate for action to be taken? No. There’s generally a presumption, if the information is plausible, that might be true and merits investigation/action. That should be the case in Haiti and too often, it isn’t. I know of multiple camp committee leaders who have tried to reach the UN and NGOs to convey their needs but are ignored. But when I contacted the same individuals for them, suddenly we had their attention. Ushahidi may not be perfect. But I think it’s an creative, highly necessary attempt to bridge these gaps and should be taken seriously.
ansel
1 Aug 10 at 23:15
[...] could just about kiss Paul Currion right on the lips for this post on crowdsourcing (don’t worry, I won’t try it). I mean, on one hand I see the huge amounts of potential in [...]
Elitist « Tales From the Hood
2 Aug 10 at 2:56
There are several ideas being discussed here, including:
1 – is data sent in by self-selected crowds (ie crowdsourcing) useful?
2 – does the implementation of certain tech platforms like Ushahidi increase the participation of local people and groups?
–
For 1, the answer has to be “sometimes”: as indicated in the 911 example, sometimes any report is a valid report — (even though it might be inaccurate or false). A report that “I am trapped in rubble” in the aftermath of an earthquake needs to be checked out.
At the same time, there are tremendous limitations in data from self-selected crowds: they are unlikely to be representative. Even in the 911 example, anyone who is trapped in rubble but unconscious or without a cell phone will be excluded, although those are theoretically some of the people you’d want to *prioritize*. Hence the need for other sources of data including search and rescue, etc.
Also, while crowdsourcing can tell you “x number of people are texting us that they need something” it doesn’t tell you the “out of y number of total people in the area”. You can’t determine “25% of the population is without clean water” or “30% of the children are malnourished, as compared to 42% one month ago” by crowdsourcing: for that you need some kind of statistically representative survey (yes, done by professionals who understand public health, sanitation, and statistics).
–
For 2, the problem is that right now Ushahidi has to be implemented by those same elites that are being criticized: in order to make Ushahidi work you need to, for example, know how to “configure a server” (not to mention that you must *own* a server). That automatically excludes many of the less-resourced organizations (who don’t tend to have in-house server-configuration expertise), which usually means you need a bunch of expensive foreign experts to set things up for you.
That means even in situations for which crowdsourced data IS appropriate you MUST have technical people, usually foreign, supporting you. And that means those technical people, usually foreign, get to choose which groups to support, and which not to support.
If Ushahidi can reduce this tech burden, and make an Ushahidi implementation as easy as using Google Maps, then the use of the tool will be democratized beyond well-funded foreign groups to *everyone* who needs to use it (including the major of Cité Soleil). That is totally possible technologically, and Ory Okolloh of Ushahidi has told me that they are laudably trying to move in exactly this direction — really good news for any fan of mapping in disasters or other circumstances!
We’ve got a great deal of similar experience in “tech democratization” with our EpiSurveyor mobile phone data collection software (www.episurveyor.org): it used to require a team from DataDyne to set up the system but now anyone with access to a browser can set it up and start collecting data with phones. As a result, in one year we went from 10 countries using the software to more than 2200 users in 130+ countries . . . 99% of whom never even need any support. No expats, no programmers, no specialists required.
Imagine a world in which really useful tech tools like Ushahidi are as easy to access, set up, and use as Google Maps or Flickr or Facebook. That won’t overcome the important “sampling” issues around the validity of crowdsourced data, but it will put those great tools into the hands of many more users who need them, the people trying to solve problems, the people Easterly calls “the searchers” — whether those searchers work for a tiny local NGO or for USAID.
Joel Selanikio
2 Aug 10 at 15:24
Hi Ansel – I did see your comment on Patrick’s post. I know that it’s not a made-up problem, which is why I specifically said in my original blog post that “this is a perennial problem for which there is no good answer” and briefly described what my preferred solution is. Quite literally everybody who works in relief work is aware of this problem (and nearly everybody has an issue with it), so this is not an amazing revelation.
The IASC’s review will make precisely no difference to the next response, where exactly the same problems will occur. Why is it so slow to change? Two reasons. First, it’s nothing to do with Haiti or Haitians, it’s another systemic problem, and not amenable to quick fixes. Second, there are entirely acceptable reasons why the Mayor might have been prevented from entering Log Base – but I wasn’t there so I can’t comment on that.
Do I like this situation? No. Have I worked to change it in my deployments? Yes. Not sure where you want to go with this, because my point was that this is an extremely poor analogy to receiving a text message.
I don’t see how Ushahidi is very different from 911 in the States.
Maybe you’re focusing on the act of physically making an emergency call rather than the process that lies behind the response service itself. They are different in many, many ways, but even if they were identical that would be an argument against rather than for Ushahidi. This is something that many people don’t seem to understand: humanitarian operations are not like the emergency services.
Ushahidi is a feedback flow from the people on the ground to the people with access to humanitarian resources. When 911 receives a call, does there have to be a known professional at the site of the call to verify that the data is accurate for action to be taken? No. There’s generally a presumption, if the information is plausible, that might be true and merits investigation/action. That should be the case in Haiti and too often, it isn’t.
I don’t accept your premise that Ushahidi is like 911 in the US, and I also don’t accept your conclusion that this is how humanitarian response should work. This is how search and rescue works, but I think international SAR is generally a waste of money, so that’s not a very strong case. You can’t programme humanitarian operations on a case-by-case basis – you have to deal with the needs of communities rather than individuals.
Ushahidi may not be perfect. But I think it’s an creative, highly necessary attempt to bridge these gaps and should be taken seriously.
In this situation – and I should emphasise that I’m not dismissing Ushahidi as software project more broadly, only specific claims about the specific application of crowdsourcing in this specific context – I don’t think a piece of software is able to make any significant impact on what is fundamentally a question of power. I also worry that the ideological baggage that crowdsourcing brings with it is potentially detrimental to meeting the needs of affected communities.
Paul Currion
2 Aug 10 at 17:18
Thanks for the reply.
You said in the post, “it’s inaccurate to say that “Haitians” were physically excluded from having a voice at LogBase, since Haitians were in fact present at LogBase – they just weren’t the Haitians that Patrick wanted to have a voice.”
Patrick’s point about the Log Base is a good one and you claimed it was wrong in your post. That’s what I responded to. Mighty curious about what legitimate reason there might be for preventing the Mayor of Cite Soleil from entering the base until white folks intervene.
“Maybe you’re focusing on the act of physically making an emergency call rather than the process that lies behind the response service itself.”
I agree that the process that lies behind the response service is generally lacking. And I’m saying that agencies should more closely partner with Ushahidi to address that, rather than attack the project as unreliable. Clearly it has uses beyond SAR. I mentioned in the comment on Patrick’s post that Ushahidi was effective in March at alerting others to a forced eviction of an IDP camp.
ansel
2 Aug 10 at 20:03
Patrick’s point about the Log Base is a good one and you claimed it was wrong in your post.
I’m struggling here, mate – what point do you think is a good one?
Mighty curious about what legitimate reason there might be for preventing the Mayor of Cite Soleil from entering the base until white folks intervene.
You really don’t see why there might be security procedures in place at the entry points to an international military installation in the aftermath of a major humanitarian disaster?
I agree that the process that lies behind the response service is generally lacking.
That was not the point that I was making (although it is also the case). My point was that a major international humanitarian operation is not like calling to ask the police to come round to your house because you’ve been burgled.
And I’m saying that agencies should more closely partner with Ushahidi to address that, rather than attack the project as unreliable.
Until somebody demonstrates how partnering with Ushahidi will actually address that problem, then I have no reason to accept your assertion.
I mentioned in the comment on Patrick’s post that Ushahidi was effective in March at alerting others to a forced eviction of an IDP camp.
That’s great, but Patrick’s claim is that Ushahidi saved hundreds of lives, not that Ushahidi caused a journalist to blog.
Paul Currion
3 Aug 10 at 13:32
Try being less condescending, you just might be more persuasive.
Here’s the article resulting from the Ushahidi alert:
http://www.ipsnews.net/news.asp?idnews=50606
ansel
4 Aug 10 at 5:22
I fail to see how I was being condescending – perhaps you could enlighten me? I was merely pointing out that the success that Patrick has claimed for Ushahidi does not fit with the available evidence, which you have just confirmed with your story. Also: try addressing the points that I’ve raised, you just might be more persuasive.
Paul Currion
4 Aug 10 at 7:59
[...] 4, 2010 · Leave a Comment Sigh indeed. Yawn, [...]
Here Come the Crowd-Sorcerers: “No We Can’t, No We Won’t” says Muggle Master « iRevolution
4 Aug 10 at 13:30
[...] has replied to my initial post, but unfortunately has decided to continue his hilarious Harry Potter analogy. So he depicts me as [...]
The Sorceror’s Apprentice at humanitarian.info
4 Aug 10 at 18:16
[...] crowdsourcing and humanitarian aid between Patrick Meier (here, here and here) and Paul Currion (here and here). Unfortunately I don’t have a audio recording to post of the conversation where [...]
Social memory and ICT4D collaboration « Wait… What?
29 Aug 10 at 20:53