Correcting crowdsourcing in a crisis

I hope that Patrick doesn’t feel that this is a hit post, because it isn’t intended to be. However I find it difficult to stay in my seat when I see the words “crowdsourcing” and “humanitarian” in the same sentence. So deep breaths, everybody. Patrick has just put up two posts, entitled Internews, Ushahidi and communication in crisis and Ushahidi: From Croudsourcing to Crowdfeeding. The posts were based on his attendance at a couple of events hosted in New York by InterNews (good organisation, by the way, we like them).

At the meetings, Patrick “introduced the concept of crowdsourcing crisis information” and says that he “should have expected the immediate push back on the issue of data validation”. These two posts taken together explain his position very well, but also explain why I am so very very unconvinced by that position – note that this doesn’t mean that I’m supportive of the status quo, which I’m not. More after the break:

Nobody noted the importance of timely information during crises, which is rather ironic since both meetings focused on sudden onset emergencies. I suspect that our demand (and partial Western obsession) for fully accurate information has clouded some of our thinking on this issue.

Both accuracy and timeliness are core Principles of Humanitarian Information management established at the 2002 Symposium on Best Practices in Humanitarian Information Exchange and reiterated at the 2007 Global Symposium +5. Have those principles been incorporated into the institutions sufficiently? Short answer, no. Is accuracy privileged at the expense of timeliness? Not in the field.

Field-based organizations frequently make decisions on coordination, humanitarian relief and logistics without complete and fully accurate, real-time information, especially right after a crisis strikes. So why is this same community holding crowdsourcing to a higher standard?

Because crowdsourcing is unfamiliar, it’s untested in the field and it makes fairly large claims that are not well backed by substantial evidence. Having said that, I’m willing to be corrected on this criticism, but I think it’s fair to say that the humanitarian community is legitimately cautious in introducing new concepts when lives are at stake.

I would also note that I don’t think the community is necessarily holding crowdsourcing to a higher standard, but exactly the same standard as our usual information systems – and if they haven’t managed to get those systems right yet, I can understand still further why they’re cautious about entertaining an entirely new and untested approach.

The figure below represents the perishable nature of crisis information. Data validation makes sense during time-periods A and B. Continuing to carry out data validation beyond time B may be beneficial to us, but hardly to crisis affected communities. We may very well have the luxury of time. Not so for at-risk communities.

This is where I get hazy – what “crisis information” are we talking about here? I would argue that ensuring that your data is valid is important at all times, so is this an attack on dissemination strategies rather than data validation? It also strikes me as odd that Patrick argues that affected communities deserve timely information but not necessarily accurate information. As he notes, it may be a trade-off – but he provides no argument for why he privileges timeliness over accuracy.

Ushahidi was created within days of the Kenyan elections in 2007 because both the government and national media were seriously under-reporting widespread human rights violations.

Does this mean that the government and national media have an obligation to report on absolutely every single violation of human rights taking place in their country? I’ll be honest, that’s not something I’d find either useful or interesting, and I’m not sure how it’s relevant to the humanitarian community. It might be relevant to the human rights community – but then you run into the validation question again, hard – and it still wouldn’t remove the need to validate that data.

This left the Ushahidi team with few options, which is why they decided to develop a transparent platform that would allow Kenyans to report directly, thereby circumventing the government, media and NGOs, who were working against transparency.

I’m not sure it’s a constructive approach to accuse NGOs of actively “working against transparency” – it strikes me that there may be some shades of grey in their attitudes towards releasing information about human rights abuses.

Here’s a question: why didn’t the human rights or humanitarian community set up a platform like Ushahidi? Why were a few tech-savvy Kenyans without a humanitarian background able to set up and deploy the platform within a week and not the humanitarian community?

I think it might be because the human rights and humanitarian communities were working on their existing projects. The argument that these organisations failed to fulfill an objective when they never actually had that objective in the first place is distinctly shakey – it seems to translate into a protest that they weren’t doing what you wanted them to do.

Having worked in humanitarian settings, we all know that accuracy is more often luxury than reality, particularly right after a crisis strikes… Rather, we need to start thinking in terms of likelihood, i.e., how likely is this piece of information to be accurate?

This, I can completely agree with; in fact I’ve been advocating for since 2000, with very little luck. The military does it much better than us, although I’m pleased to report that they’re nowhere near as good at it as we think they are (and they like us to think they are). However I would qualify that – perhaps we also have to start thinking in terms of what damage might this information do (whether true or false) if we release it.

The purpose of today’s meetings was for us to think more concretely about communication in crises from the perspective of at-risk communities. Yet, as soon as I mentioned crowdsourcing the discussion became about our own demand for fully accurate information with no concerns raised about the importance of timely information for crisis-affected communities.

I think we should consider that resistance to crowdsourcing may not be well-articulated but it is well-founded. The objections raised by the participants at the InterNews meeting included “scenarios whereby militias in the DRC would submit false reports to Ushahidi in order to scare villagers (who would receive the alert by SMS) and have them flee in their direction where they would ambush them.” Patrick thinks that their concerns are wrong, and lists the reasons why:

First of all, militias do not need Ushahidi to scare or ambush at-risk communities. In fact, using a platform like Ushahidi would be tactically inefficient and would require more coordinating on their part.

Well, they don’t need Ushahidi to achieve this, but evidence suggests that armed groups are perfectly happy to use whatever means they can acquire to achieve their goals. I fail to see why Ushahidi would be “tactically inefficient, and would require more co-ordinating” – all they need to do is send a few text messages. The entire point of the platform is that it’s easy to use, isn’t it?

The government of Sudan regularly accuses NGOs (well, those NGOs it hasn’t expelled) of misreporting human rights violations. What better tool would the government have for discrediting human rights monitoring than Ushahidi? All it would take would be a few texts a day with false but credible reports, and the government can dismiss the entire system, either by keeping their own involvement covert and claiming that the system is actually being abused, or by revealing their involvement and claiming that the system can be so easily gamed that it isn’t credible.

Second, local communities are rarely dependent on a single source of information. They have their own trusted social and kinship networks, which they can draw on to validate information. There are local community radios and some of these allow listeners to call in or text in with information and/or questions. Ushahidi doesn’t exist in an information vacuum. We need to understand information communication as an ecosystem.

That’s very true, but it’s not an argument for crowdsourcing.

Third, Ushahidi makes it clear that the information is crowdsourced and hence not automatically validated. Beneficiaries are not dumb; they can perfectly well understand that SMS alerts are simply alerts and not confirmed reports.

The idea that people pay attention to disclaimers makes me laugh out loud. I don’t think anybody’s accusing affected individuals of being dumb, but I’d be interested to see evidence that supports this claim. When does the validation take place, incidentally? And what recourse do individuals or communities have if an alert turns out
to be false?

Fourth, many of the humanitarian organizations participating in today’s meetings work directly with beneficiaries in conflict zones. Shouldn’t they take an interest in the crowdsourced information and take advantage of being in the field to validate said information?

Yes, because they don’t have enough to do. They’d like to spend their time running around validating other people’s reports, endangering their lives and alienating the government under which they’re working.

Fifth, all the humanitarian organizations present during today’s meetings embraced the need for two-way, community-generated information and social media. Yet these same organizations fold there arms and revert to a one-way communication mindset when the issue of crowdsourcing comes up. They forget that they too can generate information in response to rumors and thus counter-act misinformation as soon as it spreads. If the US Embassy can do this in Madagascar using Twitter, why can’t humanitarian organizations do the equivalent?

Is there any evidence at all that the US Embassy’s Twitter feed had any impact at all on the course of events? I mean, I know it made a good headline in external media, but I don’t see how it’s a good example if there’s no actual evidence that it had any impact.

Sixth… Ushahidi deployments are localized and the crowdsourcing is limited to trusted members of those organizations, or journalists in the case of Al-Jazeera.

That doesn’t sound like crowdsourcing – that sounds like situation reporting. So what are we actually discussing here?

Related posts:

1. Compare and contrast
2. Quickbits November 2008 UPDATE
3. The Innovation Fallacy, Part 1
4. Lights! Camera! Discussion!
5. Violence 2.0: some lessons from Ushahidi