The perils of the distributed approach
About the time of Hurricane Katrina, a lot of people getting excited about the power of the web to respond to disasters (some of which were loosely grouped under the title Recovery2.0). The idea was that the power of the web could be harnessed to improve the speed and coverage of disaster response, creating a multiplier effect with all of those people in the world that wanted to help but didn’t know how.
Needless to say, I wasn’t one of them (grouch, grouch, grouch). That wasn’t because I didn’t think the web had the potential to transform disaster response practices – I think it does have that potential and it’s already transforming our work.
(We had an interesting idea-tumble earlier this year between Jesse Robbins, Mikel Maron myself on a very specific instance around Katrina.)
My reservations were to do with the problem that we often have in responding to disasters – unqualified people showing up to help. In this case, those people are virtual rather than physical – but they still create the same co-ordination problems.
Recently, a guy called Steve Fossett disappeared while flying across the US. I’m not sure whether this made many headlines internationally, but it was quite a big deal in the US. Shortly after the search attempts began, a couple of web-based distributed searches began using satellite imagery, the most notable of which was through Amazon’s Mechanical Turk, closely followed by Google Earth.
Those efforts have now ceased, although neither Fossett or his aircraft have been found. Wired magazine just published an article entitled Online Fossett Searchers Ask, Was It Worth It? which captures some of the difficulties that the Amazon project faced without really going into the underlying problems. While feelings amongst the Turkers appears to be mixed, the opinion amongst the professionals comes through much more clearly:
“The value of the contribution is hard to quantify because ultimately we failed to find Steve, but it seems reasonable to imagine that this could work,” Chantrill says. “I don’t see any downsides to it, so long as people don’t pester the professional search-and-rescue teams with poor leads.”
Yet that is exactly what happened, much to the exasperation of Civil Air Patrol Maj. Cynthia Ryan, who says her e-mail and voicemail boxes were flooded with leads from folks working on the Mechanical Turk. Many times, they mistook search aircraft in the air for Fossett’s plane — even though it’s unlikely Fossett’s plane would have appeared intact.
“The crowdsourcing thing added a level of complexity that we didn’t need, because 99.9999 percent of the people who were doing it didn’t have the faintest idea what they’re looking for,” Ryan says.
“In the early days, it sounded like a good idea,” Ryan continues. “In hindsight, I wish it hadn’t been there, because it didn’t produce a darn thing that was productive except for being a giant black hole for energy, time and resources. There may come a day when this technology is capable of doing what it says it can deliver, but boy, that’s not now.”
The web has a clear value in disseminating information, which can by itself empower them to act. Does it go beyond that, though? Are the tools available to us through the web going to completely change the dynamic of disaster response, engaging more people than ever before, working at a distance rather than in the field.
There are two basic principles behind Disaster2.0 or Recovery2.0 or people-to-people aid or whatever you want to call it. The first is that the internet can cut out or automate the middleman in the way it has in the private sector – getting rid of some of the troublesome and costly human elements that make disaster response difficult. The problem with that is that it also eliminates the implicit knowledge needed to approach the task in a coherent way; knowledge which is only gained with your boots on the ground.
The second principle is that a distributed approach to share out repetitive tasks amongst a wide range of people will make those tasks easier. The problem with that is that if there’s nobody directing those tasks, then you may as well not bother; and if you have unqualified volunteers working on them, they’re far less effective than trained professionals and not accountable in any way for their work.
The spirit of volunteerism is the basis for all humanitarian work. We need to encourage people to engage with these issue, and I welcome these types of initiative as useful learning experiences in how to do that better. We need to remember, however, that managing volunteers has a cost as well, whether they’re physically present or virtual, and unless they have a specialised skill (and believe me, analysing remote sensing data is a very specialised skill) they can cost more than they contribute.
I also want the humanitarian sector to professionalise, to deliver more effectively to people in need, and I worry that this type of approach gives the wrong message, while at the same time raising people’s expectations about how they might be able to contribute to what is, after all, a life and death business. How do we balance our need to engage the public in order to draw on their huge potential without undermining our actual response?
![[del.icio.us]](http://images.del.icio.us/static/img/delicious.small.gif){kind=small}
Perhaps we have only seen a few specific instances, and have yet to seen the full potential of this idea worked out.
The main example you cite here is the search for Steve Fossett. Of course, this was not a disaster (if only all disasters received this level of attention). But the response is in the model of the resources we could expect in an actual disaster response. I’m not entirely sure why people were phoning and emailing — this should have been handled and managed within Mechanical Turk itself, and if people are over eager to report results, then the instructions and volunteer management should be more clear about what’s helpful and what’s not.
If some training is required, then I’d say it’s fair to put in place some virtual training before people are unleashed on the problem (without too many barriers of course). Assuming volunteers were effectively managed, and I think they could be, the question becomes is this pool of volunteers useful to the response? You seem to agree yes. So how can we structure Recovery 2.0 reponses?
The other aspect of the distributed response is requests for aid from people within the emergency. Can they SMS their needs and situation, to help guide responders? Definitely people directly affected (or their family/friends) should be able to communicate their needs .. so how can that be managed? I think this becomes even more vital beyond the initial response phase, and into the recovery phase, when the world’s attention has moved on.
Mikel Maron
9 Nov 07 at 9:28
This is the thinking behind “DisasterCamp”, a series of operational exercises to provide training and operational experience for disaster and humanitarian innovators under realistic conditions.
Jeses Robbins
10 Nov 07 at 3:11
Paul,
In another case, the Katrina effort, there was a similar information impedance mismatch as I would call it. Nearly everybody in the U.S. had somebody they knew in the affected area and very little information was coming out in the first few days. The big question is, while information is available to first responders, how do we make this data available to the rest of the country ? This data has different meaning to different people and it is somehow unfair to the rest of the population to restrict what should be made available. This is one of the reason we looked into a cheap way to do maps with Geocam:
http://sei.tamu.edu/geocam/overview.htm
One of the other ways the mechanical turk effort should be improved is to give feedback to people about what has been done and what hasn’t. What has been confirmed to be bad and what hasn’t. I think one of the reason some people spent so much time and felt eventually helpless is that they didn’t see the result of their effort. If for no other reason, you spent 250 hours on the search you should have learned some type of skill. If you do not have a way to find out if your results have been acted on or declined then you might as well as have done something else because you learned nothing in the process.
Please also note that during the search for Fossett, some planes that had been lost for a long time were found. It is by no means a bad result. Some type of closure has been attained by some families:
http://nuit-blanche.blogspot.com/2007/09/search-and-rescue-new-directions.html
Another item is not mentioned in your analysis: while it is good to gather data and have competent people making sense of it, it is extremely difficult to share this data, not because of lack of capable ears, but rather because there is little effort spent in methods for sharing this data. In the effort for finding Jim Gray, it became obvious that even current sophisticated tools developed by the Coast Guards did not convey the full picture of what had been done and what had not to the people involved in the search.
http://nuit-blanche.blogspot.com/search/label/jim%20gray
Igor.
Igor Carron
10 Nov 07 at 11:30
Mikel – I agree with you that we haven’t seen enough of what the web can do to make a really solid judgment. Like I said, I think it is already transforming things, but I also think that it’s impossible to predict these sorts of changes. You raise two points that I can comment on usefully…
1. Training. The costs of managing training across a large number of people in different locations and organisations is high. The question is, does the cost bring in enough benefits to make it worth doing? I struggle to think of tasks that wouldn’t require fairly substantial training – any thoughts? – but if there are then this might have potential.
2. This interests me more, because it’s more about empowering individuals and communities. The biggest barriers here (for me, at least) are to do with the reporting itself. To some extent we already have this system, in emergency service numbers, so all that this would do is shift that system to another platform for extra redundancy in the immediate response.
In the recovery phase, I don’t think that this would make any difference. In a large scale disaster, nobody is going to provide assistance on the basis of an SMS, phone call, email or letter – any organisation with an ounce of professionalism should provide assistance on the basis of an on-site needs assessment*. Yes, SMS can provide an additional platform for such requests, but would it remove the need for that on-site activity?
I do think there are interesting possibilities for SMS reporting problems in the system – for example, if the trucks don’t turn up on the designated distribution date, the village can SMS the NGO warehouse to find out what’s going on – a delay, an accident, an error. That would put the beneficiaries in a much better position in terms of their right to information – but might overwhelm already harassed warehouse officials!
* but don’t get me started on needs assessment, since this is an area in which our record isn’t particularly good either.
Paul Currion
10 Nov 07 at 13:32
I couldn’t agree more. One of the first lessons I learned in the field was that if the people that you are asking to contribute don’t get feedback quickly, they rapidly stop contributing – and you don’t get a second chance.
Paul Currion
10 Nov 07 at 13:46
Jesse: Would love to hear more about DisasterCamp! I assume you’re familiar with the Strong Angel series of exercises?
Paul Currion
10 Nov 07 at 13:48
Mikel:
And that raises two more questions for me about distributed approaches. Firstly, scalability. This was a relatively simple, focused task, yet we see how difficult it was to manage and co-ordinate those efforts, the relatively weak delivery and the problems with ground truthing the results. Imagine if it was a large scale disaster with multiple factors at play – how well would it work?
Secondly, sustainability. As we’ve discussed, Mikel, there are always problems with keeping peoples’ interest once an event leaves the news cycle. Do we really want to set up systems where there’s no accountability for delivering results if we can’t rely on their consistent performance?
I should stress, I am not saying that distributed approaches have no place in the system; I think that they do, and I think that we’re getting there. I would worry, however, if we made such approaches part of our mission-critical work…
Paul Currion
10 Nov 07 at 13:52
“The problem with that is that if there’s nobody directing those tasks, then you may as well not bother”
Could this be applied to the broader context of a humanitarian response, where there are a large number of different actors, working with little coordination, and certainly no central controller directing the tasks? Are you saying that we shouldn’t bother, just pack up and go home?
Despite this, I still agree with most of what you say. I personally don’t think that a “Web 2.0” approach, utilizing virtual volunteers, has much potential for a humanitarian crisis – I agree that this is an area where professionalism is important (however I think for development, or long term aid it is different story). However I do think that the “2.0” approach could be better utilized to engage the effected population. I think that there is much more potential for utilizing SMS technology for this. To use the example of a village sending an SMS to a NGO warehouse to make an inquiry about a distribution: I don’t think that any organization would be happy about opening direct communication between beneficiaries and their warehouse (I can imagine the warehouse being overwhelmed with direct requests for items). However I think that NGOs should be more open to SMS contact from communities. SMS Gateways and special software (www.forntlinesms.org) make it very easy to import SMSs into a computer, which makes it much easier to process the information. Organizations could have focal points to get the information from the warehouse (or appropriate sector), and reply to the community. Taking this a step further, it wouldn’t be unrealistic to design an automated interactive SMS based information service.
Michael Howden
19 Nov 07 at 3:18
Basically, yes. Unless we can get our act together in future, I can’t see a strong moral or practical reason for most of what currently passes for the humanitarian response to continue. There will always be a need for essential lifesaving activities, but if we as a community claim to be doing something more than that, then we need to either put up or shut up.
This is exactly what I mean by putting up or shutting up. I agree that we’d be deluged by requests from beneficiaries if we did this, but if we don’t do it, we can’t claim to really be “empowering” them. I think we can have one thing or the other, but we can’t have both.
Agreed – I think that this is an area where sms-based communication could play a huge role in the future. You wouldn’t make it the backbone of your logistics, but it would certainly strengthen the usual organisational processes and the links to the community that are so weak at present.
Paul Currion
20 Nov 07 at 18:37