Talking about Paper
A warm welcome to the blogosphere to Robert Kirkpatrick, scion of Groove and Microsoft Humanitarian Systems. With the formalities out of the way, I can tear apart his first post to get at the raw meat inside.
Robert is bang on the money when he says that “In every disaster zone and every rural development environment… paper is still king” and equally on the money in suggesting that this is because paper has a range of natural advantages over digital formats. Despite this he also predicts that “One day, hopefully soon, we won’t need paper in the field”, which seems a strange thing to wish for – the disappearance of a technology which is better than anything else we’ve yet developed.
The usual motor for such wishes is commercial: we’re sold new technologies which aren’t really better than the old ones mainly because somebody wants to make a buck out of it. This approach has a surprisingly successful track record for the obvious reason that people just aren’t that good at rational thinking. We’re more resistant to the sales pitch when the object is right in front of us and appears to work pretty well. Exhibit A: the bicycle. Few people even try to persuade us that they have something better than the bicycle to sell us, because it quickly becomes obvious that they don’t.
Unfortunately in the world of ICT, this natural check often doesn’t apply – it’s not immediately obvious that the novel solution isn’t any better because often the product in question isn’t as tangible as, say, a bicycle. This becomes a real problem when commercial companies approach non-profit organisations, offering them what looks like a free gift. Important to remember: whenever a commercial company offers you its product for free, it’s still a sales pitch.
Having said that, Robert’s offer is more interesting than my skepticism might allow. If “paper is the weakest link in your information supply chain”, then strengthening that paper will strengthen that chain – right? Well, sort of. In my opinion, in the humanitarian sector the weakest link in an information supply chain is more often to be found at the far end – the decision-making end. That’s tangential, because I do agree that there paper is a weak link, if not the weakest, taking us back to Robert’s point:
Data entry is not only a juncture where errors tend to be introduced; it’s also the point that tends to contribute most heavily to latency in the flow of humanitarian information. When critical information needed to match needs to resources reaches decision-makers too late, coordination breaks down, further delays are introduced, resources are misallocated, and too little arrives too late to help a population in need.
Again… sort of. Note to self: future post on how the humanitarian sector may suffer from serious whiplash effects in its supply chain because of the uncertain nature of most of the material requirements, the rapid turnaround required in procurement and the shifting conditions on the ground. This problem means that truly “efficient” supply chain management is not really what’s needed – as Michael says more eloquently, it’s better supply chain visibility that makes for better coordination, since it means that managers can make earlier decisions. WFP, for example, doesn’t order food based on precise headcounts, and nor would more precise headcounts cause WFP to re-think its logistics.
I could be wrong.
As a result of this dynamic, the forms designed to assess population needs at the outset of a response soon become inadequate. Questions must be added. Others must be removed. The schema of the data being collected has changed, impacting form and database design.
Surely this isn’t a technology problem – that it is, it isn’t a problem with the schema, the databases or the processes used to populate those databases? This is a political problem. The basic fields required to respond to a humanitarian crisis are almost invariant no matter what the specifics of that crisis are. The main requirement is to take into account local technologies (e.g. do we need to look for boreholes or surface water supplies), terminologies (e.g. what words do people use to describe their situation) and ontologies (e.g. what are the administrative boundaries).
We can argue about my reservations, but Robert’s post is still based on one major assumption:
As long as paper is used for data collection, error and data loss will continue to reduce the effectiveness of humanitarian coordination, and unless someone invents self-validating paper, it’s hard to see ways that technology can help here anytime soon.
Unless Robert has access to research that I don’t, my personal experience gives me no reason to believe that the levels of error in paper-based data collection are sufficiently high to significantly reduce the effectiveness of humanitarian coordination.
This project sounds interesting – but is the problem it seeks to resolve one that warrants such an investment? Does Robert make the fatal mistake of assuming that certain key processes needed to make it work (for example, people actually cleaning up data in a collaborative workspace) will somehow materialise as soon as the technology is developed? I worry, I worry, I worry…
![[del.icio.us]](http://images.del.icio.us/static/img/delicious.small.gif){kind=small}
Thanks for this, Paul. I’ve been meaning to reply to Robert’s post myself as I think it should be generating more conversation in the blogosphere. In any case, I largely agree with your feedback.
While indirectly related, I thought I’d share this project which seeks to minimize error in data entry using mobile phones:
http://irevolution.wordpress.com/2009/04/22/improving-quality-of-data-collected-by-mobile-phones
I write “indirect” because most of the points you raise still apply to mobile phones. So the latter is no panacea but the project is still interesting and could have a positive impact in some cases.
Patrick
27 Nov 09 at 2:44
Thanks Patrick, and thanks for the link. No project that seeks to improve data collection is a complete waste of time, but I’m not sure that data entry is the weak link in the chain. I’m not even sure that timeliness is the weak link in the chain either – there’s still debate to be had about the advantages and disadvantages of real-time monitoring.
Paul Currion
27 Nov 09 at 16:41
Great critique. I actually think Robert is too soft on the tech solutions .. we’re discovering that paper is actual a better solution in many cases .. witness Walking Papers which was part of the inspiration for the Talking Papers effort.
As you emphasize, transparency is another weak point in this flow, and I’d put forward that it would contribute towards better use of information in decision making due to the increasing ease of accountability. Talking Papers should emphasize transparency and openness in its design as well.
Mikel Maron
29 Nov 09 at 15:18
[...] that new technology isn’t necessarily an improvement on old technology – as per my post on paper. Pick the right technology for the task in hand, and always be questioning our use of technology to [...]
“If it’s not on the slide…” at humanitarian.info
30 Nov 09 at 12:02
Paul,
Thanks for the warm welcome and thoughtful comments!
My optimistic prediction about the day when we have a paperless field is indeed wishful thinking. I hope I do live to see that day. But my prediction was in no way intended to suggest that we are anywhere close to it being ready to leave paper behind. Talking Papers, if it performs as described, will be a clear affirmation that paper has many years, if not decades, of service yet to render to the humanitarian relief and development communities. Given the natural advantages that paper has over digital formats, I don’t believe we’ll see a truly viable successor until we have a cheap, flexible, easy to reproduce “digital paper” technology that combines the best of both worlds. Although I am certainly excited about recent advances in mobile data collection (you may recall that I chair the Open Mobile Consortium), I’ve weathered the same seemingly endless progression of clunky, unsustainable, unscalable, proprietary data collection + map apps running on ruggedized PDAs that you have. Paper has a long life ahead. My hope is that Talking Papers might help to streamline how paper in the field is used to feed centralized information management systems on the back end, where its clear that technology is superior in so many ways to paper-based processes.
I can’t tell from your post whether you were believed that there was a commercial enterprise at work. As far as I know, no one involved in Talking Papers has discussed commercialization of this project. We all believe in open source, open standards, and open innovation. We want this project to come together as a community effort. Although free software is of course like the free goldfish at the pet store – the care and feeding is anything but free – we are all quite keen to make sure that what results is available to everyone who wants it for free.
I also agree with Michael on the need for greater supply chain visibility, and its quite possible that significant progress in that domain would improve the effectiveness of humanitarian operations more dramatically than Talking Papers ever could. That said, I do believe that Talking Papers could go a long way toward improving data collection, and that such improvements would also yield tangible benefits to emergency response.
You state that
“The basic fields required to respond to a humanitarian crisis are almost invariant no matter what the specifics of that crisis are. The main requirement is to take into account local technologies (e.g. do we need to look for boreholes or surface water supplies), terminologies (e.g. what words do people use to describe their situation) and ontologies (e.g. what are the administrative boundaries).”
Well, sort of. It is rather troubling that we as a community still can’t reach agreement on a manageable number of indicators to include in a humanitarian crisis Minimal Essential Data Set. I’m still traumatized by my experience working on the “Rapid” Assessment Form for Iraq back in ’03, which consisted of 338 data elements. MEDSs are a tangent that could take us down a pedantic rabbit hole and we’d best not pursue it here, but I think it’s worth remembering that data is collected during a response to help individual organizations fulfill their various mandates, and those mandates themselves evolve as coordination improves. I’d love to get the point that we could create forms on the fly that include means to collect data for both universally appropriate MEDS indicators and locally-appropriate indicators (it’s actually already slated as the topic of a near-future blog post), but the reality on the ground, as you know, is that that forms change all the time, often after vast numbers of paper copies have been passed out to people who can no longer be reached.
Regardless of whether the root causes of this constant flux are largely attributable to the factors in the political sphere, form designers, database administrators, and data entry staff bear the brunt of it. The self-describing data+schema payload of a Talking Papers form is intended, in part, to support scenarios where wave after wave of evolving datasets have to be compiled into a repository that has no prior knowledge of updates to schema. That is, Talking Papers can’t help prevent the flux. It might help field personnel accommodate it in some useful ways.
“Unless Robert has access to research that I don’t, my personal experience gives me no reason to believe that the levels of error in paper-based data collection are sufficiently high to significantly reduce the effectiveness of humanitarian coordination.”
My assertion was simply based on the recognition that 1) the inevitable misallocation of resources during emergencies is (in my personal experience) sometimes the result of misinformation, and that 2) technologies such as PDAs can perform client-side validation and catch errors at the point of capture in ways not possible with paper. I don’t have any hard numbers here on error rates (or lost forms) and its operational impact. That would be useful research indeed. My point was to suggest that there might be another step in the process – data entry – where a system like Talking Papers could help to improve another aspect of information flow: timeliness.
Finally, I’m not assuming that a staff to perform data scrubbing will simply materialize as the result of the availability of the data scrubbing tool within Talking Papers. Field staff and HQ staff are already scrubbing humanitarian data in spreadsheets and Access databases that was collected from paper forms. I’d like to create a tool that makes better use of their time by detection and visual flagging of potential areas in need of scrubbing and collaboration features for those who have access to the Internet (a luxury far more likely to be available at HQ). I’ve personally observed a humanitarian IM anti-pattern that I’ve selfishly named “Kirkpatrick’s Conjecture”, which is that the probability of a relief worker’s ending up manually scrubbing data all night in the corner of a tent is in direct proportion to their seniority within the response. Getting the data right before passing it up the chain matters so much that those who are already burdened with responsibility often don’t feel comfortable delegating the task to anyone else. Fixing that problem is going to be tough, but in the mean time, I’d love to put better data scrubbing tools in their hands to help them get at least a couple of hours of sleep.
There’s some wonderful work on the project already underway here. Please join the group and share your concerns, ideas for features, thoughts on prioritization, and any potential opportunities to test it in the field.
The Google group is here:
http://groups.google.com/group/talkingpapers
Mike Migurski’s Github repository is here:
http://www.talking-papers.org
Robert Kirkpatrick
4 Dec 09 at 2:17