humanitarian.info

Duncan Green’s post on NGO advocacy makes a case that well-designed advocacy gets results – in this case, reform of the Social Subsidies Agency in Georgia. Yet the case study highlights a difficult question about advocacy in general – how can you tell if your advocacy has had any results, and how can you tell if the results of your advocacy have had the impact that you intended? To summarize the sequence of events as presented by Duncan (and please excuse the extended quote):

Following the ‘Rose Revolution’ of 2003 the new government made an attempt at revamping the country’s cumbersome and ineffective social protection scheme, inherited from the Soviet Union. In 2005, it set up the SSA (Social Subsidies Agency) to manage its new social assistance programme…The government and the donors declared victory, but monitoring by Oxfam and a local partner NGO – the Association of Young Economists of Georgia (AYEG) -showed that the system was not working… We held a number of closed door seminars with the representatives of the SSA (without involving the media so as to build trust) and backed them up with lobbying meetings. Nothing happened – we built up good relationships with young and motivated civil servants, but there was no appetite from their bosses for overhauling what was generally portrayed as a success. That all changed with the civil unrest of 2007, a political shock born of public disenchantment with government policy and reforms. The government was desperate to refurbish its image…Step forward a new and sympathetic Minister for Healthcare, with a background of working with NGOs, who gave his political backing…

Now this sounds (to me, at least) like quite a common story. It usually takes an external shock of some kind for any organisation to make radical  changes – in this case, first the Rose Revolution, and then public protests against the path taken since the Revolution. The problem is that this account of events also sounds (again, to me) like a large number of biases stacking up to explain something that was largely out of the two NGOs control. Duncan reports that

… our Georgia team put success down to three factors: the shift in the political environment; the rigorous use of evidence and the cultivation of contacts with a range of decision makers and officials.

Now clearly the shift in the political environment was a major factor, but the other two factors? The case looks much weaker. Duncan has described  the government as “desperate”, and I doubt that the rigor of the evidence that was presented to them was as critical as the fact that they were being offered a ready-made reform package that they could sell to the public. The “cultivation of contacts” were undoubtedly useful, but the key figure appears to have been a new Minister with an NGO background. This is crucial, because somebody from an NGO background is likely to have an automatic bias towards proposals from NGO (and the sort of approachs that NGOs propose). The Georgia team’s appraisal of what made change possible definitely has some merit – but it also looks like a way of retrospectively justifying personal and professional investment in an extremely worthwhile and challenging project.

Now I have no opinions on the actual reform proposals that Oxfam and AYEG put forward – certainly the brief description in Duncan’s article sounds good, but then the original concept of the SSA sounds pretty good as well. I have no reason to think that this reform has done anything except improve the welfare of the people the SSA is intended to support – but I also think that we need to examine the potential biases that lead us to assume that Change Y is the result of Action X, particularly in macro environments which are extremely complex.

This holds true for any area of aid work – how justified are our claims that our work helps people?

I promised myself that I’d blog every single day while I was working in Georgia. It should be fairly obvious that I didn’t. I can’t say that I was super productive while I was in Tbilisi – for a variety of reasons, including particularly dysfunctional co-ordination, but also because of the basic difficulty of getting good information in conflict situations.

In a natural disaster, government agencies and international organisations are usually relatively comfortable to share information about the situation – but in a conflict, they clam up tighter than my wallet around Christmas. This is because natural disasters have fewer political implications than complex emergencies; while in a natural disaster the worst thing you can say about a government is that they’re negligent, in a conflict situation the government is usually a belligerent,

This means that timely / reliable / accurate information is hard to come by in Georgia, as Ivan points out and Ethan overviews (is that a verb?). I find it hard to get too worked up about the lack of “citizen war reporters” even though it is my fervent hope that the web is going to change the way we do business in both complex emergencies and natural disasters. My lack of work-up is simply because even if there were shedloads of citizen journalists covering these events, I would still treat them exactly the same as any other information source – which is to say, I wouldn’t trust them at all.

As an example, the single most critical humanitarian information issue in Georgia was the numbers and locations of people displaced by the conflict. This was problematic for a number of reasons:

1. Nobody had a clue how many people had been displaced by the conflict. There were multiple government agencies involved in looking after the IDPs (frequently a euphemism for “ignoring them”, of course), each with their own figures, none of which tallied with the figures that UNHCR or the Red Cross had; and of course nobody in the humanitarian community had bothered to sit down and agree on a number we could all work to. Lesson from Afghanistan, folks: your numbers are never going to be 100% accurate, and it’s better just to agree to a number and get to planning than continually be running after the latest figures – which are also going to be wrong.
2. Nobody wanted to talk about the IDPs left over from the previous round of conflict in 1992-93; a staggering 220,000 people (not 100% accurate, of course – just run with it!) have been rotting in terrible conditions for the last 16 years, and some of their stories can be found on IDP Voices. Nearly all of us who were new to Georgia found this astonishing, since it raised a rather difficult question: what the *&%$ has the government and the UN been doing for the last 16 years? It also confuses the picture because in purely humanitarian terms many of these “old caseload” IDPs were in a worse situation than the “new caseload” – and many of the “normal” citizens live in conditions as bad as either.
3. For both old and new caseloads, the main priority is ensuring their basic shelter, which comes under the Emergency Shelter cluster. Unfortunately the UN in Georgia had decided that they didn’t want to activate the cluster system (because it’s a bit of a hassle and you have to actually take responsibility for your actions) but they did want to use some of the cluster tools (particularly the ones that give you a fat sack of cash to spend). This meant that it was like stepping into a time machine to 2004 – you remember, when “co-ordination” was a competition to see who could hold as many meetings as possible with as few outcomes as possible.
4. Notwithstanding the co-ordination problem, nobody had a clue what to do with all them displaced. The government unveiled a not unreasonable resettlement plan for the new caseload at the start of September, but that plan rapidly ran aground on the harsh reality that the stock and state of public buildings in Georgia are likely not sufficient to house the IDPs according to basic humanitarian standards, even on a short-term basis. (Some interesting discussion on this at the Social Science in the Caucasus blog.) The question is whether that government plan can be reshaped into a more realistic framework that will engage the entire humanitarian community as well as being attractive to donors…

One of the things about shelter issues is that they tend to get worse the longer you leave them. Conditions deteriorate, particularly when people are housed in buildings that were never designed for residential use. In this case, many of the new caseload had been placed in schools and kindergartens around Tbilisi and other towns – which meant that we also had to deal with the fact that those institutions were needed for the start of the new school year. This was a particular tension for UNICEF, who often run a “Back To School” program – which wouldn’t look too good if there weren’t any schools to go back to. In addition winters in Georgia can get unpleasant, especially the closer you get to mountains, and thus another constraint on resettlement.

You might have noticed that there wasn’t much talk about information in this blog post. That’s because there wasn’t much information, as I explained previously. We got hold of the complete set of school locations from the MInistry of Education (shape files ahoy!) but nobody seemed that interested. We tried to persuade the different actors – Red Cross, UNHCR, Ministries various – to consolidate the figures for collective centres and the IDPs therein, but with little luck. Paolo Palmero from OCHA had gathered a lot of data during his 2005 visit, but none of it seemed to be circulating in the agencies.

Summary version: this response showed yet again the importance of investing in information resources before an emergency hits. That doesn’t just mean getting loads of satellite images (although UNOSAT did some impressive work on damage levels) but investing in relationships with government, relationships that can be leveraged quickly to mutual benefit. It means having a basic picture already in place – locations of schools, for example – that you can then overlay new data on top of – such as the estimated IDP numbers in those schools. This really needs a collective approach – one agency alone isn’t sufficient to achieve success, although you need a focal point for the effort – but it continues to make me wonder if we should be thinking about setting up an organisation that collects and disseminates operational data like this.

At least that would avoid me feeling like a numpty, turning up at meetings with my tiny spreadsheet of schools that might need some watsan rehabilitation…

So my holiday is well and truly over, and I’m in Georgia for UNICEF on a ridiculously short contract, providing information management support for the WASH Cluster. Things are never that simple, of course, and so the work has turned out to be significantly more challenging than I expected. Right off the bat, the post-conflict situation in Georgia is a political crisis rather than a humanitarian crisis; yes, there are some tens of thousands of people displaced by the conflict, but almost none of them are in a life-threatening situation (until the winter comes, that is). Their livelihoods have been affected badly, which means that there are going to be ongoing concerns, but the scale of that problem in a middle income country doesn’t feel particularly desperate (especially now that we’re watching the footage of the monsoon floods in India which have displaced over 2 million people).

Of course that doesn’t mean that there isn’t any job here, or that I get to go back to the mountain tomorrow. There’s still a major co-ordination requirement – for IDPs that are stuck in collective centres, for IDPs that are returning home to their villages, for IDPs that have been moved into the tent camp(s) in Gori – and a real lack of decent information to support that co-ordination. Fairly obviously that’s where I come in, but the last week has not been a particularly productive one. Primarily this is because when I arrived there was absolutely no data to work with, and getting hold of it has proven to be an absolute nightmare. Information flows are incredibly weak, dialogue with the government is fragmented, the situation remains extremely fluid and there’s a lot of political sensitivies involved. On top of that, the WASH unit that I’m in didn’t exist until a couple of weeks ago; it’s been created solely because of the conflict and the need that UNICEF has to meet its obligations as the lead agency in the WASH cluster.

Bags of fun, which explains why I haven’t posted anything since I arrived. I promised myself that I was going to blog daily on the issues I was coming up against, but that’s clearly not worked out. However I will be writing a few pithy posts on specific issues, since as of two days ago data started appearing. It’s not great – patchy demographics, an improvised camp registration process, a few lists from government agencies and NGOs – but it’s a starting point. My job is to turn that data into something that can be used by the cluster to address the 5 strategic areas which we’ve identified, which are broadly:

1. Site planning of tented camps in Gori
2. Refurbishment of proposed Temporary Shelters
3. Cleaning of schools and kindergartens at national level (esp. Tbilisi)
4. Rehabilitation of existing Collective Centres (CCs) for longer-term caseload
5. Provision of village watsan for returning IDPs

As you can see, it’s not a particularly coherent set of requirements, which will make co-ordination even more difficult. The first step is to work out where the IDPs are and where they’re going to be going; the next step is to work out where the agencies are and how they’re working. Sounds simple, right?

Right.