This last couple of weeks, I’ve started to get my arms around another chunk of my work here at VSO, which is to develop the “Advocacy Strategy M&E Logframe” ….. huh?!! That stands for “Monitoring and Evaluation logical framework”, which is a structured approach for describing the line of sight from the ultimate Outcomes sought for beneficiaries, through the Outputs of your work, Indicators of progress (and how to measure them), Activities being undertaken to deliver them and associated Assumptions plus information on baseline, milestones and resources…..so in GSK-Speak, a bit of a mix between a Benefits Dependency Framework, project charter and a plan. This is a fundamental way of working in VSO and other NGOs, not least because it’s generally they way an NGO must define its objectives and report progress to the main donor organizations.
So far so logical ….until you then start twondering how on earth you measure your success in influencing something as intangible as an increase in women’s participation in civil society? And it gets even murkier when you consider that there will likely be plenty of other NGOs trying to influence on the same agenda so attributing progress to VSO’s specific activities seems nigh on impossible. A few indicators are readily quantifiable (e.g. number of signatories to a petition, number of letters written to an MP), but many are more qualitative (did a policymaker mention your initiative in a certain forum?)….and just like metrics anywhere, that which is measurable isn’t necessarily a meaningful indicator of progress. A colleague here relayed how sign-up to UK campaign petitions at the NGO he used to work at generally had a peak in the summer; much summer campaigning was done at music festivals, so maybe people were more receptive to the campaign message when high on a few pints of cider and the sounds of their favourite rock gods!
Last week, I had a meeting with a wonderful lady from Oxfam, who shared with me some of their approaches, including a smartphone app they are piloting as a data collection tool. Oh yes, that’s another challenge I haven’t mentioned yet – even if you have defined some robust measures, you then have to work out how to collect the “data” (which could be in text, numeric, video or audio form) from highly distributed and often remote locations. And to think I used to think that measuring the number of Candidate Selections in a year was difficult!
Luckily for me (and VSO!), my job isn’t to become an expert in all of this (I don’t have enough years left on this good Earth for that!), but to orchestrate and consolidate the work of others on the team into an overarching picture. I’m intrigued and looking forward to seeing what they come up with!