Posts Tagged ‘Evaluation’

Overcoming existing evaluation cultures and processes

Monday, March 29th, 2010

A post on evaluation and complexity on Rick Davies monitoring and evaluation news site has a link to a great powerpoint on evaluation and the science of complexity by Ben Ramalingam. Ben notes that for many organisations, evaluations are at the centre of a vicious circle that includes pressure to show results and impacts, and poor learning and accountability amongst others.

Further, Ben notes that “Evaluations are still largely focused on reports as opposed to changed behaviours, ways of thinking and attitudes”. This seems very true, and I am sure many of us would recollect knowing of reports that have been produced for the report’s sake, and not what is in it.

A nice slide from Ben Ramalingam's powerpoint

Image source: http://www.outcomemapping.ca/resource/resource.php?id=183


The image, taken from one of Ben’s slides, encapsulates well the idea that existing process and culture can overshadow the ability to undertake more effective evaluation.

In another post, Ben notes “Some of the issues for evaluation include the tension between learning and accountability, the limits of attribution, how evaluations are or are not used, equality and power, and ideological debates about methodologies, such as the dominance of randomised controlled trials (RCTs)……Although there is a wealth of evaluation methods in theory, in practice they are largely required to conform to scientific management principles……..In contrast, complexity theory (theories) talks about systems that are interconnected, driven by feedback, where the properties of the system are not predictable but emerge from the relationships within that system……..It may be that we need to stop focusing on projects, and look more broadly at the societies that we work in and across sectors and institutions rather than within them. Evaluations may need to be more centred on real-time learning and helping managers adapt what they do.”

This is what Show me the Change is about- discussing how as a community of practice we can overcome the real or imagined culture that can negatively impact on more novel, experimental, and altogether better evaluation practice and processes.

So if you are interested in evaluation, behaviour change and sustainability in a complex workd, take part in the conversations that matter, on 4-6 May in Melbourne.

What do we measure and Why?

Sunday, February 28th, 2010

Meg Wheatley on great questions to ask as we think about measurement, especially in complex living systems (like human communities):

Who gets to create the measures? Measures are meaningful and important only when generated by those doing the work. Any group can benefit from others’ experience and from experts, but the final measures need to be their creation. People only support what they create, and those closest to the work know a great deal about what is significant to measure.

How will we measure our measures? How can we keep measures useful and current? What will indicate that they are now obsolete? How will we keep abreast of changes in context that warrant new measures? Who will look for the unintended consequences that accompany any process and feed that information back to us?

Are we designing measures that are permeable rather than rigid? Are they open enough? Do they invite in newness and surprise? Do they encourage people to look in new places, or to see with new eyes?

Will these measures create information that increases our capacity to develop, to grow into the purpose of this organization? Will this particular information help individuals, teams, and the entire organization grow in the right direction? Will this information help us to deepen and expand the meaning of our work?

What measures will inform us about critical capacities: commitment, learning, teamwork, quality and innovation? How will we measure these essential behaviors without destroying them through the assessment process? Do these measures honor and support the relationships and meaning-rich environments that give rise to these behaviors?

via Margaret J. Wheatley: What Do We Measure and Why?.

These are great questions to consider at this Show Me The Change conference as we dive into questions on the implications for complexity on the measurements used to evaluate change in living and complex systems.

Chris Corrigan – SMTC Design Team

Cross posted from Chris’ Parking Lot here