Archive for the ‘complexity’ Category

If so much change occurs through word of mouth, how do we evaluate it?

Tuesday, April 13th, 2010

There is general agreement that word of mouth marketing is a critical element of changing behaviour. Whether it is a family member, colleague, neighbour, or friend, we are more likely to take on the advice and behaviours that are modelled by those we trust. This is the basis of effective communication (think also of the 6 degrees of separation experiment). Mark Earls, the author of Herd: how to change mass behaviour by harnessing our true nature provides great examples of how social networks are key to changing mass behaviour.

Mark recently posted a blog about how important it is to understand social networks.

Mark notes: Social networks are not channels for advertisers or for the adverts/memes you, your clients or any of your so-called “influentials” create, social networks are for all of the people who participate in the network.

So if word of mouth is an element of your behaviour change program (as it should be), how can you track its spread, and find out whom the key people are in networks? Well, social network analysis is one way! So what is a social network analysis?

Andrew Rixon, from Babelfish Group, notes in an e-booklet on enhancing collaboration that Social Network Analysis is the technique of analysing roles and social networks…. The outcomes of social network analysis provides surprising and insightful results allowing structure(s) to become visible and discussable.

Making such networks visible should surely be one of the goals of  evaluation. In this way, for those who have read Gladwell’s Tipping Point, you can find out who the mavens, connectors and salesmen are.

Interested in finding out more on Social Network Analysis?

Andrew Rixon will be holding a post-conference workshop on this very topic, so check out the program of post conference workshops and register online.

When evaluation reinforces the status quo

Friday, April 9th, 2010

I see a lot of similarities between behaviour change interventions for sustainability and international development assistance. Both fields seek to intervene to change participants’ behaviours, and generally this is done through a linear model of cause and effect, where the intervention is evaluated as the sole agent of change. In a recent post on complexity and development, Ben Ramalingam highlights a recent publication by Olivier Serrat, Head of Knowledge Management at the Asian Development Bank:

Development is a complex, adaptive process but—with exceptions—development work has not been conducted as such… development assistance often follows a linear approach to achieving outputs and outcomes……Any planning process is based on assumptions—some will be predictable, others wishful. If the assumptions are based on invalid theories of change (including cause-and-effect relationships) and on inappropriate tools, methods, and approaches derived from those, development agencies jeopardize the impacts they seek to realize.

In terms of evaluation, the risk is not solely that we jeopardise the impacts, but that we choose evaluation methods that will seek out what we want to show, whether this has actually occurred or not. If we are intent on showing a particular change, it is quite easy to (inadvertently or not) seek out what we (want to) believe actually happened, and by doing this we reinforce the perpetuation of behaviour change interventions that may not be all that successful. And in doing this we reinforce the status quo, rather than move towards better practices that account for complexity.

Here’s a nice quote from Aaron Levenstein to keep in mind:
Statistics are like bikinis. What they reveal is suggestive, but what they conceal is vital.

Exploding cylinders, complexity and evaluating behaviour change

Wednesday, March 10th, 2010

Last July, and an oxygen cylinder aboard a Qantas plane suffered what could be considered a relatively rare “behaviour change” in that it exploded, ripping a large hole in the fuselage. Following a stringent follow-up investigation (or evaluation), the cause of the exploding cylinder still remains a mystery. The news item reported that investigators even pressure tested the remaining gas bottles and none failed.
What this shows is that no matter how much knowledge we have about something, or no matter how many tests we replicate, some things remain a mystery. The reason behind the exploding cylinder are complex, and understanding the reason cannot be through attempting to replicate the problem by studying other cylinders, but through understanding the emergent properties related to that particular exploding cylinder. If only cylinders could tell a story? Except for the fact that this one is lost somewhere in the ocean!
So, what does that have to do with evaluating behaviour change? Well, people, like oxygen cylinders, are often considered similar and predictable in that what works for one is considered to work for others. But in reality, we are more like the “exploding cylinder” in that we often react unpredictably, or in a complex and unique manners, when placed in different situations. As such, it is hard to know what parameters to evaluate in behaviour change programs, as we cannot necessarily predict the outcome.
In understanding the reason for a change, what is important is asking the “one that changed”. If only investigators could get the story from the most significant (or exploding) cylinder? In evaluating behaviour change programs, we need to be more open to emergent properties, some which may be instantaneous and highly observable (like the exploding cylinder) and others that may occur over a longer term, and less visible.