I see a lot of similarities between behaviour change interventions for sustainability and international development assistance. Both fields seek to intervene to change participants’ behaviours, and generally this is done through a linear model of cause and effect, where the intervention is evaluated as the sole agent of change. In a recent post on complexity and development, Ben Ramalingam highlights a recent publication by Olivier Serrat, Head of Knowledge Management at the Asian Development Bank:

Development is a complex, adaptive process but—with exceptions—development work has not been conducted as such… development assistance often follows a linear approach to achieving outputs and outcomes……Any planning process is based on assumptions—some will be predictable, others wishful. If the assumptions are based on invalid theories of change (including cause-and-effect relationships) and on inappropriate tools, methods, and approaches derived from those, development agencies jeopardize the impacts they seek to realize.

In terms of evaluation, the risk is not solely that we jeopardise the impacts, but that we choose evaluation methods that will seek out what we want to show, whether this has actually occurred or not. If we are intent on showing a particular change, it is quite easy to (inadvertently or not) seek out what we (want to) believe actually happened, and by doing this we reinforce the perpetuation of behaviour change interventions that may not be all that successful. And in doing this we reinforce the status quo, rather than move towards better practices that account for complexity.

Here’s a nice quote from Aaron Levenstein to keep in mind:
Statistics are like bikinis. What they reveal is suggestive, but what they conceal is vital.

Last July, and an oxygen cylinder aboard a Qantas plane suffered what could be considered a relatively rare “behaviour change” in that it exploded, ripping a large hole in the fuselage. Following a stringent follow-up investigation (or evaluation), the cause of the exploding cylinder still remains a mystery. The news item reported that investigators even pressure tested the remaining gas bottles and none failed.
What this shows is that no matter how much knowledge we have about something, or no matter how many tests we replicate, some things remain a mystery. The reason behind the exploding cylinder are complex, and understanding the reason cannot be through attempting to replicate the problem by studying other cylinders, but through understanding the emergent properties related to that particular exploding cylinder. If only cylinders could tell a story? Except for the fact that this one is lost somewhere in the ocean!
So, what does that have to do with evaluating behaviour change? Well, people, like oxygen cylinders, are often considered similar and predictable in that what works for one is considered to work for others. But in reality, we are more like the “exploding cylinder” in that we often react unpredictably, or in a complex and unique manners, when placed in different situations. As such, it is hard to know what parameters to evaluate in behaviour change programs, as we cannot necessarily predict the outcome.
In understanding the reason for a change, what is important is asking the “one that changed”. If only investigators could get the story from the most significant (or exploding) cylinder? In evaluating behaviour change programs, we need to be more open to emergent properties, some which may be instantaneous and highly observable (like the exploding cylinder) and others that may occur over a longer term, and less visible.