|
The Kenya Financial Sector Deepening (FSD) programme was established in early 2005 to support the development of financial markets in Kenya as a means to stimulate wealth creation and reduce poverty. Working in partnership with the financial services industry, the programme’s goal is to expand access to financial services among lower income households and smaller enterprises. It operates as an independent trust under the supervision of professional trustees, KPMG Kenya, with policy guidance from a Programme Investment Committee (PIC). Current funders include the UK’s Department for International Development (DFID), the Swedish International Development Agency (SIDA), and the Bill and Melinda Gates Foundation.
iii The second approach was more prospective in nature. Through interviewing groups of stakeholders, and exploring what was happening in relation to the project and its wider context, it was possible to collect 'narrative fragments' that gave evidence of emerging changes and trends that were still 'growing shoots' or 'weak signals' but were not necessarily yet amenable to quantitative analysis. This method was able to show the beginnings of impact pathways which went beyond the intended theory of change. It gave indications of where project teams might monitor some of these trends, where consideration needs to be given as to how these growing impacts might help or hinder the core intentions of the project and how this might inform programming practice as well as provide a more complex view of what had been achieved.
p.2 Complexity theory provides a world-view, derived from the science of open systems and of non-linear interactions between elements at the micro-level. It describes and articulates the characteristics of a world that is interdependent and shaped by history and context, a world that is dynamic and subject to rapid shifts and the associated emergence of new futures. It provides a fundamental, scientifically-informed 'ontology', that is to say, a description of the way the world is - a perspective on 'being'. It also considers the processes of interaction and change - a perspective on 'becoming'.
p.2 Complexity theory provides explanations of the way in which patterns and order can emerge... few outcomes have single causes and few interventions or events have single outcomes. More than that, interconnections cannot in general be treated as independent of each other, as summative... their impact is more than - or less than - the sum of what their impacts would have been if each factor had operated in isolation.
p.2 Complexity theory, however, goes beyond systems thinking, which tends in the main to focus on situations, contexts and interventions as if they were stable (P. Allen & Boulton, 2011). Systems methods in general provide a description of the present; complexity theory in addition pays attention to the dynamic and evolutionary nature of the context, and the way new features can emerge. Many factors can contribute to the emergence of new features: specific events, local variations, shifting environmental factors, and reflexively-developing intentions and strategies. Complexity can be considered as a dynamic perspective on the way institutions are created and modifying or destroyed by events and the particularities of people, decisions and chance variations.
p.2 complexity theory emphasises the interplay between currently - instituted patterns of relationships and events. [JLJ - yes, this is exactly what we need for game theory. Certain patterns (in relationships) will provide insight into changes that might be worth monitoring.]
p.4 in order to explore emerging impact pathways it is necessary to be sensitive to 'weak signals' - that is to look for signs of change before they become established into new patterns.
p.4 A central goal of this research is to experiment with and develop methodologies which allow programming, managing, evaluating and assessing impact in a complex world.
p.4 Complexity theory sees change as arising at the microscopic level, in the detailed interactions in particular circumstances. Change may then aggregate from these beginnings and become more sustained. However, to understand change and spot emerging impacts which might inform programme changes or decisions to monitor new features, there needs to be ways of spotting signs of such emerging change.
p.5-17 Boring case studies... yaaaawnnnn
p.17 At its extreme, complexity theory could be taken to imply that there is no point in planning or monitoring because the whole focus is about dealing with uncertainty and adapting to changing circumstances. We would not take this view, however. For large-scale interventions... being clear - as to goals and methods and monitoring - and persisting in their achievement, remain of central importance. What changes (with a complexity worldview) is how to plan for a greater degree of flexibility to respond to the unexpected, to seize opportunities and to adapt to changing circumstances. It is also necessary to be able to spot both anticipated and unanticipated emerging signs of change by embedding processes of monitoring into the project.
p.17-18 The initial steps of the project are planned and piloted (or more than one approach may be piloted), to gain insight into what works best and under what circumstances. Based on the learning from the pilot, a wider implementation is then carried out. But the difference here is that there is a strong recognition that approaches may still need to be customised for different circumstances and need to adapt to what emerges.
p.18 In general then, a complexity-informed approach to implementation finds a new middle ground between pre-planning, analysis and persistence on the one hand and agility, experimentation and adaptation on the other. This is in contrast to traditional approaches which tend to emphasise the former over the latter. A complexity-informed approach requires shorter cycles of planning and review to respond to the dynamic nature of change and the potential for changing contexts.
p.18 These ideas are not new and would be considered good practice in much project implementation - but a complexity worldview stays the hand of [JLJ - added bullets below for readability]
- those who would cut contingency budgets,
- those who would feel that there is a ‘best practice’ approach that can be applied universally,
- those who would expect things, once planned, to go to plan and who would feel that implementation can be undertaken without too much attention to its interconnections with wider systemic factors.
p.18 The FSD approach has been to pilot options and closely evaluate their effectiveness - and the complexity view emphasises the need for continuing review and looking-out for the unexpected.
p.18 How can learning about adaptation to contexts best occur?
p.19 Implications for evaluation [JLJ - bullets below added for readability]
- Complexity thinking requires a path-dependent approach that connects the past with the present and the present with the emerging future. So any project team or management team must be adept at looking forwards - be able to scan for changes and explore scenarios and 'foresight'... They must be adept at... identifying what deep-seated institutionalised factors are in play. And they must look in detail at the present, seeing the details and particularities of the context, both wide and narrow.
- Complexity thinking also requires recognition of the episodic nature of change. This requires sensitivity to critical junctures which may become the moments for radical change to emerge...
- Finally, complexity thinking emphasises the dynamic and uncertain nature of 'the way things are' - the fact that there are inevitably unintended consequences and unexplored events and shifts.
How do we encapsulate these requirements into competencies? Box 6 provides a summary.
p.20 Box 6: How should a project team behave to be successful in a complex world?
A meta-theory of implementation
•Aspire
- Take a wide and systemic view of emerging trends - foresighting and scenario-planning.
- Articulate long-term goals and intentions and use these to develop plans, prioritise actions, and determine which opportunities to seize.
- Persist when there is little evidence of change.
•Anticipate
- Think a few steps ahead during implementation. What might be the consequences of actions, or the consequences of events in the wider context?
- Think through where critical junctures might occur.
- Scan for changes in the context.
•Adapt
- Adapt to unexpected events, unintended outcomes.
- Seize opportunities, making choices in line with long-term goals.
- Plan activities and projects but review progress regularly and take note of changing circumstances and be prepared to modify if necessary.
•Customise
- Take account of contextual and historical factors in developing plans and programmes. Do not expect there to be a universally-appropriate best approach.
- Experiment - pilot approaches, learn from what works.
- Take note of the specificity of skills and relationships and factors in the wider environment.
[JLJ - this high-level plan for dealing with complexity should be highlighted. In essence, it is the proposed 'silver bullet' for slaying the complexity monster. The authors then show how their approach follows these guidelines.]
p.21 The key requirement of impact assessment is that it is necessary to have a robust approach to inferring causality from the intervention to the results.
p.22 Impact assessment must always look for unexpected impacts.
p.24 The most effective way to try to gain evidence of contribution is for the project team to look out for signs of emerging change or changing conditions and to start to monitor such changes as implementation proceeds... As these are noted some will appear to fade away while others may become particularly important. Decision can then be made as to which to trace.
p.22-25 There are a number of further implications of complexity thinking for conceptualising and assessing the impact of interventions. Baselines... Emerging change... Impact in relation to the episodic nature of change... Time... complexity thinking emphasis the dimension of time - the need to trace sequences of events and trace how impact emerges is part of the view of things constantly 'becoming' rather than 'being'... External validity... even in apparently very similar situations, it is important to remember that local events and circumstances will always make a difference to the pathway and outcomes will never be identical in different circumstances... Based on the above considerations two related approaches are proposed through which FSD can more systematically engage with an impact assessment agenda in the context of complexity... Working prospectively... Working retrospectively
p.27 Complexity thinking provides a critique of and challenge to the level of certainty that is frequently assumed in conventional methods of programming and impact analysis. The over-arching message is that we can predict, plan, and attribute with less certainty than we would like. Complexity theory provides a worldview emphasising interconnectedness and the potential for the emergence of new factors. It can lead to a change in emphasis between planning and experimenting, between efficiency and responsiveness, between clarity of focus and breadth of perspective. It suggests that it is necessary to be somewhat tentative in concluding what works, and to be more willing to experiment. It emphasises that it is necessary to build in processes of learning and adaptation which respond to context and changing circumstances. It brings into question whether there are ‘best’ methods or unique frameworks. It recognises that taking note of history, understanding context and identifying changes through the building of a strong stakeholder group, and judging when to seize opportunities or adapt may be equal in importance to careful design and planning.
p.28 Realist evaluation focuses on understanding how the intervention mechanisms and their surrounding context produce outcomes in order to answer the question of what works, how, under which conditions and for whom (ibid:8). It focuses on identifying, collecting and testing the evidence for Contribution Mechanism Outcome theories which are intended to build middle-range theories of how change happens. General Elimination Methodology looks at the range of possible causes and eliminates those for which there is no evidence of a causal link. Process tracing draws on evidence to connect an intervention to an outcome, and then identifies causal mechanisms to connect them and the evidence that would need to be observed to verify the actual process at work. Contribution analysis builds on this more process based approach to allow for reasonable inference - in contrast to attribution which focuses on necessary and sufficient conditions (Mayne, 2011). This argues that a reasonable contribution can be claimed if (i) there is a reasoned theory of change; (ii) that the implemented activities are those set out in the theory of change; (iii) the chain of expected results can be shown to have occurred; and (iv) other influencing factors have either been shown not to make a difference or their relative contribution has been recognised.
p.29 there is an interplay between the context and the intervention. Simple interventions in chaotic contexts will be much more difficult to evaluate than simple interventions in stable contexts.
p.29-30 Patton (2011)... His thesis is that in general, environments or contexts are complex and dynamic, and thus interventions which take place in these environments initiate multiple pathways. What happens as a result of a planned intervention cannot, therefore, be determined with any certainty in advance... Patton therefore argues in considering theories of intervention, it is best to explore possibilities, try things out, adapt in accordance with what emerges... His advice, in order both to determine the impact that is emerging and to be able to modify interventions to deal with unintended outcomes or unexpected events, is to track emerging interconnections. He suggests that 'causality' is to be understood as based on pattern detection, retrospectively constructed - not on trying to work out what causes what... He states that what needs to be measured can change during the evaluation as the process unfolds and new factors emerge and that it is important to track the forks in the road and implications of key decisions as innovation evolves. He emphasises that checking whether and how unanticipated and emergent features are dealt with is a fundamental function of evaluation.
p.30 Complexity thinking supports the view that approaches to change and ways to implement will need to be sensitive to context and adapt to what emerges.
p.31 Clandinin and Connelly (2000:17) say:
'If we understand the world narratively, as we do, then it makes sense to study the world narratively. For us, life... is filled with narrative fragments, enacted in storied moments of time and space.'
The collection and analysis of narrative fragments formed the basis for our piloted process of identifying emerging impact pathways, as described in sections 3 and 4. The work described here only included one 'pass' at this process. To capture emerging trends would require more than one 'pass' spaced out over time, so it is possible to see whether emerging trends do indeed emerge, how they interact and what else happens that supports or hinders this.
|
