The Big Lottery Fund’s (BIG) Realising Ambition programme is a UK-wide initiative that has invested £25 million in outstanding projects that have proven their effectiveness in helping young people fulfil their potential and avoid the pathways into offending.
The Realising Ambition programme is doing this by helping to replicate the very best evidence based practice so that more children and young people receive the highest quality of service. During the first three years of the programme, the Tavistock Institute carried out a process evaluation to explore a range of practical issues to do with replication.
ContextThe Realising Ambition programme is the second major (and pilot) programme of Big Lottery Fund’s stream called ‘Replication and Innovation’. In the context of tightening public sector finances, the programme was created to move beyond (costly) innovation activities and experiment with a programme that would potentially show the value for money of replicating interventions that have proven to be effective. As such, Realising Ambition has three over-arching objectives:
- More young people benefiting from opportunities and support to fulfill their potential, avoiding pathways into offending. Yong people access support when they need it so they do not have to commit crimes. (Benefit to individuals)
- Organisations working with young people have better evidence of what works in avoiding pathways into youth offending and are able to replicate the most effective approaches. (Benefit to organisations)
- Big Lottery Fund and others learn about how they can best identify and support the replication of proven polices and practice (Benefit to funders and government)
ObjectivesThe process evaluation of Realising Ambition had two key objectives:
- To gain an understanding of the practical issues associated with replication, including issues emerging for organisations involved in replication themselves.
- To explore what does and doesn’t work when supporting organisations to replicate proven models, and the resources required to support different approaches to replication.
MethodologyThese objectives were met using a mixed methods approach centred around the following key components:
- An evidence review of factors that affect the success of replication and adoption of innovation more generally.
- Based on this review, the construction of a conceptual model to underpin process evaluation activities. According to this model, replication success is determined by a dynamic interplay of the intervention (and its features), the replicating organisation, the local context (of the replication destination) and the wider political economic and social environment. This is indicated by the dotted lines in the figure below.
- In-depth case study work with six Realising Ambition projects. These represented the Programme’s portfolio in terms of geographical spread, amount of funding received and level of intervention (individual, family, group, school, community, and multiple). We implemented four rounds of data collection with each case study, carrying out in-depth semi-structured interviews with project managers, deliverers and other relevant individuals. In addition we studied relevant project documents (including theories of change) and monitoring data from each case study project.
- Implementation of two ‘ad hoc’ case studies on replication of community based interventions to explore what specific challenges these kinds of interventions face in the replication process.
- Four rounds of in-depth semi-structured programme stakeholder interviews with the funder and core members of the consortium designing and running the Realising Ambition programme. These aimed to capture lessons learned from the delivery of a major replication programme.
- Analysis of programme level monitoring data on numbers of participants engaged and support delivered and attended.
- Construction of a set of replication process indicators. These were co-designed with the Realising Ambition consortium at the beginning of the process evaluation in order to ensure they were fit for purpose, both in terms of their focus and the range of data collected through the programme’s monitoring processes.
ResultsThe process evaluation of the Realising Ambition programme has generated the following key insights.Replication modelsIn Realising Ambition we have found the full range of replication business models represented (definitions taken from the social franchising manual):
- Social franchising: a legal agreement between franchisor and franchisee giving the right to use systems, brand and other intellectual property, and to use those to operate an identical business.
- Licensing: granting a license to provide a service or sell a product (rather than an entire business format or system).
- Wholly owned: the intervention owner replicates the intervention with own staff.
- Direct delivery. Here, the organisation in charge of replication also implements the intervention elsewhere, retaining all the responsibilities that come with replication, most notably recruitment of beneficiaries in the replication destinations and delivery. In Realising Ambition we have only found this model being used by organisations using the wholly owned business model. A variation of this model involves beneficiaries in the replication destination being recruited through local gatekeeper organisations (direct delivery, referral model).
- Indirect delivery. Here, the intervention owner selects and trains a delivery organisation in the replication destination to deliver the proven practice or model in-house. We have found this model being used both with the licensing and wholly owned business models. .
- Third party delivery involves a (local) organisation being contracted for replication purposes. This organisation is then responsible for recruiting beneficiaries and delivering the intervention. In difference to the indirect delivery model, delivery is not in-house but will involve identifying beneficiaries and delivering externally. A variation of this model then is gatekeeper recruitment, where the contracted organisation relies on local gatekeepers to access beneficiaries. We have found these models being used in conjunction with all three replication business models.
- Existing relationships in the replication destinations mean that knowledge of the intervention, its benefits and the integrity of the replicating organisation has proven itself already through prior work.
- This, coupled with local reputation, ensures that set-up costs such as ‘marketing’ are reduced, and control over access to target groups by those in a gatekeeping position is reduced.
- The nature of the intervention and its setting: the intervention needs to be compatible with the organisational ethos, and local or national context. Organisations and professionals usually find it easier to adapt to the rigours of replication if the intervention chimes with its usual focus and ways of working. For instance, it seems to be easier to ‘sell’ an intervention to schools if a link to the national curriculum can be demonstrated.
- Good marketing: presenting the intervention well and to the right audience emerged as an important technique to ‘win’ participants. This is especially important where relationships are weak or non-existent. Effective marketing often involved a mix of written and verbal information about the intervention.
- Internal resources to support data collection across multiple sites, which requires more discipline and planning than organisations might be used to.
- Management processes, structures and functions. Organisations need to adapt to replication by ensuring staff with appropriate seniority, autonomy and line management support are in place. Senior management and Board support is valuable particularly for the sustainability of the replication venture, and appropriate engagement should be sought from the beginning.
- Communication patterns or mechanisms with colleagues across sites and/or with partner organisations to ensure that risks are monitored and managed.
- Support for intervention specificity benefits from being front loaded: rigorous mapping of the intervention enables the very important identification of the ‘core’ and ‘peripheral’ aspects of interventions.
- Support for the organisational dimension of replication becomes more relevant and beneficial at a later stage in the replication process. It should therefore start and finish later. This work is important in ensuring the sustainability of the Realising Ambition programme after funding ends.
ImpactWe have distilled the key findings from this process evaluation in two sets of replication guidelines addressed at funders and commissioners of replication projects:
- 10 things to think about when…Selecting Interventions and Organisations for Replication covers topics such as understanding the nature and evidence base of an intervention, business models and the organisational aspects of replication. (download as PDF)
- 10 things to think about when…Supporting Interventions and Organisations for Replication covers questions such as why to support replication in the first place as well as content, timing and delivery of support. (download as PDF)