The Big Lottery Fund’s (BIG) Realising Ambition programme is a UK-wide initiative that has invested £25 million in outstanding projects that have proven their effectiveness in helping young people fulfil their potential and avoid the pathways into offending.

The Realising Ambition programme is doing this by helping to replicate the very best evidence based practice so that more children and young people receive the highest quality of service. During the first three years of the programme, the Tavistock Institute carried out a process evaluation to explore a range of practical issues to do with replication.

Context

The Realising Ambition programme is the second major (and pilot) programme of Big Lottery Fund’s stream called ‘Replication and Innovation’. In the context of tightening public sector finances, the programme was created to move beyond (costly) innovation activities and experiment with a programme that would potentially show the value for money of replicating interventions that have proven to be effective. As such, Realising Ambition has three over-arching objectives: 

• More young people benefiting from opportunities and support to fulfill their potential, avoiding pathways into offending. Yong people access support when they need it so they do not have to commit crimes. (Benefit to individuals)
• Organisations working with young people have better evidence of what works in avoiding pathways into youth offending and are able to replicate the most effective approaches. (Benefit to organisations)
• Big Lottery Fund and others learn about how they can best identify and support the replication of proven polices and practice (Benefit to funders and government)

Learning is an explicit and key feature of Realising Ambition, and the process evaluation was commissioned to support this.

Objectives

The process evaluation of Realising Ambition had two key objectives: 

• To gain an understanding of the practical issues associated with replication, including issues emerging for organisations involved in replication themselves.
• To explore what does and doesn’t work when supporting organisations to replicate proven models, and the resources required to support different approaches to replication.

The evaluation covered the first three (of five) years of the Realising Ambition programme. It included the programme’s application phase, set-up (2011-2012) and the first two years of full implementation (until March 2015). A separate impact evaluation was carried out by one of the members of the UK consortium delivering the Realising Ambition programme.

Methodology

These objectives were met using a mixed methods approach centred around the following key components: 

• An evidence review of factors that affect the success of replication and adoption of innovation more generally.
• Based on this review, the construction of a conceptual model to underpin process evaluation activities. According to this model, replication success is determined by a dynamic interplay of the intervention (and its features), the replicating organisation, the local context (of the replication destination) and the wider political economic and social environment. This is indicated by the dotted lines in the figure below.

• In-depth case study work with six Realising Ambition projects. These represented the Programme’s portfolio in terms of geographical spread, amount of funding received and level of intervention (individual, family, group, school, community, and multiple). We implemented four rounds of data collection with each case study, carrying out in-depth semi-structured interviews with project managers, deliverers and other relevant individuals. In addition we studied relevant project documents (including theories of change) and monitoring data from each case study project.
• Implementation of two ‘ad hoc’ case studies on replication of community based interventions to explore what specific challenges these kinds of interventions face in the replication process.
• Four rounds of in-depth semi-structured programme stakeholder interviews with the funder and core members of the consortium designing and running the Realising Ambition programme.  These aimed to capture lessons learned from the delivery of a major replication programme.
• Analysis of programme level monitoring data on numbers of participants engaged and support delivered and attended.
• Construction of a set of replication process indicators. These were co-designed with the Realising Ambition consortium at the beginning of the process evaluation in order to ensure they were fit for purpose, both in terms of their focus and the range of data collected through the programme’s monitoring processes.

Collaboration with the consortium in charge of designing and implementing the Realising Ambition programme was an important element of implementing this process evaluation. This ranged from coordinating visits to project to co-designing indicators, negotiating access to key monitoring information and contributing to a learning event.

Results

The process evaluation of the Realising Ambition programme has generated the following key insights.Replication modelsIn Realising Ambition we have found the full range of replication business models represented (definitions taken from the social franchising manual): 

• Social franchising: a legal agreement between franchisor and franchisee giving the right to use systems, brand and other intellectual property, and to use those to operate an identical business.
• Licensing: granting a license to provide a service or sell a product (rather than an entire business format or system).
• Wholly owned: the intervention owner replicates the intervention with own staff.

In addition, we found that replication happens through a range of organisational models:

• Direct delivery. Here, the organisation in charge of replication also implements the intervention elsewhere, retaining all the responsibilities that come with replication, most notably recruitment of beneficiaries in the replication destinations and delivery.  In Realising Ambition we have only found this model being used by organisations using the wholly owned business model. A variation of this model involves beneficiaries in the replication destination being recruited through local gatekeeper organisations (direct delivery, referral model).
• Indirect delivery. Here, the intervention owner selects and trains a delivery organisation in the replication destination to deliver the proven practice or model in-house. We have found this model being used both with the licensing and wholly owned business models. .
• Third party delivery involves a (local) organisation being contracted for replication purposes. This organisation is then responsible for recruiting beneficiaries and delivering the intervention. In difference to the indirect delivery model, delivery is not in-house but will involve identifying beneficiaries and delivering externally.  A variation of this model then is gatekeeper recruitment, where the contracted organisation relies on local gatekeepers to access beneficiaries. We have found these models being used in conjunction with all three replication business models.

These models represent decreasing levels of control over the replication process by the organisation owning the rights to the intervention. Challenges can be expected to increase as control reduces.Factors supporting and challenging replicationWe have found that four factors universally support successful replication: 

• Existing relationships in the replication destinations mean that knowledge of the intervention, its benefits and the integrity of the replicating organisation has proven itself already through prior work.
• This, coupled with local reputation, ensures that set-up costs such as ‘marketing’ are reduced, and control over access to target groups by those in a gatekeeping position is reduced.
• The nature of the intervention and its setting: the intervention needs to be compatible with the organisational ethos, and local or national context. Organisations and professionals usually find it easier to adapt to the rigours of replication if the intervention chimes with its usual focus and ways of working. For instance, it seems to be easier to ‘sell’ an intervention to schools if a link to the national curriculum can be demonstrated.
• Good marketing: presenting the intervention well and to the right audience emerged as an important technique to ‘win’ participants. This is especially important where relationships are weak or non-existent. Effective marketing often involved a mix of written and verbal information about the intervention.

Replication challenges differ according to the organisational model chosen to implement the replication endeavour. A direct delivery model can make it difficult to recruit and deliver at the same time. An indirect delivery model may import issues out of control of the grant holding organisation into the replication process. These can impact on the ability to reach beneficiaries within the set time-frames and set locations.Replication and adaptationAt its simplest, replication can be understood as implementing a project, model or process in a different geographic location or with a different group of people. However, this never means simply copying what has worked previously. Rather, it means adapting both the intervention (e.g language, delivery style or setting) and organisational processes, structures and ways of working to meet the needs of a new locality and / or target group. So that these modifications do not affect outcomes, it is essential to understand what the core unchangeable elements of the model, practice or project are that are to be replicated, and which aspects can be modified without compromising results. A strong theory of change as well as continuous monitoring and outcome measurement are therefore tools integral to the replication process.Furthermore, the process of adaptation is ongoing whilst replication continues. Each new area or target group tends to bring slightly different challenges for the delivery of the evidence based intervention. And increasing scale creates new challenges for the organisation leading the replication effort.  A strong theory of change and continuous monitoring and evaluation will ensure faithfulness to the original model and an improving knowledge of exactly what the unchangeable core is. An organisational culture that values evidence based practice, learning and risk taking and where senior managers are actively supporting the replication effort will be able to cope better with these demands than organisations where these factors are missing.Impact on organisationsReplication in the Realising Ambition programme required high standards of evidence and the taking of interventions to new contexts or target groups. This often meant adaptations were needed in the lead organisation, in the delivery organisations (or their sites), and in the relationship between the two. This included adapting: 

• Internal resources to support data collection across multiple sites, which requires more discipline and planning than organisations might be used to.
• Management processes, structures and functions.  Organisations need to adapt to replication by ensuring staff with appropriate seniority, autonomy and line management support are in place. Senior management and Board support is valuable particularly for the sustainability of the replication venture, and appropriate engagement should be sought from the beginning.
• Communication patterns or mechanisms with colleagues across sites and/or with partner organisations to ensure that risks are monitored and managed.

Besides changes to processes and structures, cultural aspects of organisational life are important for replication success. The ability to adapt and take on a replication venture appears to require similar characteristics to those necessary to take on the project in the first place: an openness to innovation, a strong ‘learning culture’ and a serious strategic interest in an evidenced-based way of working.Supporting replicationRealising Ambition projects received a significant amount of support, to refine and replicate their interventions faithfully and to aid the organisational buy-in to the concept of ‘replication’. Key learning from the first three years of programme delivery is that: 

• Support for intervention specificity benefits from being front loaded: rigorous mapping of the intervention enables the very important identification of the ‘core’ and ‘peripheral’ aspects of interventions.
• Support for the organisational dimension of replication becomes more relevant and beneficial at a later stage in the replication process.  It should therefore start and finish later. This work is important in ensuring the sustainability of the Realising Ambition programme after funding ends.

The comprehensive support architecture of Realising Ambition has been a key component in helping projects to not only deliver their targets, but to thrive and spread proven interventions. The level of support distinguished Realising Ambition from other programmes and helped ensure that (to-date) none of the 25 projects has failed. Having support for delivery, learning and influencing are important to ensure that replication is successful both now and following the end of the funding period.

Impact

We have distilled the key findings from this process evaluation in two sets of replication guidelines addressed at funders and commissioners of replication projects: 

• 10 things to think about when…Selecting Interventions and Organisations for Replication covers topics such as understanding the nature and evidence base of an intervention, business models and the organisational aspects of replication. (download as PDF)
• 10 things to think about when…Supporting Interventions and Organisations for Replication covers questions such as why to support replication in the first place as well as content, timing and delivery of support. (download as PDF)

You can also read the report Process Evaluation of the Realising Ambition ProgrammeMore about Realising Ambition:

• https://www.biglotteryfund.org.uk/realisingambition
• https://www.biglotteryfund.org.uk/research/making-the-most-of-funding/replicating-success
• http://www.catch-22.org.uk/programmes-services/realising-ambition/
• http://bigblog.org.uk/tag/realising-ambition/

Tavistock Institute contact for project:Dr Kerstin Junge (Project Director) Giorgia Iacopini (Project Manager)