This evaluation was part of a set of ex-post evaluations of the European Social Fund (ESF) aimed at reporting on the use of the ESF and assessing the effectiveness, efficiency and impact of this financial assistance instrument. However, as distinct from the other evaluations, the object of this evaluation was not the ESF Programme and its interventions per se. Rather, this evaluation moved the body of work on ESF evaluations forward by evaluating the relationship between the ESF and the Social Protection and Social Inclusion Open Method of Coordination (SPSI OMC) at the levels of objectives, interventions, stakeholders and indicators – or, in short, the support ESF provides to SPSI OMC.

The evaluation was undertaken by the Istituto per la Ricerca Sociale (Lead Contractor) in partnership with The Tavistock Institute of Human Relations (TIHR), The Research Institute for Quality of Life and The Rehabilitation Foundation.

With the creation of the Lisbon strategy the EU triggered a process that interconnects social protection and social inclusion policies (based on OMC Social Protection Social Inclusion) with those for employment and growth: “the interaction between the OMC and the revised Lisbon process should be a dual one – social protection and inclusion policies should support growth and employment objectives, and growth and employment objectives should support social objectives”.

Context

This evaluation was commissioned to assess the support ESF provided to the SPSI OMC during the 2000-2006 ESF programme period at four levels: objectives, interventions, stakeholders/target groups and indicators. The assumption of the evaluation was that these two tools, in order to be efficient, need to be coherent at all 4 levels. ‘Support’ is assessed in relation to two key criteria: coherence (defined as a measure of the correspondence and consistency between ESF and SPSI OMC) and complementarity (defined as a measure of reciprocity between the ESF and SPSI OMC). The evaluation sought to understand if the ESF could be made to be a more suitable tool for supporting the SPSI OMC. In conclusion the evaluation provided operational recommendations for the new programmes (2007-2013) and the next round of Structural Funds (post 2013), in particular the ESF as well as the next cycle of OMC, the Lisbon Strategy (EU 2020) and the future Social Policy Agenda.The evaluation proved to be fairly challenging because the ‘object’ of the evaluation was rather “elusive” - not the whole ESF itself, but the coherence and the complementarity of the ESF 2000-2006 with SPSI OMC, whilst the ESF 2000-2006 Operational Programmes were in fact programmed before the whole conceptualising of SPSI OMC. In addition, it was found that there is scarce reciprocal knowledge of the “two worlds” of ESF and SPSI OMC (that is, ESF respondents typically had limited knowledge of the SPSI OMC and vice versa). Finally, the coherence and complementarity between ESF and SPSI OMC was not a priority for many Member State representatives or for ESF Managing Authorities (which impacted on primary data collection response rates).

Objectives

As indicated the over-arching evaluation objective was to provide an assessment at four levels (objectives, interventions, target groups / stakeholders, and indicators) of the coherence and complementarity of ESF Programmes with SPSI OMC.Five key evaluation questions were to be answered: 

1. How coherent and complementary the objectives of ESF Programmes are with the SPSI OMC?
2. How coherent and complementary the interventions and target groups of ESF Programmes are with the SPSI OMC?
3. How coherent and complementary public institutions and other main stakeholders involved with the ESF Programmes are with the SPSI OMC?
4. How coherent and complementary the used indicators in ESF Programmes are with SPSI OMC?
5. How suitable is the ESF as a tool to progress in the field of the OMC? And how the tool can be improved to ensure better coherence and complementarity with the SPSI OMC?

The evaluation essentially had three purposes starting from a descriptive one – to map and make sense of the ways in which the ESF Programmes are coherent with and complementary to the objectives of the SPSI OMC, and in particular its underpinning of ‘social inclusion’ agendas. More importantly however, the evaluation had an analytical and exploratory purpose: creating knowledge and improving understanding about how the ESF Programmes support the SPSI OMC. Finally, the study was meant to apply what has been learned to support the next ESF programming round and structural funds negotiations, as well as the next cycle of SPSI OMC, the future Lisbon Strategy (EU 2020) and the future Social Policy Agenda.

Methodology

The evaluation methodology involved triangulation of different data sources and methodologies, reflecting the different ‘voices’ of the various stakeholders involved. It included desk research and a literature review; interviews with Commission officials and national/regional stakeholders; a mapping exercise of ESF Programmes and SPSI OMC national documents including Country Fiches giving a general overview of the 25 Countries covered by the evaluation, Operational Programme Fiches which present an analysis of the coherence between ESF and SPSI OMC at the level of macro-interventions and target groups, Operational Programme Fiches based on a sample of Operational Programmes with the analysis illustrating the inter-relationship between the ESF and the SPSI OMC in terms of interventions and indicators, and Equal Project Fiches covering a sample of projects which show coherence with SPSI OMC; an on-line survey with managing authorities; case studies; and a stakeholder validation and learning workshop.

Results

The Final report can be downloaded here.