Evaluation

Evaluating how closely your service or process change meets its objectives

Our approach to evaluation is pragmatic, working with clients to develop a tailored strategy that assesses the extent to which a programme has achieved its desired outcomes and impacts, as well as noting unanticipated outcomes. Our evaluations enable meaningful learning from the process of introducing new services, models of care or quality improvements and inform service improvement and commissioning decisions. We regularly carry out both process and outcome evaluations.

 

What we do

Our evaluations are delivered using a variety of qualitative and quantitative techniques including:

  • Collating and analysing activity and epidemiological data
  • Interviewing stakeholders
  • Running engagement events such as focus groups and workshops
  • Developing, implementing and analysing online surveys
  • Suggest recommendations to inform future service delivery
  • Produce outputs including reports, presentations, web content and journal articles to disseminate findings

Using a combination of methods allows us to triangulate information from different sources to fully inform conclusions and recommendations.

Most evaluations will include several broad objectives:

  • Describing the nature of the programme
  • Reviewing the activity of the programme
  • Understanding how closely the processes used to deliver the project or programme matched those originally planned and identifying learning from this
  • Understanding the project or programme outcomes
  • Assessing satisfaction with the programme from the perspective of key stakeholders
  • Identifying strengths and areas for improvement
  • Producing materials to help share learning

Each evaluation will be underpinned by a bespoke framework, supported by a theory of change and/or logic model where appropriate. This allows us to plan from the outset the identification of stakeholders, data collection requirements and qualitative information collection requirements, and consider information governance issues. A quality assurance framework will be developed alongside this to ensure that we quality assure all our work throughout the lifetime of the evaluation. We will work closely with you to ensure we deliver to your requirements

Why choose SPH?

We have extensive experience over many years in working with health and care commissioners to evaluate projects and programmes. Some recent examples of our work are given below. Please get in touch via our contact page.

Case Studies

Fuel Poverty – an evaluation of the ‘Warm Homes’ programme for a local authority

A place partnership within a borough council received funding for residents living in areas of high deprivation with fuel poverty or fuel stress, with chronic health conditions exacerbated by living in a cold and/or damp home.  Eligible residents were offered home improvements including the installation of replacement gas boilers, loft insulation and/or cavity wall insulation.   We conducted an impact evaluation to identify whether the intervention was fully taken up, warmed the client's homes, improved their health and wellbeing, reduced household carbon emissions and whether it was cost-effective.  We carried out semi structured interviews with stakeholders including Council leads responsible for programme development and delivery, a GP leading on referring people to the programme and case managers involved in operational delivery.  These interviews demonstrated the scope for benefits provided by the Warm Homes programme in terms of individual health and comfort. Quantitative data identified potential benefits in terms of reducing health inequalities and contributing to a reduction in carbon emissions . Read about it in more detail here:  SPH case study_Warm Homes

An evaluation of a community organising pilot

SPH was commissioned by a County Council to evaluate a community organising initiative that had been operating across three electoral wards in a large local town.  The community organising project involved recruiting community leaders from local civic institutions to form a network.  The network engaged with their local communities to identify and prioritise issues impacting on health and wellbeing in the local area.  Actions to address the prioritised issues were developed and taken forward either by the network or by statutory agencies or both of these following a period of negotiation with community leaders.  The agreed actions were presented to the public at a specially convened event.  The County Council commissioned two external organisations to help establish the network and support its community organising activities. SPH completed a qualitative impact evaluation of the community organising project through a combination of face to face and telephone interviews with stakeholders and a document review.

Evaluation of the Discharge to Assess initiative (Home First) for a Clinical Commissioning Group

We were commissioned by a CCG to evaluate a ‘Discharge to Assess’ initiative aimed at reducing the secondary care inpatient length of stay of elderly frail people who no longer have any medical needs. Discharge to Assess schemes aim to safely support the transfer of elderly people back to their home and their short-term recuperation. Once people are at home and have recovered as much as possible of their day-to-day functioning they can be assessed, if necessary, for on-going home care or transfer to residential care. The evaluation SPH undertook was based on a mixed model of quantitative and qualitative research and helped: 
  • Provide a detailed understanding of all aspects of the initiative for commissioners 
  • Link the original strategic objectives of the initiative with the day-today experience of people providing the service in managerial and operational roles and the challenges they face 
  • Link the original strategic objectives with activity and cost of the initiative 
  • Recommend improvements and a plan for future evaluation 
The evaluation informed the commissioning of the service in the longer term and identified immediate actions needed to strengthen the service in order for it to play a significant role in easing winter pressures at the local hospital NHS Trust.   The Local Government Association website signposts to national guidance and some examples of local initiatives:  Home First / discharge to assess | Local Government Association  

An evaluation of continuing fitness to practice pilot for GPhC

The General Pharmaceutical Council implemented a new revalidation framework after an extensive period of research, testing and consultation. SPH supported the GPhC by evaluating their pilot of the new process.  Working with GPhC we developed a logic model and conducted a mixed methods evaluation  including phone interviews, an online survey, analysis of entries made into the pilot system, a focus group and desk-based research.  From analysing and presenting the wealth of information gathered in a clear and concise way we were able to give the GPhC valuable insights to help them in their next steps. The report is available here: consultation_on_revalidation_for_pharmacy_professionals_april_2017.pdf   

An evaluation of a cancer programme in a London borough

The programme promoted the early detection of cancer by improving population awareness of the signs and symptoms of cancer and to remove barriers to seeking help. The programme also sought to ensure that GPs had the knowledge and skills to make the most appropriate referrals and supported pharmacists to engage with customers to increase their awareness of cancer signs and symptoms and visit their GPs if appropriate. SPH used a combination of quantitative and qualitative methods to assess the overall effectiveness of the programme. These included analysis of routinely available cancer indicators along with locally available data, reviewing data collected by the programme such as survey responses, reviewing project documentation including costings, and conducting interviews and focus groups. These were collated to form a comprehensive and insightful evaluation report, which the customer used to support future priorities setting.

An evaluation of an end of life care education programme

SPH evaluated an education programme, targeted at staff within GP practices to improve end of life care. The evaluation included an assessment of the effectiveness and efficiency of the training visits through interviews and analysis of service data, an audit of the impact of the programme on practice, including changes in behaviour and the accuracy of end of life care registers, and an analysis of hospital admissions data. The evaluation found improvements in the skills, behaviours and competence of staff around end of life care in the period following the training programme.
More case studies >
Testimonials