Evaluation

Evaluating how closely your service or process change meets its objectives

Our approach to evaluation is pragmatic, working with clients to develop a tailored strategy that assesses the extent to which a programme has achieved its desired outcomes and impacts, as well as noting unanticipated outcomes. Our evaluations enable meaningful learning from the process of introducing new services, models of care or quality improvements and inform service improvement and commissioning decisions. We regularly carry out both process and outcome evaluations.

 

What we do

Our evaluations are delivered using a variety of qualitative and quantitative techniques including:

  • Collating and analysing activity and epidemiological data
  • Interviewing stakeholders
  • Running engagement events such as focus groups and workshops
  • Developing, implementing and analysing online surveys
  • Suggest recommendations to inform future service delivery
  • Produce outputs including reports, presentations, web content and journal articles to disseminate findings

Using a combination of methods allows us to triangulate information from different sources to fully inform conclusions and recommendations.

Most evaluations will include several broad objectives:

  • Describing the nature of the programme
  • Reviewing the activity of the programme
  • Understanding how closely the processes used to deliver the project or programme matched those originally planned and identifying learning from this
  • Understanding the project or programme outcomes
  • Assessing satisfaction with the programme from the perspective of key stakeholders
  • Identifying strengths and areas for improvement
  • Producing materials to help share learning

Each evaluation will be underpinned by a bespoke framework, supported by a theory of change and/or logic model where appropriate. This allows us to plan from the outset the identification of stakeholders, data collection requirements and qualitative information collection requirements, and consider information governance issues. A quality assurance framework will be developed alongside this to ensure that we quality assure all our work throughout the lifetime of the evaluation. We will work closely with you to ensure we deliver to your requirements

Why choose SPH?

We have extensive experience over many years in working with health and care commissioners to evaluate projects and programmes. Some recent examples of our work are given below. Please get in touch via our contact page.

Case Studies

An evaluation of a community organising pilot

SPH was commissioned by a County Council to evaluate a community organising initiative that had been operating across three electoral wards in a large local town.  The community organising project involved recruiting community leaders from local civic institutions to form a network.  The network engaged with their local communities to identify and prioritise issues impacting on health and wellbeing in the local area.  Actions to address the prioritised issues were developed and taken forward either by the network or by statutory agencies or both of these following a period of negotiation with community leaders.  The agreed actions were presented to the public at a specially convened event.  The County Council commissioned two external organisations to help establish the network and support its community organising activities. SPH completed a qualitative impact evaluation of the community organising project through a combination of face to face and telephone interviews with stakeholders and a document review.

Improving allocative value decision-making in the stroke pathway (STAR tool)

The Mid and South Essex stroke stewardship group wanted to better understand which of the interventions they delivered were high value and which were low value to inform decision-making on use of resources and investment. Arden & GEM’s Healthcare Solutions team provided training and support to enable the group to use the Socio-Technical Allocation of Resources (STAR) process in value based decision-making. As a result, the MSE stroke stewardship group has the insight needed to improve the allocation of their existing resources and improve outcomes for their patients. SPH were part of the project team, conducting a librarian review of economic evidence relating to cost-effectiveness of the stroke interventions to feed into the first workshop. This comprised a review of existing economic cost-effectiveness assessments (ideally by NICE as these are recognised as the guiding authority within the NHS). Where an economic assessment had not been undertaken then this was noted for further consideration. More detail here

Health Needs Assessment for Children & Young People

SPH carried out a Health Needs Assessment for a local authority that was recommissioning its Healthy Child Programme (HCP). The purpose was to review whether existing provision and service configuration of the HCP was meeting the needs of children and young people in the county, and to identify gaps and potential options for improvement. This is a preventative programme which includes Health Visiting and School Nursing Services, Family Nurse Partnership (FNP) Programme and healthy start vouchers and vitamins, linking with childhood immunisation services, community paediatrics, speech and language, services for children with a learning disability and complex needs and child and adolescent mental health services (CAMHS), among others. The HNA was carried out jointly by SPH and the local authority, with the epidemiological needs assessment and service description provided by the local authority. The report was compiled using a range of publicly available and locally collected epidemiological and service use data. In addition, SPH carried out a consultation of key corporate stakeholders as well as children and young people, their parents and carers. This included semi-structured interviews with key stakeholders in a range of organisations, a survey of a wider range of staff who work with children and young people and a survey of children and young people, their parents and cares which was distributed through the services.

Evaluation of the Discharge to Assess initiative ‘Home First’ for a Clinical Commissioning Group

SPH was commissioned by a CCG to evaluate a ‘Discharge to Assess’ initiative aimed at reducing the secondary care inpatient length of stay of elderly frail people who no longer have any medical needs. Home First is the name given to these Discharge to Assess schemes that aim to safely support the transfer and short-term recuperation of elderly people home. Once people are at home and have recovered as much as possible of their day to day functioning they can be assessed, if necessary, for on-going home care or transfer to residential care. The evaluation SPH undertook was based on a mixed model of quantitative and qualitative measures. It provided a number of functions including:
  • Providing a detailed understanding of all aspects of the initiative for commissioners
  • Linking the original strategic objectives of the initiative with the day to day experience of people providing the service in managerial and operational roles and the challenges they face
  • Linking the original strategic objectives with activity and cost of the initiative
  • Recommending improvements and a plan for future evaluation
The key outcome of the evaluation was to inform the commissioning of the service in the longer term and identify immediate actions needed to strengthen the service in the short term in order for it to play a significant role in easing winter pressures at the local hospital NHS Trust. For a more in depth look at the rationale behind Home First schemes see this article published by The Guardian on 10 October 2017: https://www.theguardian.com/social-care-network/2017/oct/10/home-first-model-aims-to-stop-patients-being-stranded-on-hospital-wards The Local Government Association website signposts to national guidance and some examples of local initiatives: https://www.local.gov.uk/our-support/our-improvement-offer/care-and-health-improvement/systems-resilience/overall-approach/discharge-to-assess

An evaluation of continuing fitness to practice pilot for GPhC

The GPhC recently implemented a new revalidation framework. After an extensive period of research and testing, they consulted on the proposed new framework. SPH supported the GPhC by evaluating their pilot of the new process.  After developing and testing our evaluation logic model with the customer, we used a mixed methods approach including phone interviews, an online survey, analysis of entries made into the pilot system, a focus group and desk-based research.  From analysing and presenting the wealth of information gathered in a clear and concise way we were able to give the GPhC valuable insights to help them in their next steps. More background and the report are available here: Continuing Fitness to Practice Pilot - evaluation  

An evaluation of a cancer programme in a London borough

The programme promoted the early detection of cancer by improving population awareness of the signs and symptoms of cancer and to remove barriers to seeking help. The programme also sought to ensure that GPs had the knowledge and skills to make the most appropriate referrals and supported pharmacists to engage with customers to increase their awareness of cancer signs and symptoms and visit their GPs if appropriate. SPH used a combination of quantitative and qualitative methods to assess the overall effectiveness of the programme. These included analysis of routinely available cancer indicators along with locally available data, reviewing data collected by the programme such as survey responses, reviewing project documentation including costings, and conducting interviews and focus groups. These were collated to form a comprehensive and insightful evaluation report, which the customer used to support future priorities setting.

An evaluation of an end of life care education programme

SPH evaluated an education programme, targeted at staff within GP practices to improve end of life care. The evaluation included an assessment of the effectiveness and efficiency of the training visits through interviews and analysis of service data, an audit of the impact of the programme on practice, including changes in behaviour and the accuracy of end of life care registers, and an analysis of hospital admissions data. The evaluation found improvements in the skills, behaviours and competence of staff around end of life care in the period following the training programme.
More case studies >
Testimonials