Our evidence
Our approach to evidence
Why we carry out research
Part of our role is to share people’s experiences with the Government, NHS, and local authorities. Our research helps us hear and understand people’s views.
The stories people share form a big part of our evidence – the local Healthwatch network supports over a million people to have their say on care and get the right advice and information every year. While many of these stories are negative, local Healthwatch also often hear more positive stories through their direct work with communities.
Our data sources
We mainly take a qualitative approach to data. Other organisations, including those across the NHS, already carry out large-scale data analysis, so our research aims to complement this quantitative approach.
Ensuring that people can share their stories and have their voices heard is at the core of what we do. The stories we hear about health and social care offer a human dimension and a different perspective to NHS performance data. Experiences mentioned by one or a few people still matter, and understanding those outliers is a major part of why qualitative approaches are used.
As well as the stories we hear from local Healthwatch, we use established social research methods that offer depth of insight into people’s experiences of health and social care services. We use methods that will let us share the public’s concerns quickly, and work with stakeholders to make improvements.
These include:
- Surveys
- Interviews
- Focus groups
- Deliberative events that are a blend of research and consultation
Collecting data
We’re constantly capturing data so we can respond to emerging themes, policy changes, and shifts in people’s priorities. This includes working to hear from traditionally excluded groups. Areas our data comes from include:
Local Healthwatch
Legally, local Healthwatch must collect local people’s views and experiences of services and share them with Healthwatch England. This is our main source of evidence.
They might gather views through events and outreach sessions, social media, online forms, research projects, email, telephone and post. They also build links with underrepresented groups in their communities, enriching their data further and creating unique, valuable insight.
This data is largely story-based, with people sharing their experiences in their own words. It may also gather statistical information about the type of services, demographic information, key themes and sentiment.
Our online form
People can share experiences with us directly. We target adverts on social media where we need insight about specific areas.
Our form lets people share experiences through free text boxes, but also gathers statistical data like the type of services, demographic information, and whether the story is positive or negative. We share this data with local Healthwatch to add to their insight.
Surveys and polling
We commission external polling and panel surveys if we need nationally representative samples, or are trying to identify differences between different groups’ experiences. We ask for “boost samples” or quotas of people from specific minority groups to ensure we have enough data to compare and contrast their experiences.
When an external agency can’t meet our needs – for example, if we’re looking for feedback from people with specific experiences – we also design our own surveys. These are “self-selecting” rather than nationally representative, meaning that we rely on whoever is willing to complete the survey.
Commissioned qualitative research
We commission qualitative research where we need to understand people’s experiences in more depth, particularly people who don't always have their voices heard.
Our surveys and commissioned surveys may not reach the most excluded groups in society. To address this, we usually work with local Healthwatch who have built links with those groups to gather the insight we need.
Talking about our data
We always explain:
- Our data sources
- How we collected data
- How many people we reached
- Any biases in the data
- How we’ve analysed the data
When talking about quantitative data, we use proportions, percentages, and actual figures.
We usually avoid quantifiers like “sometimes” or “rarely” when we talk about qualitative data, as this looks at a small number of people’s experiences in depth.
Other data sources and evidence
To support the conclusions we draw from the data described above, and make the strongest case for our recommendations, we use other data sources too, including:
- Freedom of Information (FOI) requests
- Other organisations’ data and findings
Sources of data that we don’t use and why
Our financial and staff resources are limited, so we don’t have the capacity to analyse large raw datasets that we would have to pay a large fee for access to, such as the NHS Hospital Episode Statistics.
We also don't use analyses of social media data in our research and insight pieces. The data doesn't contain demographic information, and it's unclear whether the information originated from England. We also found that it often doesn't provide useful patient experience data.
Evidence mentioned in press and media articles
In most circumstances, we won’t use secondary sources like blogs, media articles and programmes to support our data analysis and conclusions because we can’t be sure the information is accurate or unbiased. If there is no other source of information, we may use it with caveats about the data source.
How our evidence informs our work
Since we launched in 2013, we’ve shown how public feedback can help the health and care system understand what’s working, spot issues and make positive changes.
We choose specific areas to focus on through horizon scanning in health and social care policy, themes in our data, and conversations with stakeholders. We consider how to link the issues we hear about to opportunities to improve services, and whether we’re best placed to take the issue forward.
It's essential to look at all aspects of health and social care, not just those we hear about the most. So we take a targeted approach to our data analysis. Each month, we analyse data about a broad service area, such as the experience of health and social care for different groups of people in the most excluded communities, and cross-cutting issues.
We then use the insight to underpin our policy and communications work as we deliver our strategy.
This glossary explains the terms we use for data collection, usage and analysis.
Glossary
Analysis
Analysis is our interpretation of data, where we draw meaning from it. Analysis does not just mean crunching numbers, i.e. quantitative analysis. It can also mean looking for meanings and themes in people's spoken or written experiences, i.e. qualitative analysis.
Data
Our data is the information we hold about a topic. It can be qualitative (non-numerical) or quantitative (numerical).
Data saturation
This is the point where enough data has been collected and analysed to draw necessary conclusions, and any further data collection or analysis will not add more value to our insights. It is a critical indicator that a sample is adequate for the topic studied – that data collected have captured the diversity, depth, and nuances of the issues explored.
Coding/tagging
The process of systematically categorising excerpts in qualitative data to find themes and patterns.
Evidence
Evidence is the subset of our data that use to draw conclusions, including policy recommendations.
Feedback
This is what we hear from people on a topic.
Findings
These are the conclusions we draw from our analysis.
Insight
This is a small piece of analysis, providing some understanding of people's experiences of a particular topic.
Self-selecting
Where the sample is self-selected and there is no control or attempt to make it nationally representative. We use this to describe the results of surveys that we design and promote ourselves.
Sentiment
How people feel about the service they have experienced.
Solicited data
This is data gathered due to a specific request for information, e.g. in research or engagement projects.
Unsolicited data
This is data gathered via methods that permit people to give their feedback spontaneously. In Healthwatch terms, this is data collected from people who contact us of their own volition to share an experience of care – for example via a web form, social media or by people who contact Healthwatch to get advice and information.
