This page is split into the following sections:
- What is evidence synthesis?
- What is the difference between systematic methodology and rapid systematic methodology?
- What are the characteristics of different types of review?
- What assistance can KLS provide for a systematic or other type of review?
What is evidence synthesis?
Evidence synthesis is the process of bringing together information and knowledge from a range of sources and disciplines to inform debates and decisions on specific issues [Royal Society – Evidence synthesis: a statement of principles].
The main types of evidence synthesis products are systematic reviews, rapid reviews, mapping reviews, scoping reviews and umbrella reviews. They all follow explicit, systematic methods to collate and synthesise findings of studies that address a clearly formulated question [Cochrane Handbook for Systematic Reviews of Interventions] [JBI Manual for Evidence Synthesis].
The main stages of the evidence synthesis process are shown in the diagram below.
What is the difference between systematic methodology and rapid systematic methodology?
In a systematic review, the screening, data extraction, critical appraisal and evidence synthesis stages are done independently by 2 reviewers (‘in duplicate’), with discrepancies being resolved by discussion between the 2 reviewers, or with involvement of a third reviewer.
Systematic reviews are the gold standard in evidence synthesis, showing a high level of rigour and reduced risk of errors thanks to the duplicate process. However, they are time-consuming and resource-intensive, and it can take up to one or 2 years to conduct a systematic review.
Rapid systematic methodologies use modified systematic review methods to accelerate the review process in a resource efficient manner while maintaining systematic, transparent and reproducible methods to ensure integrity. [Updated recommendations for the Cochrane rapid review methods guidance for rapid reviews of effectiveness].[ Rapid reviews to strengthen health policy and systems: a practical guide].
Examples of rapid systematic methods that can be used to accelerate the process could be:
- searching a smaller number of databases or restricting grey literature searches
- including only one study type, setting, population group or outcome
- screening only a small percentage (e.g. 10% or 20%) in duplicate
- shorter timeframe
- limiting analysis and interpretation
- updating a previous systematic review
Rapid systematic methodologies can be applied to systematic reviews (‘rapid systematic review’ or ‘rapid reviews’), mapping reviews (‘rapid mapping reviews’), scoping reviews (‘rapid scoping reviews’) and umbrella reviews (‘rapid umbrella reviews’).
What are the characteristics of different types of review?
| Evidence synthesis |
||||||||
|---|---|---|---|---|---|---|---|---|
| Protocol | Systematic search strategy | Screening | Data extraction | Critical Appraisal | Narrative | Meta-analysis | Visual synthesis | |
| Systematic review | Yes | Yes | Yes | Yes | Yes | Yes | Optional | No |
| Scoping review | Yes | Yes | Yes | Yes | Optional | Optional | No | Optional |
| Mapping review | Yes | Yes | Yes | Yes | Optional | Optional | No | Yes |
| Umbrella review | Yes | Yes | Yes | Yes | Yes | Yes | Optional | No |
Rapid systematic methodologies can be applied to all the above review types.
Systematic reviews
Systematic reviews should have a defined review question and clearly defined eligibility criteria, usually using the PICO (population, intervention or exposure, comparator, outcome) framework. Study data is summarised using narrative text and tables; sometimes systematic reviews include a meta-analysis, a statistical technique that provides an overall summary measure of effect.
The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement was developed to encourage transparent and complete reporting of systematic reviews, and has been recently updated [PRISMA 2020 explanation and elaboration].
The PRISMA 2020 statement includes a checklist of 27 items to help authors report their systematic reviews. A PRISMA flow diagram should be included to show the progress of citations through the review.
Rapid systematic methodologies can be applied to systematic reviews (‘rapid systematic reviews’ or ‘rapid reviews’).
Examples of systematic review:
- Hallmaier-Wacker L et al. Incidence and aetiology of infant Gram-negative bacteraemia and meningitis: systematic review and meta-analysis. Arch Dis Child, 2022
- Chu D et al. Physical distancing, face masks, and eye protection to prevent person-to-person transmission of SARS-CoV-2 and COVID-19: a systematic review and meta-analysis. The Lancet, 2020
- Wolfenden L et al. Strategies for enhancing the implementation of school‐based policies or practices targeting diet, physical activity, obesity, tobacco or alcohol use. Cochrane Database of Systematic Reviews, 2022
Examples of rapid systematic review:
- Duval D et al. Long distance airborne transmission of SARS-CoV-2: rapid systematic review. BMJ, 2022
- Noone C et al. Video calls for reducing social isolation and loneliness in older people: a rapid review. Cochrane Database Syst Rev, 2020
- Rios P et al. Preventing the transmission of COVID-19 and other coronaviruses in older adults aged 60 years and above living in long-term care: a rapid review. Syst Rev, 2020.
Scoping reviews
Scoping reviews are concerned with the ‘state of the evidence’; they are appropriate when presenting an in-depth analysis about the evidence and its content on a broad review question. They are similar in process to mapping reviews, but are based on a topic, rather than on a specific review question, and tend to follow the PCC (Participant, Context, Concept) framework rather than the PICO framework.
Note that scoping reviews are not useful for questions such as ‘what works…’, as the aim is to explore a topic, field, concept or issue to inform future research (where and how) or policy development by clarifying key concepts, definitions or methods. [Mapping reviews, scoping reviews, and evidence and gap maps (EGMs): the same but different— the “Big Picture” review family]
Scoping reviews tend to be exploratory and may involve refining the process of searching, data extraction and analysis as the review progress. They tend to have in-depth data extraction, and synthesis can include qualitative analysis, narrative synthesis and/or maps and tables. The critical appraisal of included studies is optional.
The PRISMA Extension for Scoping Reviews (PRISMA-ScR) can be used for transparent and complete reporting of scoping reviews. [PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation]
Rapid systematic methodologies can be applied to scoping reviews (‘rapid scoping reviews’).
Examples of scoping reviews:
- O’Toole F et al. What evidence is available on smoking-related diseases beyond those acknowledged by Public Health England? A Scoping Review. Public Health England, 2019
- Borschmann R et al. The health of adolescents in detention: a global scoping review. The Lancet Public Health, 2020
- Belita E et al. A scoping review of strategies to support public health recovery in thetransition to a “new normal” in the age of COVID-19. BMC Public Health, 2022
Mapping reviews
Mapping reviews are used to map out and categorise the literature, and may be used for broad questions covering a range of interventions, outcomes, and/or populations. They are similar in process to a scoping review, but are based on a specific review question, rather than a topic, and tend to follow the PICO framework.
Note that mapping reviews are not useful for questions such as ‘what works…’, as the aim is to present information about where the evidence is, usually through a visual summary known as an evidence gap map, to inform research priorities, research funding, and set a strategic agenda. [Mapping reviews, scoping reviews, and evidence and gap maps (EGMs): the same but different— the “Big Picture” review family]
Unlike scoping reviews, mapping reviews should follow a predefined framework of the ‘coding categories’ that will be used for the evidence gap map. Apart from the coding categories, data extraction tends to be more limited than for a scoping review. Critical appraisal of included studies is optional, although when conducted it can be represented on the map. [Guidance for producing a Campbell evidence and gap map]
The PRISMA Extension for Scoping Reviews (PRISMA-ScR) can also be used for transparent and complete reporting of mapping reviews. [PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation (Annals of Internal Medicine)]
Rapid methodologies can be applied to mapping reviews (‘rapid mapping reviews’).
Examples of mapping reviews:
- Winters N et al. Using mobile technologies to support the training of community health workers in low-income and middle-income countries: mapping the evidence. BMJ Global Health, 2019
- Saran A et al. Evidence and gap map of studies assessing the effectiveness of interventions for people with disabilities in low-and middle-income countries. Campbell Systematic Reviews, 2020
Examples of rapid mapping reviews:
- Duval D et al. Non-pharmaceutical interventions to reduce COVID-19 transmission in the UK: a rapid mapping review and interactive evidence gap map. Journal of Public Health, 2024
- Bosworth M et al. Health equity impact of climate change: a rapid mapping review. UKHSA Evidence Review collection page (GOV.UK), 2024
Umbrella review
An umbrella review or “review of reviews” identifies multiple systematic reviews on related research questions and analyses their results across agreed outcomes. Umbrella reviews normally address a broad scope.
Meta-analysis may be performed in order to provide an overall summary measure of effect.
Umbrella reviews can only capture evidence that has already been examined in an existing systematic review, so any new evidence would not be included. There are some limitations to these reviews, such as the comprehensiveness of available information provided about the included primary studies.
Examples:
- Shah N et al. National or population level interventions addressing the social determinants of mental health – an umbrella review. BMC Public Health, 2021
- Shi X et al. Environmental risk factors for non-Hodgkin’s lymphoma: umbrella review and comparison of meta-analyses of summary and individual participant data. BMJ Medicine, 2022
- Abu-Odah H et al. Identifying barriers and facilitators of translating research evidence into clinical practice: A systematic review of reviews. Health Soc Care Community, 2022
What assistance can KLS provide for a systematic or other type of review?
It is essential that a member of KLS is involved in your review team as early as possible in order to discuss the methodology and process of conducting a review. Any type of systematic review needs to have a well-designed and comprehensive search strategy behind it – if you do not retrieve some of the relevant studies available, the review results will be limited and potentially biased.
Designing a good review takes time, including developing and testing search strategies, which can be an iterative process. Please contact us to discuss, or see our literature search request page.
UKHSA staff can also receive support to conduct rapid reviews, see our Request a rapid review page
Protocol
This describes the rationale and planned methods for your review. It should be written before your review begins and followed when you conduct the review. Guidance on writing a protocol is available in PRISMA for systematic review protocols (PRISMA-P).
You should register your systematic review in the PROSPERO registry – this should be done before data extraction begins [Planning a systematic review? Think protocols].
Literature searching
KLS can provide complex literature searches to help in the production of a systematic review. This involves:
- conducting preliminary scoping searches to find similar reviews (to reduce duplication of effort)
- developing the search strategy
- translating the strategy for other databases
- running the searches on a variety of sources
- de-duplicating citations
- providing the results in Endnote or another suitable format
- citation or similar article searching can also be provided, using sources such as Web of Science, Scopus, PubMed and Google Scholar
Screening
In some instances, KLS can also provide help with screening the search results – this can be agreed on a case by case basis.
Software tools
We can provide training and guidance on use of Endnote or EPPI-Reviewer, if appropriate.
Write-up
KLS can write up the methods for your review and help with production of the PRISMA flow diagram.
