Systematic Reviews

This page is split into the following sections:

• What is a systematic review?
• What are the characteristics of different types of review?
• What assistance can KLS provide for a systematic or other type of review?
• What type of review should I do?

What is a systematic review?

A review that uses explicit, systematic methods to collate and synthesise findings of studies that address a clearly formulated question [Cochrane Handbook for Systematic Reviews of Interventions].

The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement was developed to encourage transparent and complete reporting of systematic reviews, and has been recently updated [PRISMA 2020 explanation and elaboration].

The PRISMA 2020 statement includes a checklist of 27 items to help authors report their systematic reviews. A PRISMA flow diagram should be included to show the progress of citations through the review.

The stages of conducting a systematic review are shown in the diagram below.

Note: the diagram and icons above are used with kind permission of the Evidence and Evaluation team in OHID, DHSC.

 

What are the characteristics of different types of review?

Characteristics of different types of review

	Protocol	Systematic search strategy	Screening & data extraction	Critical appraisal	Characteristics of studies	Evidence synthesis	Meta-analysis	Visual synthesis
Systematic review	Yes	Yes	Yes	Yes	Yes	Yes	Optional	No
Umbrella review	Yes	Yes	Yes	Yes	Yes	Yes	Optional	No
Rapid review	Yes	Yes	Yes	Yes	Yes	Yes	Optional	No
Scoping review	Yes	Yes	Yes	Optional	Yes	No	No	No
Mapping review	Yes	Yes	Yes	Optional	Yes	No	No	Yes

Table compiled by the Evidence and Evaluation team in OHID, DHSC

 

Systematic

Systematic reviews should have a narrowly defined question. Study data is summarised using narrative text and tables; sometimes systematic reviews include a meta-analysis, a statistical technique that provides an overall summary measure of effect. The risk of bias is minimised by using critical appraisal methods. Screening and other parts of the process such as data extraction are usually carried out in duplicate by two reviewers. The methodology must be reported transparently.

Full systematic reviews can be very time and resource intensive.

Examples: Hallmaier-Wacker L et al. Incidence and aetiology of infant Gram-negative bacteraemia and meningitis: systematic review and meta-analysis. Arch Dis Child 2022 Jun 16

Danielli S et al. Systematic review into city interventions to address obesity. eClinicalMedicine Volume 32, February 2021

Chu D et al. Physical distancing, face masks, and eye protection to prevent person-to-person transmission of SARS-CoV-2 and COVID-19: a systematic review and meta-analysis. The Lancet 2020, 395, 1973-1987

Wolfenden L et al. Strategies for enhancing the implementation of school‐based policies or practices targeting diet, physical activity, obesity, tobacco or alcohol use. Cochrane Database of Systematic Reviews. 29 August 2022 (issue 8).

 

Scoping

Scoping reviews are concerned with the ‘state of the evidence’; they are appropriate when there is a high volume of literature available or to identify gaps in the evidence. They are similar in process to a mapping review, but are based on a topic, rather than on a specific review question. 

Note that mapping and scoping reviews are not useful for questions such as ‘what works…’, as the aim is to categorise the characteristics of the studies, not synthesise their results, and critical appraisal of included studies is therefore optional.

Examples: O’Toole F et al. What evidence is available smoking related diseases on beyond those acknowledged by Public Health England? A Scoping Review. Public Health England 2019.

Borschmann R et al. The health of adolescents in detention: a global scoping review. The Lancet Public Health 2020, 5(2), E114-E126

Belita E et al. A scoping review of strategies to support public health recovery in the transition to a “new normal” in the age of COVID-19. BMC Public Health 2022, 22

 

Mapping

Mapping reviews are used to map out and categorise the literature, and may be used when there is a large, varied evidence base. They are similar in process to a scoping review, but are based on a specific review question, rather than a topic.

Note that mapping and scoping reviews are not useful for questions such as ‘what works…’, as the aim is to categorise the characteristics of the studies, not synthesise their results, and critical appraisal of included studies is therefore optional.

The main output of a mapping review is usually an evidence gap map incorporating included studies as a visual matrix.

Examples: Using mobile technologies to support the training of community health  workers in low-income and middle-income countries: mapping the evidence. BMJ Global Health 2019;4:e001421

Evidence and gap map of studies assessing the effectiveness of interventions for people with disabilities in low-and middle-income countries. UNICEF 2020.

An Evidence Map of Research Linking Dietary Sugars to Potentially Related Health Outcomes. Current Developments in Nutrition, Volume 2, Issue 11, November 2018

 

Rapid

In 2020, as a result of the COVID-19 pandemic, the number of rapid reviews published increased hugely.

More than 30 different terms have been used to describe a rapid review and there is no current standardisation of terms or methods [An international survey and modified Delphi approach revealed numerous rapid review methods].

One definition that has been proposed is: “a type of knowledge synthesis in which steps of a systematic review are streamlined, accelerated or omitted” [Rapid reviews to strengthen health policy and systems: a practical guide 2017].

Conducting a rapid review is useful when policy makers’ need timely access to information in order to make a decision and cannot wait many months for an answer.

The depth and breadth of a rapid review is reduced by limiting specific elements of a systematic review but retaining the rigour. These limits could be:

• searching a smaller number of databases or restricting grey literature searches
• including only one study type, setting, population group or outcome
• screening only a small percentage (e.g. 10%) in duplicate
• shorter timeframe
• performing critical appraisal without the use of a formal tool
• limiting analysis and interpretation
• updating a previous systematic review

Examples: Duval D et al. Long distance airborne transmission of SARS-CoV-2: rapid systematic review. BMJ 2022;377:e068743

Noone C et al. Video calls for reducing social isolation and loneliness in older people: a rapid review. Cochrane Database Syst Rev. 2020 May; 2020(5): CD013632

Rios P et al. Preventing the transmission of COVID-19 and other coronaviruses in older adults aged 60 years and above living in long-term care: a rapid review. Syst Rev. 2020; 9: 218.

 

Umbrella

An umbrella review or “review of reviews” identifies multiple systematic reviews on related research questions and analyses their results across agreed outcomes. Umbrella reviews normally address a broad scope.

Meta-analysis may be performed in order to provide an overall summary measure of effect.

Umbrella reviews can only capture evidence that has already been examined in an existing systematic review, so any new evidence would not be picked up. There are some limitations to these reviews, such as the comprehensiveness of available information provided about the included primary studies.

Examples: Shah N et al. National or population level interventions addressing the social determinants of mental health – an umbrella review. BMC Public Health 2021 Nov 18;21(1):2118

Shi X et al. Environmental risk factors for non-Hodgkin’s lymphoma: umbrella review and comparison of meta-analyses of summary and individual participant data. BMJ Medicine 2022;1:e000184

Abu-Odah H et al. Identifying barriers and facilitators of translating research evidence into clinical practice: A systematic review of reviews. Health Soc Care Community 2022 Jul 1

 

What assistance can KLS provide for a systematic or other type of review?

It is essential that a member of KLS is involved in your review team as early as possible in order to discuss the methodology and process of conducting a review. Any type of systematic review needs to have a well-designed and comprehensive search strategy behind it – if you do not retrieve some of the relevant studies available, the review results will be limited and potentially biased.  Designing a good review takes time, including developing and testing search strategies, which can be an iterative process. Please contact us to discuss, or see our literature search request page. 

 

Protocol

This describes the rationale and planned methods for your review. It should be written before your review begins and followed when you conduct the review. Guidance on writing a protocol is available in PRISMA for systematic review protocols (PRISMA-P).

You should register your systematic review in the PROSPERO registry – this should be done before data extraction begins [Planning a systematic review? Think protocols].

 

Literature searching

KLS can provide complex literature searches to help in the production of a systematic review. This involves:

• conducting preliminary scoping searches to find similar reviews (to reduce duplication of effort)
• developing the search strategy
• translating the strategy for other databases
• running the searches on a variety of sources
• de-duplicating citations
• providing the results in Endnote or another suitable format
• citation or similar article searching can also be provided, using sources such as Web of Science, Scopus, PubMed and Google Scholar

 

Screening

In some instances, KLS can also provide help with screening the search results – this can be agreed on a case by case basis.

 

Software tools

We can provide training and guidance on use of Endnote, Rayyan or EPPI-Reviewer, if appropriate.

 

Write-up

KLS can write up the methods for your review and help with production of the PRISMA flow diagram.

 

What type of review should I do?