Systematic reviews are well recognized as a critical component of evidence-based healthcare and research. They plays an essential role in shaping the trajectory of evidence and hence, scientific knowledge. ESCA embraces this approach; enabling and conducting systematic reviews is central to our mission.

This section offers a practical overview of the steps. We will proceed chronologically, addressing some fundamental steps necessary for conducting systematic reviews and meta analysis. It is important to note that the length of these sections does not reflect the actual time required for their conduct in practice. For example, based on our experience, statistical synthesis (analyses) account for, at most, 10% of the total time invested in a systematic review, far less than the effort devoted to the other steps, but it requires us to delve into that with the necessary details in a separate guide under the name meta analysis. Defining well-tailored criteria, systematically searching for studies, selecting them from a large pool of others, and accurately extracting the data are critical yet relatively more time-demanding tasks.

Some steps are already covered, while others will be added soon. The methodological background offered here is designed for non-experts and those new to evidence synthesis. Other method-driven approaches, such as scoping reviews and umbrella reviews, shared some steps i.e., a predefined protocol with systematic searches but are not the focus of this section.

Start by considering your research question. How specific is it? What is the primary purpose of your review? As an example a typical aim of a systematic review is to aggregating evidence on a treatment or intervention when existing studies report conflicting results: some showing benefit, others suggesting harm

Frameworks such as PICO (Patient/population; Intervention; Comparison; Outcome) or FINER (2013) (Feasible, Interesting, Novel, Ethical, and Relevant) are available to researchers and can assist them in ensuring that their research question has covered all relevant components. Using such framework can help refine and evaluate your question before moving forward (10).

Next, do a topic search. Confirm the availability of (high-quality) primary studies that are candidates for inclusion in your review.  Then check whether previous systematic reviews have already addressed this question. If no such reviews exist—or if existing ones are outdated, methodologically limited, or fail to address important secondary questions—there may be justification for a new review. Tools like AMSTAR 2 can help you critically appraise the methodological quality of existing reviews.

A good starting point is to perform a topic search in databases like PubMed and the International Prospective Register of Systematic Reviews (PROSPERO). These searches can help you clarify your research focus and ensure that your proposed review is feasible, addresses an interesting gap in the evidence.

The PICO framework (Population, Intervention, Control, Outcome) is useful for defining the inclusion and exclusion criteria and aligning them with research objectives. These criteria should be transparent and reproducible to enhance credibility.

Don’t forget:

An additional and important component of the eligibility criteria involves clearly defining which types of study designs will be included (e.g., randomized controlled trials [RCTs], observational studies). In evidence-based medicine, RCTs: where participants are randomly assigned to intervention or control groups—are often considered the gold standard due to their reduced risk of bias and confounding. However, limiting a systematic review to only RCTs is not always appropriate or feasible.

Observational studies, such as cohort and case-control designs, can provide valuable information and are increasingly included in systematic reviews. In some cases, the research question itself is inherently observational in nature and requires the inclusion of non-randomized evidence. This is particularly relevant when addressing questions about rare exposures, long-term outcomes, or real-world effectiveness, where RCTs may be impossible, unavailable or scarce.

It is also important to recognize that different study designs often require different approaches to data extraction, statistical analysis, and interpretation. Therefore, explicitly defining eligible study designs at the outset helps ensure consistency in the review process and informs the choice of appropriate analytical methods.

Another important consideration in defining eligibility criteria is the cultural and linguistic scope of included studies, as well as whether to include unpublished or grey literature. This is particularly relevant given that much of the available research is based on WEIRD populations (Western, Educated, Industrialized, Rich, and Democratic societies), which may limit the generalizability of findings to other cultural contexts (Henrich, Heine, and Norenzayan, 2010).

In disciplines such as the social sciences, cultural factors can significantly influence the expression and interpretation of studied phenomena. Therefore, excluding studies based on language alone may systematically bias the evidence base. While translation can pose logistical challenges, the increasing availability of translation tools makes it more feasible to include non-English-language studies. Researchers should weigh the trade-offs and consider whether restricting to English-language publications is justified by the research question and resources available.

• What is grey literature?
• Grey literature can be defined as all types of research materials that have not been made available through conventional publication formats. This includes research reports, preprints, working papers, or conference contributions. Dissertations also often count as grey literature, although many of them are indexed in electronic bibliographic databases today (Schöpfel and Rasuli 2018).

Finalizing the inclusion and exclusion criteria as part of the predefined protocol is essential for ensuring consistency and methodological rigor, and it lays the foundation for the systematic search and subsequent review steps

Literature Search for Systematic Reviews

Carrying out a systematic review is an elaborate process and requires knowledge of searching techniques in several databases.  A good literature search requires:

• selection of databases
• knowledge of search techniques in every database
• setting up a good search strategy
• document the search and search history in a logbook
  

Search support for Amsterdam UMC researchers

Information specialists from the Medical Library are experts in literature searches. The library services are a collaboration between the University Libraries and the Amsterdam UMC Library. They offer access to a very large collection of journals and databases.

The information specialists can conduct the literature search of your systematic review. This includes the selection of databases, creation of comprehensive search strategies, removal of duplicates from search results, and can provide you of the final search results. More information about their support can be found at:

Amsterdam UMC-Medical Library

or directly the intake form 

Selecting relevant studies typically involves screening titles and abstracts based on predefined eligibility criteria. This process is often supported by tools such as Rayyan—a web-based platform that enables blinded and collaborative screening. Study selection is increasingly supplemented by AI tools designed to enhance efficiency. This section will be completed soon.

Extracting and managing data involves systematically collecting key information from included studies, such as study design, population characteristics, and outcomes. It may also involve focused extraction of relevant numerical data for meta-analysis. Increasingly, AI-powered tools are being explored to support or automate parts of this process, helping reduce manual workload and improve consistency. This section will be completed soon.

Assessing the methodological quality or risk of bias in included studies is a key step. Several tools have been developed by expert groups to support this process. ESCA will provide guidance to help users identify the appropriate risk of bias or quality assessment tool based on their specific research question and study design. This content will appear here soon.

The related content is explained in detail in the meta-analysis section of ESCA.

The related content is explained in detail in the GRADE section of ESCA.

We would like to emphasize that the conclusions of a systematic review depend heavily on the quality and quantity of the studies it includes. If the available studies are scarce, biased, or methodologically flawed, the review’s conclusions will be equally uncertain or unreliable. While assessing study quality or risk of bias can help interpret the findings more cautiously, systematic reviews cannot compensate for a predominance of low-quality evidence or overall low certainty. In such cases, the main conclusion may simply be that the evidence is inconclusive or insufficient: highlighting the need for better-designed, higher-quality studies. Even this outcome can play a valuable role by guiding future research efforts.