Skip to Main Content
It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.

Systematic Reviews

Evaluating the results

After a thorough search, the next phase of the systematic review is the analysis phase.  This is the phase references will be screened and graded.

Some eligibility criteria should have been developed in the planning phase.  Additional inclusion and exclusion criteria can be added, and used to select which references to keep for the review.

Inclusion and Exclusion Criteria

Inclusion criteria are the elements of an article that must be present in order for it to be eligible for inclusion in the systematic review.

Exclusion criteria are the elements of an article that disqualify the study from inclusion.

Examples of common elements:

  • Publication date range
    You may only want to look at new research conducted in the last 10 years.
  • Population
    The population may need to be a certain age or experienced a particular condition.
  • Geographic Location
    The studies may need to take place in a certain state, country, or region.
  • Reported Outcomes
    Studies may be excluded based on how they were reported (using objective measures vs self-reported)
  • Setting
    Study participants may need to be in a particular environment (hospital, in patient, school, etc)
  • Study Design
  • Type of publication
    Randomized controlled trials, cohort studies, case reports, etc
  • Publication Language
  • Sample size

Appraise the Literature

The first review of the results can begin with screening the title and abstract.  Begin by looking at the results and removing any duplicates.  It is important to take notes of citation numbers during this process.  For example, if you start with 1,000 articles, and 300 are duplicates, you need to note this to explain why a large number of citations were removed.  The next step is to scan the article titles and remove any that are clearly irrelevant.  The remaining articles will be screened by two reviewers.

Abstracts should be reviewed next using the eligibility criteria for inclusion.  The number of studies excluded from this part should also be documented.  The full text of relevant citations should be pulled next for further screening.  Use the library's resources to either download the full text or request articles.  A team of reviewers should analyze the full text to minimize biases and reduce the risk of human error.  One team member should be assigned as a tie breaker, in case there are disagreements on a particular citation.  Documentation of full text review should also take place, including reasons why a study was removed or kept.

Critical Appraisal

Critical appraisal of the articles will occur after finalizing what citations to include.  Reviewers will evaluate the validity of the research and it's relevance to the review question.  Generally, the study design of the included studies are noted.  If a valid quality assessment tool is used, it will be used for the critical appraisal.  Results will be tabulated and summarized.

Extract the data

The work involved extracting the data should be done with a statistician.  Generally the process involves a plan for synthesis, a list of data elements, a method to collect the data, and summarizing the data.

Working with a Librarian

As a team member, the librarian can review a sample of excluded citations to make sure no relevant studies were removed by mistake, serve as the tie breaker for citation selection and keep the review team on task.  The librarian will not be involved in the data synthesis part, but can provide useful resources and tools on the process.

If the librarian is serving as an advisory role, assistance and education is provided on the following items.

  • Educate team members on article assessments tools
  • Educate team members on common eligibility criteria
  • Educate team members on article screening tools
  • Provide resources on data synthesis