Screening search results

Once you have completed every component of your search and exported all of your search results, you will apply the inclusion/exclusion criteria to your entire set of search results. This is a significant part of the knowledge synthesis process and it will require screening hundreds or thousands of your results. 

Managing the screening process

There are several ways of conducting your screening process.

If the results list is quite small, you can potentially manage the screening in your citation management software (Zotero) and a spreadsheet. You would need to set up folders to manage moving records through the screening process and spreadsheets to keep track of decisions. You need to keep track of your numbers throughout the process. 

For a larger review, you will need to utilize a screening tool. Please see "Utilize a Screening Tool." These tools will provide structure for moving your records through the process and will help manage inter-reviewer conflict resolution. 

Piloting the process

Prior to starting the screening process, we suggest taking a selection of 25-50 of your results and review them with your team against the inclusion/exclusion criteria to ensure agreement. It may be necessary to edit/add/remove some of your inclusion/exclusion criteria in order to be more specific or to clarify your intent. 

Regardless of the application or tool you choose to conduct this process, you will need to ensure you keep track of your dataset as it moves through this process. Please see "Reporting your Results with PRISMA."

1. Record the total number of studies retrieved from each source - databases, grey literature, hand searching/citation chaining, etc...
2. Using either a citation management tool or a knowledge synthesis screening tool, remove duplicate records and record the number you have removed. 
3. At each step of the screening process you must keep track of the number of articles you have removed and the number being carried forward to the next stage. 

Managing Conflicts

Each study is screened by two people, and if they make the same decision (Yes or Maybe – include; No – reject) then the study is moved into an Include or Exclude group. If different decisions are made by the screeners - “conflicts” - then a final consensus decision has to be made to move the study into the correct group. Having two screeners at each stage minimizes risk of bias, but is not always feasible (especially for students). If you are working as a team, before you start screening, you must have a shared understanding of what counts as relevant. Otherwise you will get constant conflicts. There will always be some conflicts, but normally only a few per cent.

In the first stage of screening, at least two reviewers from the review team will independently scan titles and abstracts of articles that were retrieved from your search, and make decisions whether to include or exclude articles. To do this in a streamlined, unbiased, and method-driven way, reviewers should adhere to the pre-defined inclusion/exclusion criteria.

Keep these tips in mind during the first stage of screening:

1. Reviewers should be overly-inclusive at this stage; a maybe is always a yes at this point
2. Screen for outcomes with caution if at all; they're poorly reported in the title and abstract. Save those for full-text!
3. Ask yourself: Is there enough information in the title and abstract to exclude this study at this stage, with absolute certainty?

The second level of screening is a more thorough, detailed process in which the articles that were included during the first stage of screening are read in full-text. Similar to the first-level of screening, this is done independently by at least two reviewers from the review team, and the eligibility criteria that was used as a guideline for the first-level of screening is largely the same. However the second level of screening differs in these important ways:

1. The reason(s) for exclusion must be recorded and reported
2. You may choose to screen for outcomes. Ask yourself: does the study report on the outcome(s) you're interested in?
3. Although the eligibility criteria is the same as the earlier screening, it will require additional detail (clarifying questions may arise during the first stage of screening)