Review Guidance

We have conducted and published several reviews at the eHealth Observatory. Below is some guidance on conducting reviews based on published literature and our experiences with tools/approaches that have worked well for us in our past reviews.

Types of Reviews

A systematic review is a structured, comprehensive synthesis of published evidence to answer a question on a particular topic. It generally follows a systematic approach consisting of a literature search, selection of relevant publications, data extraction, quality assessment, synthesis, and reporting. Two variations of the systematic review are the scoping review and realist review which follow the same general approach but have slightly different aims.

• Scoping Review: Is designed to explore what evidence exists on a topic but doesn't necessarily synthesize all the evidence. An example is "Use of Information Technology in Medication Reconciliation: A Scoping Review" in which the aim was to determine how IT has been used to support medication reconciliation.
• Realist Review: Seeks to understand causality and has an explanatory focus. This type of review combines theoretical understanding with empirical evidence. An example is "Impact of Electronic Medical Record on Physician Practice in Office Settings: A Systematic Review" which not only determined positive or negative impact, but also identified contributing factors.

Review Plan

A review is essentially a project that needs to be planned carefully. There are several steps involved:

The Review Protocol

Regardless of the type of review being undertaken, a helpful tool is the review protocol which should be developed prior to beginning the review. It acts as a roadmap for the review team to refer to and outlines many of the key decisions that need to be made. In our experience, the protocol is not a document set in stone. During the course of the review, it may certainly be necessary to revise portions.

See links below for more details on a few key sections of the protocol.

1. Search Strategy: seek out potential papers to include in the review
2. Paper Selection: narrow down the search results to the most relevant papers to include
3. Quality Assessment: assess quality of the papers included
4. Data Extraction and Synthesis: pull out the data to be synthesized

References

Centre for Reviews and Dissemination, University of York. (2009). CRD’s guidance for undertaking reviews in health care. Layerthorpe, York: CRD, University of York.

Egger, M.,  Smith, G.D.,  Altman, D.G.  (Eds.). (2001). Systematic reviews in health care: Meta-analysis in context 2nd edition. London: BMJ Books.

Garg, A. X., Adhikari, N. K. J., McDonald, H., Rosas-Arellano, M. P., Devereaux, P. J., Beyene, J., et al. (2005). Effects of computerized clinical decision support systems on practitioner performance and patient outcomes. Journal of the American Medical Association, 293(10), 1223-1238.

Green, S. (2005). Systematic reviews and meta-analysis. Singapore Medical Journal, 46(6), 270-274.

Hunt, D. L., Haynes, R. B., Hanna, S. E., et al. (1998). Effects of computer-based clinical decision support systems on physician performance and patient outcomes: A systematic review. Journal of the American Medical Association, 280(15), 1339-1346.

Johnston, M. E., Langton, K. B., Haynes, R. B., & Mathieu, A. (1994). Effects of computer-based clinical decision support systems on clinician performance and patient outcome. Annals of Internal Medicine, 120(2), 135-142.

Kastner, M., Tricco, A. C., Soobiah, C., et al. (2012). What is the most appropriate knowledge synthesis method to conduct a review? Protocol for a scoping review. BMC Medical Research Methodology, 12, 114.

 

Additional Resources

• Cochrane Handbook for Systematic Reviews of Interventions
• Five Steps to Conducting a Systematic Review by K.S. Khan, R. Kunz, J. Kleijnen, & G. Antes
• Systems to Rate the Strength of Scientific Evidence by the Agency for Healthcare Research and Quality