Scope and Outputs

Scope

This systematic review will focus on extant systematic reviews on health information systems (HIS).

Outputs

A summary report, presentation materials and a series of paper(s) on the review process and results will be prepared.

Timeline

It was estimated to take six months to complete the review, but has been in development for two years. The following table provides estimated durations for each part of the final review process conducted between January and May 2009 (and ongoing), recognizing that a significant amount of data extraction was previously done and some organizational strategy already implemented. Note that tasks generally overlapped so that the total time taken is slightly less than the sum. Tasks may also not have been completed in the order specified:

Part	Estimated Time
Location of missing electronic articles, folder reorganization and documentation of data tables; Organization and documentation of all electronic and print articles for retrieval, including abstracts lists.	4 weeks
Development of process for identifying and managing previously-completed data extraction	1 week
Draft Data Extraction Template and use to update and edit previously-completed Data Extraction	3 weeks
Modify Data Extraction Template and use to enter Data Extraction (Phase II) from pre-2006 articles provided as Phase I	1 week
Creation of Table Views for easy definition of the tables to be used in the meta-Review articles	1 week
Confirm, adapt, and use Review of Reviews search strategy, new Literature Search (2007-2009); select papers and conduct Data Extraction Phase III	2 weeks
Monitor and update Review Protocol, including details from Data Extraction Template to provide a Companion Report (i.e. guide for tabular completion)	2 weeks
Conduct any additional Data Extraction missed in Phase I and II, for instance Quality analysis by Study	2 weeks
Further table modification and data revision according to evolving needs	4 weeks
Produce Systematic Review of Systematic Reviews Paper(s)	Unknown

Methods Used

General Details

General Details

The Inclusion Criteria used in the review focused on published systematic review articles in English that addressed HIS in practical use by healthcare providers in specific healthcare settings – where the definition of HIS was broadly defined to cover different types of systems for information processing, decision support and management reporting. In order to necessarily and sufficiently respond to the Review Question(s) a review could also be excluded if: it provides either insufficient or no detail for the Data Extraction with respect to Study Designs, Participants, and Settings; or it presents HIS as only one of several potential interventions for a clinical issue. Certain types of HIS systems are further excluded from the meta-review, following their explicit exclusion from the prior meta-Review: telehealth / telemedicine applications, systems used directly by patients, and systems only for patient or provider education. Exploration of handheld EMR (PDA) use was not explicitly excluded in the prior meta-Review write-up, but emerged as an additional Exclusion Criteria. Reviews on these topics were excluded because separate meta-Reviews in these specific areas were planned for future publication.

Inclusion and Exclusion Criteria Diagram

Inclusion and Exclusion Criteria Diagram

This diagram is a simplified and condensed version of the Inclusion and Exclusion criteria diagrams included in the Review of Reviews Protocol.

Review terminology is removed, due to the database’s defined Systematic Review content.

Article-based Search Strategies

Article-based Search Strategies

References of Papers Selected

For each paper selected from 1982 to 2006, references were be searched and citations extracted for relevant Systematic Review articles; those included or excluded from Data Extraction already were removed from the list derived, and the remainder readied for evaluation. Inclusion/Exclusion criteria were then applied. From this process 2 prepublication duplicates and 19 other Systematic Reviews (span: 1990 to 2007) were identified and excluded, the latter based on failing Inclusion and meeting Exclusion Criteria. See the following summary:

Not HIS:	1
Insufficient/No Detail:	6
Conceptual:	8
Telemedicine/Handheld EMR	4

7 Reviews from the last two calendar years were included for Data Extraction:

Nies, J.	2006	Proc AMIA Annu Symp 2006:594	CDSS: Use/Evaluation
Randell, R.	2007	J Health Serv Res Policy 12:242	CPOE: Use/Evaluation
Eslami, S.	2007	JAMIA 14:400	CPOE
Handler, S.M.	2007	JAMIA 14:451	HIS: Use to detect ADEs
Shebl, N.A.	2007	Pharm World Sci 29:342	CDSS: Use/Evaluation
Sintchenko, V.	2007	Med Inform Internet Med 32:225	CDSS
Eslami, S.	2007	Int J Med Inform 77:365	CPOE: Use/Evaluation

Hand Search of Journals

Based on publication of a systematic review having been included in previous data extraction, the following journals were identified and searched for reviews (2007–2008). Reviews found were excluded if no HIS relevance can be determined based on the title, abstract, and content (i.e. it had insufficient or nonexistent detail), and marked for further investigation if promising for the Review of Reviews. With additional inclusion/exclusion criteria applied 6 Reviews were identified:

Eslami	2008	Int J Med Inf 77:365	cP(M)OE
Dorr	2007	JAMIA 14:p156	HIS
Eslami	2007	JAMIA 14:p400	CPOE
Handler	2007	JAMIA 14:p451	HIS for ADE detection
Ammenwerth	2008	JAMIA 15:p585	E-Prescribing
Sintchenko	2007	Med Inform Internet Med 32:225	CDSS

Initial Results

Initial Results

Initial Results

The systematic review of existing systematic reviews on health information system field evaluation studies is being conducted in order to: (a) describe the different health domains, health information system features and healthcare settings that have been examined; (b) examine the different study designs, research methods and performance/outcome measures that have been used to date; (c) offer guidance on how one should design, conduct and appraise health information system field evaluation studies; and (d) discuss the implications of findings on the future need, design and conduct of health information system studies. The links in this section provide Tabular Views derived from data extraction which will facilitate the noted goals, and a list of citations for the 46 reviews currently included in the meta-Review.

Summary

Summary

Note: a summary table will added

Quality Scales

Quality Scales

Note: a quality scales table will be added

Sorted/Grouped By Topic

Sorted/Grouped By Topic

A topical grouping table will follow

Sorted/Grouped by Metrics

Sorted/Grouped by Metrics

A metrics grouped table will be added

Corpus Table Excerpt

Corpus Table Excerpt

The Corpus of 1200 studies includes approximately 360 studies with associated quality scores, since only a subset of the 46 Reviews from which they are derived included quality analysis, and even fewer quantified the analysis. Note that the Corpus presented here includes past versions of Reviews not included in the final 46: Hunt, Johnston are updated by Garg 2005; Walton 2001 is updated by Durieux 2008.

A corpus table will be added

Reviews Included in the Meta-Review

Reviews Included in the Meta-Review

Links to Free Abstracts or Summaries are Provided Where Available

1. Ammenwerth E, Schnell-Inderst P, Machan C, Siebert U (2008). The Effect of Electronic Prescribing on Medication Errors and Adverse Drug Events: A Systematic Review. Journal of the American Medical Informatics Association 15(5): 585 – 600. [PUBMED]
2. Austin SM, Balas EA, Mitchell JA, Ewigman BG (1994). Effect of physician reminders on preventive care: meta-analysis of randomized clinical trials. Proc Annu Symp Comp App Med Care: 121 – 124 [PUBMED]

3. Balas EA, Austin SM, Mitchell JA, Ewigman BG, Bopp KD, Brown GD (1996). The clinical value of computerized information services. A review of 98 randomized clinical trials. Arch Fam Med 5(5): 271 – 278 [JAMA & Archives]
4. Balas EA, Krishna S, Kretschmer RA, Cheek TR, Lobach DF, Boren SA (2004). Computerized knowledge management in diabetes care. Med Care 42(6): 610 – 621
5. Bennett JW, Glasziou PP, (2003). Computerised reminders and feedback in medication management: a systematic review of randomised controlled trials. Med J Australia 178(5): 217 – 222 [Free e-Journals]
6. Bryan C, Austin Boren S (2008). The use and effectiveness of electronic clinical decision support tools in the ambulatory/primary care setting: a systematic review of the literature. Informatics in Primary Care 16: 79 – 91 [Free e-Journals]

7. Buntinx F, Winkens R, Grol R, Knottnerus JA (1993). Influencing diagnostic and preventive performance in ambulatory care by feedback and reminders, a Review. Fam Pract 10(2): 219 – 228 [Oxford Journals]
8. Byrne N, Regan C, Howard L (2005). Administrative registers in psychiatric research: a systematic review of validity studies. Acta Psychiatr Scand 112(6): 409 – 414 [Wiley InterScience]
9. Car J, Black A, Anandan A, Cresswell K, Pagliari C, McKinstry B, Procter R, Majeed A, Sheikh A (2008). The Impact of eHealth on the Quality & Safety of Healthcare: A Systemic Overview & Synthesis of the Literature. Report for the NHS Connecting for Health Evaluation Programme. [eHealth News.EU]
10. Chatellier G, Colombet I, Degoulet P (1998). An overview of the effect of computer-assisted management of anticoagulant therapy on the quality of anticoagulation. Int J Med Inf 49(3): 311 – 320 [Elsevier Science Direct]
11. Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, Morton SC, Shekelle PG (2006). Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med 144(10): 742 – 752 [Free e-Journals]
12. Colombet I, Chatellier G, Jaulent MC, Degoulet P (1999). Decision aids for triage of patients with chest pain: a systematic review of field evaluation studies. Proc AMIA Symp: 231 – 235 [PUBMED]
13. Cramer K, Hartling L, Wiebe N, Russell K, Crumley E, Pusic M, Klassen TP (2003). Computer-Based Delivery of Health Evidence: A Systematic Review of Randomised Controlled Trials and Systematic Reviews of the Effectiveness on the Process of Care and Patient Outcomes. Alberta Heritage Foundation (Final Report), January 2003 [Summary by Klassen]
14. Delpierre C, Cuzin L, Fillaux J, Alvarez M, Massip P, Lang T (2004). A systematic review of computer-based patient record systems and quality of care: more randomized clinical trials or a broader approach? Int J Qual Health Care 16(5): 407 – 416 [Oxford Journals]
15. Dexheimer JW, Talbot TR, Sanders DL, Rosenbloom ST, Aronsky D (2008). Prompting clinicians about preventive care measures: a systematic review of randomized controlled trials. Journal of the American Medical Informatics Association 15(3): 311 – 320 [PUBMED]
16. Dorr D, Bonner LM, Cohen AN, Shoai RS, Perrin R, Chaney E, Young AS (2007). Informatics Systems to Promote Improved Care for Chronic Illness: A Literature Review. Journal of the American Medical Informatics Association 14(2): 156 – 163 [PUBMED]
17. Durieux P, Trinquart L, Colombet I, Niès J, Walton RT, Rajeswaran A, Rège Walther M, Harvey E, Burnand B (2008) Computerized advice on drug dosage to improve prescribing practice. Cochrane Database of Systematic Reviews (3): Art. No.: CD002894 [Cochrane Reviews]
18. Eslami S, Abu-Hanna A, De Keizer N (2007). Evaluation of Outpatient Computerized Physician Medication Order Entry Systems: A Systematic Review. Journal of the American Medical Informatics Association 14(4): 400 – 406 [PUBMED]
19. Eslami S, De Keizer N, Abu-Hanna A (2008). The impact of computerized physician medication order entry in hospitalized patients - A systematic review. Int J Med Inf 77: 365 – 376 [Elsevier Science Direct]
20. Fitzmaurice DA, Hobbs FD, Delaney BC, Wilson S, McManus R (1998). Review of computerized decision support systems for oral anticoagulation management. Br J Haematol. 102(4): 907 – 909 [Wiley InterScience]
21. Garg AX, Adhikari NK, McDonald H, et al (2005). Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA 293(10): 1223 – 1238 [JAMA]
22. Georgiou A, Williamson M, Westbrook JI, Ray S (2007). The impact of computerised physician order entry systems on pathology services: A systematic review. Int J Med Inf 76: 514 – 529 [Elsevier Science Direct]
23. Handler SM, Altman RL, Perera S, Hanlon JT, Studenski SA, Bost JE, Saul MI, Fridsma DB (2007). A Systematic Review of the Performance Characteristics of Clinical Event Monitor Signals Used to Detect Adverse Drug Events in the Hospital Setting. Journal of the American Medical Informatics Association 14(4): 451 – 458 [PUBMED]
24. Hogan WR, Wagner MM (1997). Accuracy of data in computer-based patient records. Journal of the American Medical Informatics Association. 4(5): 342 – 355 [PUBMED]
25. Jackson CL, Bolen S, Brancati FL, Batts-Turner ML, Gary TL (2006). A systematic review of interactive computer-assisted technology in diabetes care. J Gen Intern Med 21(2): 105 – 110 [PUBMED]
26. Jimbo M, Nease Jr DE, Ruffin 4th MT, Rana GK (2006). Information technology and cancer prevention. CA Cancer J Clin 56(1): 26 – 36 [Free e-Journals]
27. Jordan K, Porcheret M, Croft P (2004). Quality of morbidity coding in general practice computerized medical records: a systematic review. Fam Pract 21(4): 396 – 412 [Oxford Journals]
28. Kaushal R, Shojania KG, Bates DW (2003). Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med 163(12): 1409 – 1416 [Highwire Press Free]
29. Kawamoto K, Houlihan CA, Balas EA, Lobach DF (2005). Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 330(7494): 765 – 772 [PUBMED]
30. Liu JLY, Wyatt JC, Deeks JJ, Clamp S, Keen J, Verde P, Ohmann C, Wellwood J, Dawes M, Altman DG (2006). Systematic reviews of clinical decision tools for acute abdominal pain. Health Technology Assessment (NHS R&D HTA) 10(47) [NIHR HTA]
31. Mitchell E, Sullivan F (2001). A descriptive feast but an evaluative famine: systematic review of published articles on primary care computing during 1980-97. BMJ 322(7281): 279 – 282 [PUBMED]
32. Montgomery AA, Fahey T (1998). A systematic review of the use of computers in the management of hypertension. J Epidemiol Community Health 52(8): 520 – 525 [Highwire Press]
33. Niès J, Colombet I, Degoulet P, Durieux P (2006). Determinants of Success for Computerized Clinical Decision Support Systems Integrated into CPOE Systems: a Systematic Review. Proc AMIA Symp: 594 – 598 [PUBMED]
34. Oren E, Shaffer ER, Guglielmo BJ (2003). Impact of emerging technologies on medication errors and adverse drug events. Am J Health-Syst Pharm 60(14): 1447 – 1458 [No Free Abstract Source]
35. Poissant L, Pereira J, Tamblyn R, Kawasumi Y (2005). The impact of electronic health records on time efficiency of physicians and nurses: a systematic review. Journal of the American Medical Informatics Association. 12(5): 505 – 516 [PUBMED]
36. Randell R, Mitchell N, Dowding D, Cullum N, Thompson C (2007). Effects of computerized decision support systems on nursing performance and patient outcomes: a systematic review. J Health Serv Res Policy 12(4): 242 – 249 [No Free Abstract Source]
37. Rothschild J (2004). Computerized physician order entry in the critical care and general inpatient setting: a narrative review. J Crit Care. 19(4): 271 – 278 [Elsevier Science Direct]
38. Shea S, DuMouchel W, Bahamonde L (1996). A meta-analysis of 16 randomized controlled trials to evaluate computer-based clinical reminder systems for preventive care in the ambulatory setting. Journal of the American Medical Informatics Association 3(6): 399 – 409 [PUBMED]
39. Shebl NA, Franklin BD, Barber N (2007). Clinical decision support systems and antibiotic use. Pharm World Sci 29: 342 – 349 [SpringerLink]
40. Sintchenko V, Magrabi F, Tipper S (2007). Are we measuring the right end-points? Variables that affect the impact of computerised decision support on patient outcomes: A systematic review. Med Inform Internet Med 32(3): 225 – 240 [No Free Abstract Source]
41. Shiffman RN,  Liaw Y, Brandt CA, Corb GJ (1999). Computer-based guideline implementation systems: a systematic review of functionality and effectiveness. Journal of the American Medical Informatics Association 6(2): 104 – 114 [PUBMED]
42. Tan K, Dear PRF, Newell SJ (2008). Clinical decision support systems for neonatal care. Cochrane Database of Systematic Reviews (2): Art. No.: CD004211. [Cochrane Reviews]
43. Thiru K, Hassey A, Sullivan F (2003). Systematic review of scope and quality of electronic patient record data in primary care. BMJ 326(7398): 1070 – 1074 [PUBMED]
44. Urquhart C, Currell R, Grant MJ, Hardiker NR (2008). Nursing record systems: effects on nursing practice and healthcare outcomes. Cochrane Database of Systematic Reviews (1): Art. No.: CD002099 Pub. ‘2009’ [Cochrane Reviews]
45. Van der Sijs H. Aarts J. Vulto A. Berg M (2006). Overriding of drug safety alerts in computerized physician order entry. Journal of the American Medical Informatics Association. 13(2): 138 – 147 [PUBMED]
46. Wolfstadt JL, Gurwitz JH, Field TS, Lee M, Kalkar S, Wu W, Rochon PA (2008). The Effect of Computerized Physician Order Entry with Clinical Decision Support on the Rates of Adverse Drug Events: A Systematic Review. J Gen Intern Med 23(4): 451 – 458 [SpringerLink]