Metrics
Introduction
The following are some of the design components and metrics to evaluate HIS adoption/use.
Exhibits 1- 7
Exhibit 1
Exhibit 1.
Four study cohorts can be defined according to the adopter/user and HIS type.
Legend: *means only EMR or eDrug is adopted/used but not both; +/- means either EMR or eDrug is present for basic-adopters/users
Cohort | EMR | eDrug | EHR |
---|---|---|---|
A: Non-adopter/user | |||
B: Basic-adopter/user* | +/- | +/- | |
C: Intermediate-adopter/user | √ | √ | |
D: Advanced-adopter/user | √ | √ | √ |
Exhibit 2
Exhibit 2.
Each cohort in 3a can be further divided into three care settings each with different HIS types
Setting | EMR | eDrug | EHR |
---|---|---|---|
Physician Office | √ | √ | √ |
Community Pharmacy | √ | √ | |
Acute/Tertiary/continuing care | √ | √ |
Exhibit 3
Exhibit 3.
Each cohort in 3a can be further divided into three provider types each involved with different HIS types
Provider | EMR | eDrug | EHR |
---|---|---|---|
Physician | √ | √ | √ |
Pharmasist | √ | √ | |
Authorized health professional | √ | √ |
Exhibit 4
Exhibit 4.
Different study design options available based on composition of the cohorts and time periods being examined. Legend: N is sample size; T is time period; subscripts 0 to 5 denote the sample sizes and time periods for each cohort; ∑ refers to the column, row and grand total sample sizes. For example, Cohort-A at Time-0 can be defined to include all three types of providers across all three settings who have not adopted/used EMR, eDrug and EHR at the baseline period.
Examples | Time-0 | Time-1 | Time-2 | Time-3 | Time-4 | Total |
---|---|---|---|---|---|---|
Cohort-A | A(N1T0) | A(N2T1) | A(N3T2) | A(N4T3) | A(N5T4) | A(N1-5) |
Cohort-B | B(N1T0) | B(N2T1) | B(N3T2) | B(N4T3) | B(N5T4) | B(N1-5) |
Cohort-C | C(N1T0) | C(N2T1) | C(N3T2) | C(N4T3) | C(N5T4) | C(N1-5) |
Cohort-D | D(N1T0) | D(N2T1) | D(N3T2) | D(N4T3) | D(N5T4) | D(N1-5) |
Total | ∑(N1T0) | ∑(N2T1) | ∑(N3T2) | ∑(N4T3) | ∑(N3T4) | ∑(N1-5) |
Exhibit 5
Exhibit 5.
Five kinds of studies to examine the information/system/service quality and use/satisfaction dimensions in the BE Framework. Legend: Cohorts-A, B, C, D are non-adopters/users with no HIS, basic-adopters/users with eDrug or EMR, intermediate-adopters/users with eDrug and EMR, and advanced-adopters/users with eDrug, EMR and EHR, respectively. Time-0 is the baseline period, Time-X is the next period, Time-Y the period after Time-X, and Time-Z after Time-Y, etc. Cohort-A with no HIS is excluded since these studies all require specific types of HIS being adopted/used.
Studies on BE Quality/use Dimensions | Cohort-A | Cohort-B | Cohort-C | Cohort-D |
---|---|---|---|---|
(1) Comparison: documented features vs. best practices | Time-0 to Time-X/Z | Time-X to Time-Y/Z | Time-X to Time-Y/Z | |
(2) Usability: ease of use, functionality and usefulness | Time-0 to Time-X/Z | Time-X to Time-Y/Z | Time-X to Time-Y/Z | |
(3) Qualitative case: survey, interview and observation on on percieved/actual use | Time-0 to Time-X/Z | Time-X to Time-Y/Z | Time-X to Time-Y/Z | |
(4) Quantitive case: system use log triangulated with qualitative case findings | Time-0 to Time-X/Z | Time-X to Time-Y/Z | Time-X to Time-Y/Z | |
Data quality audit: patient electronic records vs. manual chart | Time-0 to Time-X/Z | Time-X to Time-Y/Z | Time-X to Time-Y/Z |
Exhibit 6
Exhibit 6.
Five kinds of studies to evaluate the care quality and productivity dimensions in the BE Framework. Legend: Cohorts-A, B, C, D refer to non-adopters/users with no HIS, basic-adopters/users with eDrug or EMR, intermediate-adopters/users with eDrug and EMR, and advanced-adopters/users with eDrug, EMR and EHR, respectively. Time-0 is the baseline period, Time-X is the next period, Time-Y the period after Time-X, and Time-Z after Time-Y, etc.
Studies on BE Benefits Dimensions | Cohort-A | Cohort-B | Cohort-C | Cohort-D |
---|---|---|---|---|
(1) Efficiency: Call-to-pharmacy; special authority request adjudication | Time-0 to Time-X/Y | Time-0 to Time-X/Y | Time-X to Time-Y/Z | Time-X to Time-Y/Z |
(2) Coordination: review/reconciliation of medication management | Time-0 to Time-X/Y | Time-0 to Time-X/Y | Time-X to Time-Y/Z | Time-X to Time-Y/Z |
(3) Patient safety: adverse drug event and medical error prevention and detection | Time-0 to Time-X/Y | Time-0 to Time-X/Y | Time-X to Time-Y/Z | Time-X to Time-Y/Z |
(4) Effectiveness: special authority compliance; disease management | Time-0 to Time-X/Y | Time-0 to Time-X/Y | Time-X to Time-Y/Z | Time-X to Time-Y/Z |
Exhibit 7
Exhibit 7.
Two examples of time series delayed controlled matching cohorts. First is the two matching Pharmacy cohorts where baseline measures are taken at Time-0 with the original PharmaNet. Then Pharmacy1 deploys the new eDrug in Time-1 while Pharmacy2 continues with PharmaNet. Then Pharmacy2 deploys new eDrug in Time-2 while Pharmacy1 continues to use eDrug. Then Pharmacy1 deploys a new EMR while Pharmacy2 continues to use eDrug. Another example is the matching Physician cohorts for deployment of EMR, eDrug and EHR.
Examples | Time-0 | Time-1 | Time-2 | Time-3 | Time-4 |
---|---|---|---|---|---|
Pharmacy 1 | Original PharmaNet | New eDrug | eDrug | eDrug + New EHR | eDrug + EHR |
Pharmacy 2 | Original PharmaNet | Original PharmaNet | New eDrug | eDrug | eDrug + New EHR |
Physician 1 | Exsisting EMR | EMR + New eDrug | EMR + eDrug + New EHR | EMR + eDrug + EHR | EMR + eDrug + EHR |
Physician 2 | No HIS | New EMR | EMR + New eDrug | EMR + eDrug + New EHR | EMR + eDrug + EHR |
Reactive Analysis
Reactive Analysis of an Information System, from the IT Interaction Model by Silver et al. [xx]
- Was the information system's design objective to IMPROVE the organization incrementally or to TRANSFORM it?
- What are the information system's FEATURES? What does it do?
- How does the information system FIT the firm's EXTERNAL ENVIRONMENT?
- How does the system FIT the firm's STRATEGY?
- How does the system FIT the firm's BUSINESS PROCESSES?
- How does the system FIT the organizational STRUCTURE and CULTURE?
- Can the organization's existing IT INFRASTRUCTURE support the system? Does the information system leverage the infrastructure? Does it extend it?
- How and how effectively was the system IMPLEMENTED?
- Who USES the system and how do they USE it? As intended?
- What are the CONSEQUENCES of the system for PERFORMANCE, PEOPLE, and FUTURE FLEXIBILITY? Did the system accomplish its objectives?
Guiding Questions
Guiding Questions from Kotter's Organizational Change Model [xx]
- Has the sense of urgency for change been established within the organization?
- Has a guiding coalition been created within the organization?
- Has the vision and strategy been developed within the organization?
- Has the vision and strategy been communicated throughout the organization?
- What is the mechanism to empower broad-based action for change throughout the organization?
- What are the short-term wins for the organization that can be generated from this change?
- What are the efforts to consolidate gains and produce more change within the organization?
- How are the new approaches being anchored within the organizational culture?

News and events
- Publications, presentations, and projects sections of website updated
- Paper published in International Journal of Health Information Management Research (2014)
- Paper published in BMC Medical Informatics and Decision Making (2014)
- Presented at Queen’s Health Policy Change Conference Series (May 15-16, 2014)
- Presented at CAHSPR 2014 Conference (May 12-14, 2014)
- Paper published in Healthcare Quarterly (2014)
- Paper published in Journal of American Medical Informatics Association (2014)