Models

Clinical Adoption Meta Model

The eHealth Observatory has developed the Clinical Adoption Meta Model to describe how evaluation should evolve over the lifecycle of a health information system (HIS) implementation. The impact of a health information system is seen as something that changes over time (Figure 1). Thus, to provide appropriate evaluation, one should be aware of expectations of adoption at any given point in time. This model describes four main types of measurements after a system is implemented. A fifth type of measurement, baseline / design, is described to ensure appropriate evaluation of the system occurs prior to implementation.  Briefly, the five aspects in the adoption model are:

Figure 1: eHealth Observatory's Clinical Adoption Meta Model

 

0. Measure Baseline / Analyze Design - This early aspect of adoption is measured prior to implementation. Baseline measures should contain elements of the other four aspects in the model, allowing for comparison to current state. Also, prior to implementation, evaluation can occur to assess the design of the information system. This is helpful to develop early evidence to support design concepts prior to more expensive development / deployment. Design evaluation can also help refine the information system, expectations for adoption and scope of implementation and training requirements.

1. Measure Accessibility – The measurements after implementation are those related to accessibility. These include quantitative measures of data in repositories numbers of users with access to the information system (e.g. those with physical access, login credentials, training). Accessibility measures can be more complex as well, relating user access to specific information / content types.

2. Measure System Use – After access, use of the system is the next type measurement to consider.  As more information becomes accessible[1] and users’ trust in the system increase over time, it is expected that rates of use will increase. Measuring system use includes quantitative usage metrics, but also includes efficiency, usability, and user satisfaction.

3. Measure Clinical Behaviours – Changes in clinical behaviours are the next expected change that can be measured. These occur as people adapt to the system and respond to its benefits. Clinical behaviours differ from system use in that they are incorporating information and functions provided by the information system into activities not directly related to the use of the system. For example, behaviour changes could be reduction in face to face visits.

4. Measure Clinical Outcomes – Finally, as clinical behaviours change, they could have an impact on clinical outcomes (both surrogate and final). These are expected to change last in the Clinical Adoption Maturation Model. Clinical outcomes are difficult to attribute directly to a clinical information system as there are many other elements of the health system that can also impact outcomes.

Together, these five measurements provide a structure to help establish a staged evaluation for HISs over time, extending from formative and baseline evaluation to assessing changes in clinical outcomes.