Notes

Strengths and weaknesses

Strengths:

• The approach works well for observing non-centralized tasks – because the observer is free to travel wherever the subject goes, it makes no difference if the process task isn’t centralized to a confined area (i.e. an observer could follow a physician throughout the entire hospital as the physician does his/her rounds)
• Time-Motion studies can capture short tasks or frequently changing tasks – since the observer is following just the one subject, every task can be monitored and recorded by the observer, regardless of how quickly the task is performed. For observational methods where the observer to subject ratio isn’t 1:1, such short, frequently changing tasks could be entirely missed during the observation (as the observer may be with another subject whilst the short task is completed)
• Time-Motion studies create measures with a high degree of depth– during a Time-Motion study, all process tasks are observed and recorded, meaning the depth of a process is entirely captured

Weaknesses:

• The 1:1 observer to subject ratio is extremely resource intensive – it is costly (with respect to both time and money) to dedicate observers to single subjects
• Close observation can be a disturbance to the subjects – subjects can often times have ill feelings towards being so closely under the microscope for lengthy periods of time. A dedicated observer to a research subject may make the subject feel like they are being personally monitored for working poorly, making the subject feel that their job is in jeopardy. It must be made clear to the subjects that this is not the case, and that the observations are simply being used to make their job better.
• The Hawthorne effect – it can be argued that the observation of subjects causes them to change their behaviour; thus, causing the observation results to be unrepresentative
• The distance from the observer to the subject can cause observation limitations – the further away the observer is to the subject, the more difficult it is for the observer to recognize what the subject is doing (i.e. if the subject is witnessed being on the phone, at a distance, it would be impossible to determine whether or not the subject is making a personal phone call, or if the subject is performing a critical process task)
• Low degree of breadth in measures – the activities of few may not be indicative of the activities of all. Unless considerable time is spent observing numerous subjects (which can be quite costly at a 1:1 observer to subject ratio), Time-Motion studies can often times have a low degree of breadth (due to the fact that the way one person performs a process may not necessarily be the same as how another person performs the process). For example, if physician use of a new EHR system was under investigation in a hospital with 200 physicians, a Time-Motion study may only be able to feasibly monitor 10 of those physicians (due to the costs of conducting the 1:1 observer to subject observations). These 10 observations may not accurately cover the breadth of physician use of the EHR system, since those 10 physicians who were observed using the EHR system may use it differently than the remaining 190 physicians at the hospital.

Return to tabs

Plan

Sample selection

Sample selection

The sample selected for a Time-Motion study needs to be representative of the process under study’s total participant population. The user profile set, created in the previous phase of the RR framework, should already provide the study designer with a listing of this information; thus, the only remaining tasks left to perform for the sample selection process are: to determine the number of each process participant type to include in the study, and to select the individuals who fit these criteria. For instance, the user profile set may indicate that a prescription ordering process at the clinic under study may involve: patients, physicians, nurses, pharmacists, and medical office assistants. Knowing this, the study designer should determine exactly how many of each user type is involved in the process (i.e. 2 physicians, 3 nurses, 2 pharmacists, and 5 medical office assistants). Then, members of each process participant group should be selected for the study so that an approximation of the process, for all process members, can be formed. Not every participant from the user group needs to be selected; however, at least one member from each participant group needs to be selected, and the number of subjects selected from each group should be enough to acquire a close approximation to what the rest of the group does in the process. The actual selection of the individuals from each of the participant groups can be accomplished randomly (via the use of a random number table), or, if necessary, voluntarily (if only certain group members are willing to participate in the study).

Return to tabs

Category development

Category development

To conduct a Time-Motion study, a process-task coding scheme first needs to be established. This coding scheme should contain a code and a definition for every task conducted in the process under study. These codes will be used by the researchers to make note of the process tasks, as they occur, in real-time, during the observation period of the Time-Motion study. For this reason, the codes need to be both clear and concise, so that they can be well understood, and quickly recalled by the researchers whilst they observe the study subjects perform their work processes.

The content of the process task coding scheme is dependent on the particular process under study. Ideally, a ready-made, previously validated coding scheme should either be used or customized to save preparation time. Regardless of whether or not the coding scheme was derived from a previously created set or created from scratch, the coding scheme should be consistent with other literature in the field to allow for cross comparisons to be made between articles. The finalized coding scheme should also promote inter-observer agreement between researchers as they observe process tasks being conducted (i.e. if two researchers observe the same process task, they should be able to select the same task code from the coding scheme without clarifying with one another).

An example process task coding scheme for physician inpatient activities is defined in (Overhage, Perkins, Tierney, & McDonald, 2001). This coding set has been used in several previous studies (McDonald, Overhage, Tierney, & al., 1992) and (Tang, Jaworski, Fallencer, & al., 1995), and has been validated and revised by numerous physicians over the past several years. The coding scheme is broken down into major and minor categories to better facilitate rapid code lookup by the process researchers in the field. The major categories in the Overhage coding scheme are: ‘Direct Patient Care’ (which involve such minor categories as: patient examination, a lab test procedure, etc.), ‘Indirect Patient Care’ (which include such minor categories as: checking the computer for a drug reference, getting test results on the phone, etc.), ‘Administrative’ (which includes such minor categories as: checking emails, reading the schedule, etc.), and ‘Miscellaneous’ (which includes such minor categories as: eating, personal phone calls, etc.).

Return to tabs

Select observation schedule

Select observation schedule

After determining who will be measured and what process tasks will be measured, the next step for a Time-Motion study designer is to determine the observation schedule. The first step in this task is to determine the number of observations that need to be made for each subject. Generally speaking, the more routine a subject’s process is, the fewer observations will need to be made in order to determine a close approximation to the process. On the contrary, the more varied a subject’s process is, the more observations will be needed in order to attain a close process approximation. Also, if there are several subjects chosen from a single process participant type (i.e. under the user profile: ‘Physicians’), then there will be fewer repeat observations made on the same subject (i.e. if there are 20 different physicians being observed in the study, each performing the same process, then there is less of a need to observe each physician multiple times). If there are, however, few members of each process participant type in the study, then it is more likely that repeat observations on the same subjects will need to be made (i.e. if there is only one physician being observed in the study, then it may be necessary to observe him/her several times in order to attain a better understanding of the process).

Once the number of observations has been established, a schedule can be created to determine:

• when the observations will take place;
• how long each observation will be;
• who the subject under observation will be;
• who the researcher will be that will conduct the observation.

It is important when creating the observation schedule to observe a ‘typical’ day (meaning the study subjects should not have to change their schedule, or wait for a particularly calm day for the study to take place). The duration of the observation should encompass the entire process under investigation, with added time both before the process begins and after the process finishes to give the researcher time to both setup and help the subjects become use to being observed.

Return to tabs

Selecting the observers

Selecting the observers

An observer in a Time-Motion study must be able to:

• understand the process task coding scheme
• recognize process tasks as they occur in the field
• fit comfortably into the process environment (i.e. in a clinical setting with patients, physicians, etc.)
• use process recording tools (i.e. PDAs and their applications)

Under these criteria, in a medical environment, it is believed that clinicians are the ideal candidates for Time-Motion study observers. If the process observer already has background knowledge in the process (i.e. if it is a physician observing physician tasks), then it is much easier for them to recognize the process tasks as they occur. Also, especially in a clinical setting, it is imperative that not only should the observer feel comfortable in the process environment, but the process participants (the study subjects) need to feel comfortable around the observer. If the observer is a clinician, then there should be far fewer problems with having them observe a fellow clinician who is with a patient. If, on the other hand, a non-clinician was used as an observer in a clinical setting, then the patients may not feel comfortable enough with having the third-party observer around during their private visits, and the clinicians would be less accepting of having someone watching their work. Med-students can also make for good observer candidates, for not only do they meet the above mentioned criteria, but many also receive time off from studies to observe medical practices, meaning their participation in the research could be mutually beneficial.

The number of observers required in a study is dependent on the number of observations needed in the study, and the speed at which the study needs to be completed. Having more observers in a study means that there can be more observations taken in a shorter period of time. Extra observers also means, however, that the study’s training costs will increase.

Return to tabs

Software and hardware selection

Software and hardware selection

Although it isn’t necessary to use computer software and hardware to measure and analyze workflow processes (such measurements and analysis can be performed with a pen, paper, and a watch), modern technology can make these tasks far more efficient.

The hardware that is recommended for Time-Motion studies includes:

• a portable computing device (i.e. a handheld computer) – used on site to record the processes as they occur. Examples of their use can be found in (Overhage, Perkins, Tierney, & McDonald, 2001), who used the Newton Message Pad 120, Newton Inc., Cupertino, California, and (Hollingworth, et al., 2007), who used a Palm® Tungsten handheld PDA in their study.
• a computer – used to store the recorded data, and to run the data analysis software (i.e. SAS, Microsoft Excel, etc.)

The software that is recommended for Time-Motion studies includes:

• process task recording and timing software (i.e. TimerPro - http://www.acsco.com/TimeandMotion.htm - (Hollingworth, et al., 2007))
• process task analysis software (i.e. Microsoft Excel)

By utilizing these technologies, the following benefits can be reaped:

• Automated task coding – the hardware device can be pre-populated with the task codes (from the coding scheme), enabling rapid coding of the tasks as they occur in the field
• Automated timing – the hardware can automatically provide a task start-up timestamp whenever a task is selected on the device, and a task finish timestamp whenever a new task is selected (i.e. a start time is automatically assigned to the code ‘Direct Patient Care – Patient – Phone’ when the code is selected on the handheld device, and a stop time is assigned to the task the second the code ‘Indirect Patient care - Computer – Writing Note’ is selected, signifying that one task has stopped because another has started)
• Electronic notes – the observer can add notes to their coded observations to further clarify what the subject is doing (in case there are any discrepancies in the observation)
• Population into PC for analysis – once the observations are taken, they can be transferred over to a computer for safe (backed-up) storage and future analysis
• Automated data analysis – such computer applications as SAS and Excel can make conducting statistical analysis on the time-measurement figures much more efficient

Return to tabs

Training observers

Training observers

Before the selected observers can go into the field and start collecting data through observations, it is first necessary to provide them with a brief training session. During the training session, the process observers should be informed about:

• the purpose of the study;
• how to use the hardware/software that will be used in the study for data capture and analysis;
• and how to interpret and use the codes in the coding scheme.

When describing the purpose of the study, it may only be necessary (and recommended) to simply inform the observers that the timing of the given process is under study. If, for example, a controlled Time-Motion study was conducted to test the physician time efficiency gains or losses in a clinical practice due to a new EHR system, it may be a hindrance to the study to inform the observers that the study aims to measure the effect of the EHR on the physician’s practice, as opposed to simply stating that the study aims to measure the physician’s practice (since informing the observer about the EHR system being tested may inject a personal bias into the measurements).

Training on how to use the hardware and software for the study should be straight forward, but it is imperative none the less, as the researchers will need to be able to use the tools of the study very quickly in order to keep up with the high paced clinical environment.

Perhaps the most important training exercise for the study process observers is making sure that all of the observers understand the process task codes in the coding scheme, and know how to apply them when observing in the field. If any of the process task codes or definitions are misunderstood by the observers, and later recorded during the study observations, then the Time-Motion study results will be invalid. To assure that this doesn’t happen, each code in the coding scheme should be discussed in detail, providing the observers with information about when the code should be used and when the code should not be used. After each code has been clarified, test observational sessions should be conducted, to provide the coders with a chance to use the coding scheme as well as the observational software and equipment before the study commences. Once this testing session is completed, the observers should discuss their test results and clarify any discrepancies between their observation recordings. For any discrepancies found in the test observation coding results, further clarification on the coding should be given to the observers about the codes in question, and some of the codes from the coding scheme may also need to be revised (if they were found to be unclear or unrepresentative). This training session can typically be completed in the span of around two hours (Overhage, Perkins, Tierney, & McDonald, 2001).

Return to tabs

Do

Data collection

During the data collection phase of Time-Motion studies, the observers will shadow the test subjects as they complete the tasks needed for the fulfillment of the process under study. For each observation, one observer will follow one subject so that every movement by the subject can be followed in great detail by the observer. At the beginning of a scheduled observational session, the observer will record their name, their subject’s name, as well as the start time and date of the observation. The observer will then record the tasks as they occur, using the data collection tool (i.e. a handheld computer) to input the task codes and timestamps. At the end of the observational period, the observer will record an end time and date in the data collection tool, then load the observational results into a computer for analysis and secure storage.

Survey

As a secondary means of retrieving process task data, after the observational periods are completed, the subjects can be given a survey (i.e. an interview or a questionnaire) to further discuss their work processes. These surveys can be used to collect more qualitative information that can be used to support the quantitative data gathered from the Time-Motion analysis. Where the Time-Motion study will answer the how questions for the process tasks (i.e. “How was the process completed?”), the survey could answer the why questions (i.e. “Why did you perform that task at that time?”). For more information on how to construct and deliver surveys, please see the PLAN section of Phase 2 – Know the Users.

Return to tabs

Study

Data analysis

The data analysis phase of Time-Motion studies is spent calculating and summarizing the various recorded task times from the processes under study. Normally, the observational data is loaded into a computer for analysis using a statistical tool such as SAS or MS Excel. Once a clear breakdown of the process tasks is made, it then becomes possible to dissect the process into its measured, task components. Benchmarking can be used to compare the efficiency of these measured tasks and processes with the efficiency of the same tasks and processes from other, best-practice sites. If the process is shown to be less-efficient than other similar processes from other practices, then it can be assumed that the process can be improved through a re-modeling and refinement process.

Process refinement

Because Time-Motion studies are solely used to measure processes, and not to refine them, if it is determined that the current processes and tasks are inefficient, then a process refinement methodology (i.e. a Kaizen Event) can be used to increase the efficiency of those tasks and processes. For more information on how to refine the process, please review the Kaizen Event PDSA cycles section. The results from the Time-Motion study can be used as input into the Kaizen Event as a near-to-exact representation of the current workflow. The Kaizen Event team could then take this current workflow mapping and use the Kaizen processes (defined in the Kaizen Events section) to implement incremental process improvements.

Return to tabs

Act

There are two possible courses of action after completing the analysis of a Time-Motion study. Either the results can be compiled into an evaluative report of the current work-practices or they can be fed into a process refinement methodology as a mapping of the current workflow. This choice is dependent on whether or not there were any problems found in the current workflow (if it isn’t broke, don’t fix it), and whether or not there are enough resources to fund the process refinement stage if the process does need refinement.

Return to tabs

PDSA Summary

The Time-motion methodology involves researchers continuously observing subjects while they perform their work, and recording each task and its duration into a log file (based on a coding scheme). A 1:1 researcher to subject ratio is required during data collection. When observations are completed, the task log files can be compiled and analyzed to represent a near-to-exact depiction of subjects’ work.

Plan

• Select the study sample based on the user profiles – Which user groups will be monitored?
• Develop the study measurement coding scheme – What will be monitored?
• Create the observation schedule – When will the observations take place?
• Select the observers – Who will collect the study data?
• Obtain needed software and hardware – What tools are needed to collect the data?
• Train the observers – How will the observers collect the study data?

Do

• Collect the observational data
• Survey the users to clarify the observational data

Study

• Use the data to develop a well-defined map of the current work process

Act

• Either summarize the study results into a Time-Motion evaluative process report, or use the results in a process refinement methodology (e.g., Kaizen Event)

Return to tabs

References

References

• Hollingworth, W., Devine, E. B., Hansen, R., Lawless, N. M., Comstock, B. A., Wilson-Norton, J. L., et al. (2007). The Impact of e-Prescribing on Prescriber and Staff Time in Ambulatory Care Clinics: A Time-Motion Study. Journal of the American Medical Informatics Association , 14, 722-730.
• McDonald, C. J., Overhage, J. M., Tierney, W. M., & al., e. (1992). The Regenstrief Medical Record System: 20 years of experience in hospitals, clinics and neighborhood health centers. MD Comput. , 9 (4), 206-217.
• Overhage, M. J., Perkins, S., Tierney, W. M., & McDonald, C. J. (2001). Controlled Trial of Direct Physician Order Entry - Effects on Physicians' Time Utilization in Ambulatory Primary Care Internal Medicine Practices. Journal of the American Medical Informatics Association , 8 (4), 361-371.
• Tang, P. C., Jaworski, M. A., Fallencer, C. A., & al., e. (1995). Methods for assessing information needs of clinicians in ambulatory care. Proc 19th Annu Symp Comput Appl Med Care. , 630-634.

Return to tabs