Spring 2006
Metrics for Data Management Process Control
Presented by Nigel Freemantle, Merck Research Laboratories
In the clinical development process, performance measures are often concentrated on high level cycle times such as the time between key milestones in the progress of a clinical trial. Of these the most often quoted is the time between “Last Patient Last Visit” and “Database Lock”.
However if we want to assess the operational performance of a data management process, in particular against the requirements of continuous data flow, we must focus attention at the patient visit level. The measures associated with visit level cycle times have often been simplistic and unsuitable either for monitoring the underlying process or assessing performance improvement.
The cumulative cycle time distribution, represented graphically as a simple S-shaped curve, meets both needs. It defines a language, expressed in terms of percentiles, which naturally describes ongoing process control and also lends itself to the setting of meaningful process improvement goals. If the tail of the distribution indicates a control issue, outlier or exception values can be identified and analysed so that timely remedial action can be taken. Process goals can also be set in distributional terms, for example 95% of visits to be reviewed within five days of entry.
In order to realise such goals, tools must be available to the individuals involved in each stage of the process, to allow them both to evaluate their current performance and to identify potential outliers and take corrective action. A number of technical, training and cultural considerations for the implementation of such tools will be discussed and examples from corporate web-enabled and local spreadsheet environments will be presented.
Related Reading:
Beck, Bryan (1996). Clinical trials: Decision tools for measuring and improving performance. Buffalo Grove, Ill: Interpharm. Clinical trials: a practical approach. New York: John Wiley
Nigel presented an interesting perspective to the capture of metrics and the true purpose of capturing relevant data to analyse process and performance improvements. Moving away from the traditional high level cycle time metrics of “last patient last visit” (PLV) to “database lock” (DBL), Merck devised a system to capture and track more defined cycle time targets, focusing at the visit level, also enabling web based reporting and analysis, moving away from capturing metric information on spreadsheets. Visit cycle times were defined as:
Entry of visit data
External vendor data
Data review
Medical and Statistical review
Clean visit
Nigel presented various metrics figures derived for a range of studies, looking at data entry to data review times and was able to demonstrate significant increase in the number of visits completed within eight weeks and an overall median cycle time reduction of 4.7 weeks.
For process improvement, it is important to have retrospective analysis of performance after completion and assess process to make refinements and also set more realistic targets and challenges. Process control, on the other hand, requires the need to analyse performance to date on an ongoing basis. In both scenarios, communication of progress and changes is critical.
The importance of using metric data from a business perspective was also discussed. By determining where process, performance and training gaps are, via analysis of cycle time targets, real justifications for developing a business strategy for process improvement and implementation of changes are supported.
Merck developed a system and process for capturing these metrics called e-C3: Clinical Cycle-time Control. This was a web based system that enabled users to drill down into information for countries and sites to identify problem areas and also share best practice. Comparing different cycle time targets also enabled trends to be spotted at separate parts of the process. Using a web based system also provided a more powerful tool to warehouse metric data across multiple studies and drug projects.
Lisa Goodwin