October 2004
Electronic Data Validation: to V or Not to V
Mention data validation and the expectation is that it will be electronic. Many hours are spent in the preparation and testing of electronic validation plans and checks in pursuit of the perfect database. Manual validation is all but forgotten. But is manual validation really the under-dog?
The big plus-points for electronic validation are in its consistency. It is fast (eventually!). It is well documented and tracked; you know exactly what has been checked each time batch validation runs. Comparisons are run across single or distributed databases, across sites, across countries. Error rates are calculable, allowing poor performing sites to be identified and improvement processes to be put into place. You can run it whilst doing something else; you need not wait for anyone. And checks can constantly be improved, upgraded, redesigned.
But at what cost? Reliance upon automatic checks can hide a multitude of hazards. Once programmed, the broader picture is easily missed. There may be no regular expert review. Incorrectly programmed checks may present a rose coloured view of the data. It could be check specification that is at fault rather than the programming itself. Those checks that produce no hits... why are they there? Are they of use? Are they suitable for the purpose? Have you missed something altogether?
Enter Manual Validation – save the day with grey matter and perseverance (variously known as data management). Manual validation is seen as boring, tedious and time consuming. It can be difficult to document and can suffer from a difference of personal interpretation and human error. Could we use the time more effectively?
Yes and No.
In the words of ACDM lecturer and Quintiles global product specialist Gerhard du Toit, "Automatic checks are just as good as your test data are". And test data relies on grey matter to specify and test every conceivable angle so that the electronic check really does work. Without that expert manual input, batch validation is as useful as a chocolate teapot!
Visually scanning database output can instantly give peace of mind. You can spot new variants and apply clinical logic to therapy area checks. Programming complex checks for a small trial may eat into the allocated resources, whereas review of listings can easily serve up anomalies on a plate. Manual review at an early stage in the trial may suggest more appropriate checks; you are more adaptable than a set of programmed checks (the programming resource for which may long since have moved on to another trial). And in place of lengthy test plans, your trained brain can easily spot holes in the plan; can devise new angles through which to ensure a cleaner result. And, yes – it can be fun! Spotting that needle in the haystack can bring with it a sense of achievement. That odd character in the listing stands out like a sore thumb.
And were those electronic checks entirely electronic? No: they relied on manual checks all the way; manual specification, manual preparation of test plans, manual review at every stage from protocol and CRF design through to manual review of data listings at the end of trial. Electronic production of queries to site – is it entirely electronic? Is the query substantial or will an obvious correction document allow you to shelve this problem? Do you need to press the button? Print the query? Deliver the query? Does the monitor review the query? Post the query? Track the query out and in? All require manual intervention.
There is nothing to replace the manual check: electronic just removes much of the tedium. The plethora of manual checking dwarfs the electronic. From the employment and training of staff through every facet of the trial to the final report, every aspect constitutes validation in a trial. Manual validation is the ultimate controller: no electronic system is more effective.
Manual is the big V. We just think of it as being electronic!
References and Bibliography:
Gerhard du Toit, Global Product Specialist, Product Development, Quintiles Ltd.
Kingston University ACDM Module "Data Handling/Data Management Processes, January 2004
Frances Fancourt, Data Manager
Novo Nordisk Ltd, Crawley