It’s been rare in clinical research to see an organization measure deviations and issues as occurrences other than where someone failed to follow an SOP or related controlled document. The problem with this approach is that it measures how many times someone has deviated from the approved, published process. That approach does not measure whether the output of the process is any good. 100% adherence to a process that produces lackluster results doesn’t reflect what clinical research is all about and it certainly doesn’t reflect what modern clinical research should be about.
The phrase “Lies, Damn Lies and Statistics” is often attributed to Mark Twain although it’s exact origins are not completely known. What is known and the reason this quote has survived is that it’s true. It calls out that statistics can really be used in any argument in the wrong way, for the wrong reasons.
In the last decade in particular, clinical research has accelerated and become a lot more complex with the abundance of data, analytical tools, adaptive trials, genomics, personalized medicine and clinical research as a care option. We’ve got too much going on in clinical research to be measuring compliance with SOPs. We need to be measuring the outcomes of processes and using those measures to improve the processes themselves.