|Metrics provide the foundation for
business intelligence for clinical trials,
measuring success or failure.
Why are metrics important in starting clinical trials? This question may seem counter intuitive, as we are exposed almost daily to the dire performance of clinical trials and their spiraling costs resulting from incurred delays.
According to a recent study by KPMG (1), within the pharmaceutical industry, the return on R&D expenditure has fallen from an industry average of approximately 20 percent 20 years ago, to 10 percent now, with the average cost of developing a drug rising during that period at a rate 7.4 percent higher than inflation, with the increasing costs of conducting clinical trials responsible for most of this increase (2). It is estimated that it now costs upwards of $2 billion dollars to bring a new drug to market (3).
And perhaps most disturbing is the fact that cycle time associated with starting clinical trials (i.e., steps involved in study startup (SSU), such as the selection of investigative sites at which to conduct the study, and activation of the site to receive first subject) have not changed in more than two decades (4). According to the Tufts Center for the Study of Drug Development (CSDD), 37% of sites selected for clinical trial studies under-enroll, and 11% fail to enroll a single subject. Eventually, 89% of studies meet enrollment goals, but often at the expense of sponsors faced with doubling the original timeline due to poor enrollment (5). At a time when it takes an estimated eight months to move from pre-visit through to site initiation (6), with the associated cost of initiating one site ranging from $20,000 to $30,000 (7). Overall, poor site selection, the inability of sites to predict the rate of enrollment, and the subsequent need for study rescue may increase cost of trials by 20% or more (8).
Metrics are indeed critical to efforts to rein in clinical trials that are either poorly initiated or have incurred unforeseen events which place the original timelines and/or budgets at risk of overages. They also drive competitive performance among those organizations performing trials.
Metrics and business intelligence
Metrics provide the foundation for business intelligence, affording clinical research teams an opportunity to intervene before the effects of a risk have been occurred. This risk mitigation is therefore optimal using systems which can provide timely, preferably real-time data on trial bottlenecks, which indicate red flags to be reviewed and addressed or at least tracked carefully throughout the trial.
Business Intelligence (BI) has become an increasingly popular topic in clinical trials as clinical project managers (9) are expected to make smarter decisions on intelligence derived from clinical trial data and sponsors/contract research organizations (CROs) are looking for ways to incorporate BI into the eClinical systems they are already using to empower oversight—turning raw trial data into actionable information.
By 2020 (10) 72% of clinical trials are anticipated to be outsourced, up from just 23% in 2012. With this in mind, technology that can provide sponsors with real-time insights into clinical operations is essential. This technology should also provide CROs with automated alerts for workflows and sponsors with multiple reporting options, including on-demand static reports, snapshot reports with status data that can be manipulation for further analysis, and full access.
Having technology which can automate or assist in the timely monitoring of trials is a huge improvement over the current status quo of manual methods such as spreadsheets, which are cumbersome and erroneous, not to mention only provide a dated view of trial performance. But how do metrics drive performance competitiveness?
Benchmarking of trial data allows clinical research teams to gauge their performance and progress against internal data, as well as externally (i.e., trails run by other organizations.) It allows them to see at a glance if they are on par with past trials the organization has run of a similar size, geographic footprint, therapeutic area, indication, etc. If not, why not? But equally as important, and maybe arguably more so, is how is the clinical research team performing against other organizations? This is particularly important in the case of a CRO vying for a pharma outsourced study contract or for pharma’s needing to justify the continued outsourcing relationship. A review of benchmarking data may indicate red flags not otherwise raised during the monitoring of the trial, and may be country specific.
But benchmarking is not without its challenges.
“Benchmarks should be generated from standardized, well-conceived data elements and performance metrics,” said Linda Sullivan co-founder and president at the industry group Metric Champion Consortium. “Additionally, the data needs to have sufficient metadata associated with it so you can make meaningful comparisons and correlate benchmark results with best practice outcomes.”
From an internal perspective, organizations can capture cycle-time metrics on whichever artifacts they deem important to measure, and as long as these metrics have clear definitions and are measured consistently between trials then these measurements become internal benchmarks, upon which future trials can be gauged. But for external purposes, allowing organizations to gauge performance against one another, clear, consistent and concise industry-wide standards are required. This ensures a true ‘apples to apples’ comparison that has the added benefit of improving trial data quality, because data that might not have been previously recorded, such as, certain start or stop dates is now required. Negative cycle times or cycle times that are outliers should be reviewed to ensure accurate data entry.
Interestingly today, a number of organizations claim to have the ‘best cycle time metrics in the industry’, a claim which should be treated with much skepticism, in the absence of globally recognized standards. With standards in place that can be applied across all studies, global milestones need to be utilized. Global milestones are important because they recognize that the nomenclature of artifact naming conventions is not consistent across organizations, or even countries, and nor will it ever be. For example, these are dependent on an organization’s SOPs where the events Activated, IP release, and Site Initiated could be synonymous. Nevertheless, what is important is that these cycle-time metrics can be accurately measured and mapped to an industry defined standard.
With industry standards and global milestones in place, the goal of benchmarking in clinical trials is achievable. But is this the end of the story? No, in reality it is just the beginning...
Benchmarking and gamification
Benchmarking allows for gamification, which could be extended beyond country or CRO (if the trial is outsourced to multiple vendors) to be based on role assignment, with associated financial incentives. Some might view this option as unethical or raise questions of quality, of individuals potentially gaming the system to reap the rewards and accolades of exceeding the threshold of industry performance for their position. But nevertheless, it is a logical progression and many of these arguments don’t carry much weight in a system with numerous check and balances. Moreover, it allows for greater transparency in the process of conducting clinical trials and would allow management the opportunity to highlight those Clinical Research Associates (CRAs) and others that are star performers.
Gamification in the pharmaceutical industry has been used to improve relationships with patients by using games to encourage disease management, with Sanofi (11), Boehringer Ingleheim and Eli Lilly (12), developing apps. In the context of clinical trials, gamification presents an excellent opportunity to improve performance and reduce costs. There are a number of areas that hold promise, including; patient recruitment, patient and clinical staff retention, disease research, investigator and site training, and improving site performance.
Take the case of T.J. Sharpe. Faced with the prospect of a Stage IV melanoma diagnosis back in 2012, he vowed to never give up, determined to see his two young children grow up with a father. Working with his oncology team, T.J. identified the best possible treatment option, which at the time came in the form of a promising immunotherapy clinical trial nearly four hours away from his home. After packing up his family and relocating to Tampa, he learned shortly after his arrival that the trial was delayed due to a pending signature on a clinical trial agreement (CTA), a contract. Without the luxury of time on his side, T.J.’s new oncologist suggested that he may have to consider “plan B.” Determined not to let a document sitting on someone’s desk get in the way of a potentially life-saving treatment for him or for other patients in the study, T.J. got to work. After a long process, which involved contacting the trial sponsor, finding the right person, and telling his story, the study team resolved the startup hurdle in a timely enough manner for T.J. to receive treatment. After a long journey, which included participation in second clinical trial, today T.J. is in remission and healthier than ever. T.J.’s story is regrettably all too common – incentives could reduce these unnecessary delays.
Benchmarking would also allow for efficient resource allocation. A review of subpar performance may indicate that this is simply due to staffing issues, affording executives the option of either allocating more staff to critical steps in the progress, called crashing the schedule (13), in project management terminology or opting to incur the subsequent financial ramifications from a delayed launch to the market. Ultimately, sponsors stand to lose up to $8 million dollars daily due to a trial delaying a product’s development and launch (14).
Lastly, benchmarking is the precursor to predicative analytics or forecasting, enabling clinical research teams to estimate future outcomes based on their current state of progress. This is critical to risk mitigation and a preemptive weapon in the fight against the dreaded rescue study.
CROs, often seen as the innovators in clinical trials, are leading this charge into the BI foray. Top CROs have been aggressively acquiring data sources to leverage in data mining. In 2013, PPD acquired Acurian (15) to gain analytics-driven feasibility capabilities, LabCorp acquired Covance (16) for collective data resources to drive greater R&D productivity, and Quintiles will merge with IMS Health (17) this year to improve clinical trial execution using patient data.
Informatics is the new frontier in their innovations efforts, as they look to gain insights in operational data and drive improvements via targeted enrollment efforts. A theme that was central to the recent Disrupting Clinical Operations from the CRO Perspective (18)presentation at DPharm 2016 by top level CRO executives.
What is the common thread? We now operate in a data driven environment.
Follow T.J.’s story at www.philly.com/patient1/
Craig Morgan is a technology and life sciences management professional with more than 15 years experience in the application of informatics and bioinformatics to drug discovery. He currently heads up the marketing and brand development functions at goBalto, working with sponsors, CROs and sites to reduce cycle times and improve collaboration and oversight in clinical trials. Craig can be contacted at firstname.lastname@example.org.
1. Radical changes needed in pharmaceutical industry to improve patient outcomes, KPMG LLP UK Press Release, August 4, 2014. https://home.kpmg.com/uk/en/home/media/press-releases/2014/04/radical-changes-needed-in-pharmaceutical-industry-to-improve-patient-outcomes-says-kpmg.html
2. “Rapidly rising clinical trial costs worry researchers”, Canadian Medical Association, February 3, 2009 http://www.cmaj.ca/content/180/3/277.full.pdf+html
3. Examination of Clinical Trial Costs and Barriers for Drug Development. US Department of Health and Human Services. July 25, 2014. Available at: https://aspe.hhs.gov/sites/default/files/pdf/77166/rpt_erg.pdf
4. Getz K, Lamberti ML. 89% of trials meet enrollment, but timelines slip, half of sites under-enroll. Impact Report. Tufts Center for the Study of Drug Development. Jan/Feb2013;15(1).
5. Getz K, Lamberti ML. 89% of trials meet enrollment, but timelines slip, half of sites under-enroll. Impact Report. Tufts Center for the Study of Drug Development. Jan/Feb2013;15(1).
6. Kramer JM. Schulman KA. Transforming the economics of clinical trials. Institute of Medicine. April 13, 2012.
7. START Study Tufts CSDD-goBalto, 2012. Ken Getz’s presentation entitled: Uncovering the drivers of R&D costs.
8. Unclogging the Patient Recruitment Bottleneck, PharmaVoice, Feb. 2011 http://www.pharmavoice.com/article/2182/
9. Business Intelligence In Clinical Trial Management, Clinical Leader, May 12, 2015 http://www.clinicalleader.com/doc/business-intelligence-in-clinical-trial-management-0001
10. Research and Markets: The New 2015 Trends of Global Clinical Development Outsourcing Market, January 20, 2015 http://www.businesswire.com/news/home/20150130005621/en/Research-Markets-2015-Trends-Global-Clinical-Development#.Vh50c-xVhHx
11. Sanofi launches diabetes gaming app for children, PMLive, September 5, 2014
12. Boehringer Ingelheim and Eli Lilly & Company launch type 2 diabetes educational game, “Complications Combat”, May 24, 2013 https://www.boehringer-ingelheim.com/press-release/boehringer-ingelheim-and-eli-lilly-company-launch-type-2-diabetes-educational-game
13. Project Management Learning Series: Fast Tracking vs Crashing Simplilearn August 24, 2015 http://www.simplilearn.com/fast-tracking-vs-crashing-article
14. Drug Companies Lose Millions Due to Clinical Trial Inefficiencies, Cutting Edge Information, March 9, 2005 http://www.prnewswire.com/news-releases/drug-companies-lose-millions-due-to-clinical-trial-inefficiencies-54192787.html
15. PPD Acquires Acurian, Industry Leader in Patient Recruitment and Retention, August 27, 2013 http://www.ppdi.com/News-And-Events/News/2013/PPD-Acquires-Acurian.aspx
16. LabCorp to Acquire Covance for $6.1B, November 3, 2014 https://www.genomeweb.com/archive/labcorp-acquire-covance-61b
17. IMS Health and Quintiles to Merge, Quintiles IMS to Become Industry-Leading Information and Technology-Enabled Healthcare Service Provider, May 3, 2016 http://www.quintiles.com/news/2016/05/ims-health-and-quintiles-to-merge
18. Disrupting Clinical Operations from the CRO Perspective, DPharm 2016 http://theconferenceforum.org/conferences/disruptive-innovations-us/2016-agenda/