Life Science Leader Magazine Supplements

CRO Supplement 2015

The vision of Life Science Leader is to help facilitate connections and foster collaborations in pharma and med device development to get more life-saving and life-improving therapies to market in an efficient manner. Connect, Collaborate, Contribute

Issue link:

Contents of this Issue


Page 28 of 51

29 LIFESCIENCELEADER.COM THE CRO LEADERSHIP AWARDS 2015 Howley notes this is exactly what many districts are using to try and evaluate teachers. "Schools are using a value-added measurement system," he says. "Given all of the variables of a class (socio-economic status, past performance on tests, etc.), administrators can predict what stu- dents will learn. This is how you get to the essence of performance, not by comparing a class to an industry average. What if the whole industry is below average? That statistic really doesn't tell you anything. In clinical trials, a CRO could find itself in a situation where it is above average, but still underperforming." MEASURING THE QUALITY It's the customer who always determines service quality. In pharma, that is the drug sponsor. When a trial is outsourced, the sponsor assigns functional area executives to oversee specific areas of the trial. His measurement strategy has those managers evaluate how the CRO performed on areas they directly oversee. Some of the ques- tions they could be asked are: How did the project manager perform? How was their general knowledge? How was their knowledge of your specific trial? How was their GCP and regulatory knowledge? If he was attempting to evaluate recruit- ment, he might ask: How well did they do enrolling patients? How did they do on first and last patient in? How did they perform in regard to keeping you informed on how enrollment was progressing? How did they perform in enrolling patients who met your criteria? How did they perform at retaining patients once they were enrolled? "Those are informative questions that will provide more value than asking the "They do a pretty good job defining what needs to be measured, and an amazing job identifying items for what they want to measure. Where pharma companies stumble is on the last two steps. Companies do well when collaborating with each other, but do poorly when collaborating with statisticians and psychometricians." When performing the first two steps, sponsors might end up with 300 or 400 different measures, and decide to col- lect data on all of them. They then have to benchmark all of those metrics and then spend millions on a dashboard to tell them how they're performing against the averages. According to Howley, they are spending millions on software to perform 1940s-type statistics. "They are taking averages and comparing them to the mean," he says. "I tell them we can do better than that." METRICS ARE IMPORTANT, NOT RANKINGS In conducting his research and the find- ings, Howley is quick to point out his focus is on monitoring, not producing rankings. In fact, he notes companies would refuse to share their data if they knew it would result in rankings. He also believes rank- ings would do nothing to improve the level of quality in trials. By now you may be wondering why do all this work if not for the rankings. Howley's hope is to be able to perform predictive analytics. "Our focus should be on monitoring the trial as it unfolds, looking at leading indicators that may eventually lead to degradation in qual- ity," he says. "We want to try and identify quality issues before the whole trial goes off the tracks." In the future, if a CRO receives an RFP and wants to bid on a trial, Howley's research would allow the CRO to predict the quality of its trial. "Based on the data from 10,000 trials that we have in our database, we could predict what that final outcome would be," he states. "While that may sound futuristic, it's exactly what we do today in many other areas, including academia." number of days to enroll a patient," says Howley. "Days to enroll is a common met- ric because sponsors believe it is a reliable statistic. I don't agree that it is. Are people going to go back and look in their calendar for the exact date they started enroll- ing and then count forward? Even if you know it took 73 days to enroll a patient, is that good or bad? For a pediatric oncol- ogy trial, you would be a rock star. For an eczema trial, it's terrible. But you see the problem here: Any metric whose meaning depends on the individual context inevita- bly lacks validity." The industry is still a long way from adopting uniform standards on quality, but Howley believes we are moving in the right direction. Sponsors are beginning to see the value in monitoring trials for signs of trouble, and increased access to data will help researchers like him provide better information to sponsors and CROs. "Sponsors will be able to find CROs that are predicted to do well in the types of studies they are conducting and at the same time, these predictive models could help CROs focus on the types of trials where they are most likely to succeed," adds Howley. "We hope this will be a win for both sides." L Companies do well when collaborating with each other, but do poorly when collaborating with statisticians and psychometricians. M I C H A E L H O W L E Y , P H . D . Associate Clinical Professor in the LeBow College of Business at Drexel University

Articles in this issue

Links on this page

Archives of this issue

view archives of Life Science Leader Magazine Supplements - CRO Supplement 2015