When I try to explain clinical science to laboratory scientists I always start by asking them the following question: “Imagine that, in doing a controlled experiment, it took two years to get approval to run the experiment, that the control arm was highly variable in its genetic background, that the experiment could only be performed once, at a cost of millions of dollars, and that if you got any of the experimental conditions wrong (drug dose, say, or schedule or duration of treatment) you would not be allowed to repeat the experiment. The experiment itself would take five or more years to run, and might succeed or fail based on whether an army of other researchers living across a continent shared your enthusiasm for the question being asked. Then imagine that you would be humiliated by being required to report the experiment’s failure in front of several thousands of your colleagues, your own personal Golgotha, instead of just moving on to the next experiment. Would you be willing to perform this experiment? A show of hands, please.”
Only the rare brave soul raises a hand, usually from somewhere in the back of the room where the rest of the group cannot take notice. And yet this experiment is exactly what clinical researchers perform every time a phase III trial is launched. Are clinical trialists brave, or just foolish?
I thought of this in reading articles in the last two issues of the New England Journal of Medicine, both on breast cancer. The first was by Dennis Slamon and colleagues, reporting the results of BCIRG-006, an important trial examining adjuvant trastuzumab strategies. The second, by Robert Coleman and colleagues, reported the AZURE trial, which tackled the issue of adjuvant bisphosphonate therapy. They are both well-conducted trials by internationally respected researchers, and both raise as many questions as they answer.
BCIRG-006 was one of the first large adjuvant trastuzumab trials, and like other previously reported trials it confirms the crucial role of this drug in the setting of HER2-positive early-stage breast cancer. The trial compared two trastuzumab-containing arms to a backbone regimen of doxorubicin and cyclophosphamide followed by docetaxel, and both were superior to the chemotherapy-alone group.
So far so good. The arguments revolve around the two experimental arms, one anthracycline-based, one based on impressive preclinical data using a platinum/taxane combination. The arms clearly vary in toxicity (particularly in the rare but important toxicity of congestive heart failure), and are not statistically significantly different with regard to efficacy, though there is a modest numerical advantage for the anthracycline-containing arm.
To Adria or not to Adria? That is the question, and it has in recent years become almost a theological dispute among breast cancer doctors, with arguments centered around biology and the history of chemotherapy trial results. My bias doesn’t matter, so I won’t share it. Nor will I get into the pros and cons here, other than to say that BCIRG-006 is simply not powered to answer the secondary question of whether one chemotherapy backbone is equivalent in efficacy to the other. Yet that is exactly the question we want to know today, when the value of trastuzumab is already widely accepted. Given infinite resources and time, every question could be answered, and if we were dealing with syngeneic mice bearing tumors derived from a single cell line, we would just repeat the experiment and have the results ready in three months. But, alas, we’re dealing with that most expensive and problematic of experimental protocols, the human phase III adjuvant trial.
Let’s switch to AZURE, which raises another problem with phase III trials: reproducibility. Previous trials, including a number of smaller phase III trials, have suggested that adjuvant bisphosphonate therapy reduces the risk of recurrence in early-stage breast cancer, and a wealth of preclinical data had provided a solid biologic rationale. Several of the clinical trials were fairly small, not all were positive, and the bisphosphonate used varied from trial to trial. AZURE (we hoped) was sufficiently large that it would be likely to answer the question once and for all.
What did it tell us? There was no difference in disease-free survival between the zoledronic acid-containing arm and the control arm for the overall group (p = 0.79, so not a hint of benefit). But the subgroup analysis, ah ha! Postmenopausal women receiving zoledronic acid had a hazard rate for recurrence of 0.75, with a p-value of 0.02.
Score! Right? Well, maybe not. A previous positive zoledronic acid trial, ABCSG-12, had suggested a significant benefit for premenopausal women, and yet there was not a hint of benefit for premenopausal women in AZURE. Real scientists adore reproducibility, but clinical trials are often annoyingly irreproducible. The authors valiantly attempt to explain the differences in the patient populations enrolled in the two trials, but the bottom line is that the two trials flagrantly disagree on who benefits. Subgroup analyses, absent some decent a priori biology-based hypothesis, are fraught with hazard. Are the results the sort of confessions data make when you torture them long enough, or do they actually point to something real and of value? We don’t know. Let me repeat that: WE DON’T KNOW. Which probably won’t prevent some partisan from running out and declaring “a new standard of care” (one of the most overused phrases in oncology).
Again, this was a well-conducted, intelligently designed trial. The problem, I submit, is the very nature of clinical trials, which are large, expensive contraptions, built on the shifting sands of clinical practice and scientific theory. You’d have to be very brave or very foolish to trust your career to the outcome of a phase III trial, yet the world never seems to run out of brave fools. I have been a brave fool myself, both as an investigator and as a cooperative group disease chair, so I understand the delusional behavior quite well.
The world would be a better place if we could do these trials more quickly, or more cheaply, or with fewer patients. Then we might be able to approach the relative purity of the lab, where the act of varying experimental conditions and demonstrating reproducibility are (relatively) easy. Quick, cheap, smaller trials are not impossible; clinical trialists are attempting all three in a variety of formats. The solutions are partly regulatory, partly statistical, and partly biologic: simplify entry criteria, speed up the trial development process, require larger relative benefits before declaring success, and make better use of surrogate endpoints tied to real clinical benefits.
But I doubt a phase III trial will ever look or act like a lab experiment: we still wouldn’t get too many hands raised. Maybe those lab guys are smarter after all. Maybe that guy raising his hand in the back of the room only wants to ask a question: do you really think you are a scientist? Well, yes, I do, actually, thank you very much.
Only the rare brave soul raises a hand, usually from somewhere in the back of the room where the rest of the group cannot take notice. And yet this experiment is exactly what clinical researchers perform every time a phase III trial is launched. Are clinical trialists brave, or just foolish?
I thought of this in reading articles in the last two issues of the New England Journal of Medicine, both on breast cancer. The first was by Dennis Slamon and colleagues, reporting the results of BCIRG-006, an important trial examining adjuvant trastuzumab strategies. The second, by Robert Coleman and colleagues, reported the AZURE trial, which tackled the issue of adjuvant bisphosphonate therapy. They are both well-conducted trials by internationally respected researchers, and both raise as many questions as they answer.
BCIRG-006 was one of the first large adjuvant trastuzumab trials, and like other previously reported trials it confirms the crucial role of this drug in the setting of HER2-positive early-stage breast cancer. The trial compared two trastuzumab-containing arms to a backbone regimen of doxorubicin and cyclophosphamide followed by docetaxel, and both were superior to the chemotherapy-alone group.
So far so good. The arguments revolve around the two experimental arms, one anthracycline-based, one based on impressive preclinical data using a platinum/taxane combination. The arms clearly vary in toxicity (particularly in the rare but important toxicity of congestive heart failure), and are not statistically significantly different with regard to efficacy, though there is a modest numerical advantage for the anthracycline-containing arm.
To Adria or not to Adria? That is the question, and it has in recent years become almost a theological dispute among breast cancer doctors, with arguments centered around biology and the history of chemotherapy trial results. My bias doesn’t matter, so I won’t share it. Nor will I get into the pros and cons here, other than to say that BCIRG-006 is simply not powered to answer the secondary question of whether one chemotherapy backbone is equivalent in efficacy to the other. Yet that is exactly the question we want to know today, when the value of trastuzumab is already widely accepted. Given infinite resources and time, every question could be answered, and if we were dealing with syngeneic mice bearing tumors derived from a single cell line, we would just repeat the experiment and have the results ready in three months. But, alas, we’re dealing with that most expensive and problematic of experimental protocols, the human phase III adjuvant trial.
Let’s switch to AZURE, which raises another problem with phase III trials: reproducibility. Previous trials, including a number of smaller phase III trials, have suggested that adjuvant bisphosphonate therapy reduces the risk of recurrence in early-stage breast cancer, and a wealth of preclinical data had provided a solid biologic rationale. Several of the clinical trials were fairly small, not all were positive, and the bisphosphonate used varied from trial to trial. AZURE (we hoped) was sufficiently large that it would be likely to answer the question once and for all.
What did it tell us? There was no difference in disease-free survival between the zoledronic acid-containing arm and the control arm for the overall group (p = 0.79, so not a hint of benefit). But the subgroup analysis, ah ha! Postmenopausal women receiving zoledronic acid had a hazard rate for recurrence of 0.75, with a p-value of 0.02.
Score! Right? Well, maybe not. A previous positive zoledronic acid trial, ABCSG-12, had suggested a significant benefit for premenopausal women, and yet there was not a hint of benefit for premenopausal women in AZURE. Real scientists adore reproducibility, but clinical trials are often annoyingly irreproducible. The authors valiantly attempt to explain the differences in the patient populations enrolled in the two trials, but the bottom line is that the two trials flagrantly disagree on who benefits. Subgroup analyses, absent some decent a priori biology-based hypothesis, are fraught with hazard. Are the results the sort of confessions data make when you torture them long enough, or do they actually point to something real and of value? We don’t know. Let me repeat that: WE DON’T KNOW. Which probably won’t prevent some partisan from running out and declaring “a new standard of care” (one of the most overused phrases in oncology).
Again, this was a well-conducted, intelligently designed trial. The problem, I submit, is the very nature of clinical trials, which are large, expensive contraptions, built on the shifting sands of clinical practice and scientific theory. You’d have to be very brave or very foolish to trust your career to the outcome of a phase III trial, yet the world never seems to run out of brave fools. I have been a brave fool myself, both as an investigator and as a cooperative group disease chair, so I understand the delusional behavior quite well.
The world would be a better place if we could do these trials more quickly, or more cheaply, or with fewer patients. Then we might be able to approach the relative purity of the lab, where the act of varying experimental conditions and demonstrating reproducibility are (relatively) easy. Quick, cheap, smaller trials are not impossible; clinical trialists are attempting all three in a variety of formats. The solutions are partly regulatory, partly statistical, and partly biologic: simplify entry criteria, speed up the trial development process, require larger relative benefits before declaring success, and make better use of surrogate endpoints tied to real clinical benefits.
But I doubt a phase III trial will ever look or act like a lab experiment: we still wouldn’t get too many hands raised. Maybe those lab guys are smarter after all. Maybe that guy raising his hand in the back of the room only wants to ask a question: do you really think you are a scientist? Well, yes, I do, actually, thank you very much.
Disclaimer:
The ideas and opinions expressed on the ASCO Connection Blogs do not necessarily reflect those of ASCO. None of the information posted on ASCOconnection.org is intended as medical, legal, or business advice, or advice about reimbursement for health care services. The mention of any product, service, company, therapy or physician practice on ASCOconnection.org does not constitute an endorsement of any kind by ASCO. ASCO assumes no responsibility for any injury or damage to persons or property arising out of or related to any use of the material contained in, posted on, or linked to this site, or any errors or omissions.
Comments
Anees B. Chagpar, MD, MPH, MBA, MSc, MA, FACS, FRCS, FASCO
Oct, 16 2011 6:49 PM
Nicely said, George. One final plea . . . while there are many trials of trialists, we all rely on data that emerges (warts and all). It is therefore incumbent upon us to support clinical trials, for there is nothing worse than investing time, energy and money – oh, and the goodwill of patients – only to have a trial that closes prematurely due to poor accrual, leaving all of us to still wonder about the answer to important questions. The ACOSOG Z-11 trial was a classic example – it closed after achieving enrollment of less than half of the required subjects. In part, it was because the event rate was so low; but in part, it was because many surgeons chose not to participate. Indeed, so many of us thought we knew what the outcome would be that we did not enroll patients to help answer the question. But, as is often the case, we rarely know what we think we do. So now we are left looking at the data and thinking – hmmm, I wonder what it means, and (how) do I incorporate these findings into my clinical practice? And so, while many of the trials of trialists cannot be avoided, the issue of poor accrual is something that can be overcome by the commitment of oncologists to put aside their biases in search of the truth . . . or as close to that as we can get.
Mehmet Sitki Copur, MD, FACP
Oct, 23 2011 5:53 PM
As usual Dr. Sledge hits the nail on the head. I can add "To Bevacizumab or not to Bevacizumab", "To Inaparib or not to Inaparib",To Eribulin or not to Eribulin" and list can go on and on. Medicine is not only a Science but is also an Art. We are not only scientists but artists. Eighty-five percent of cancer patients are diagnosed and treated in the community setting where there is a lack of infra-structure to conduct clinical trials,even if there is an infra-structure there is a huge regulatory burden to conduct clinical trials, and the last but not the least the training of medical oncoloogy fellows does not put emphasis on "think clinical trial first" concept, all making oncologists in the community less effective in enrolling their patients into clinical trials as opposed to lack of commitment due to their biases.