The State of Oncology Trial Design: Where Protocols Fail, and Why the Fix Is Human

← Back to Knowledge Hub

Executive summary

Oncology drug development is a sequence of consequential decisions made under uncertainty. The most impactful are made early (in protocol design, endpoint selection, and regulatory strategy), when the least is known and the cost of being wrong is highest. The evidence bears this out: in a 2022 industry benchmarking study, 91.1% of oncology protocols had at least one amendment,¹ and across therapeutic areas, implementing a single amendment took an average of 260 days from identifying the need to final oversight approval.² Amendments are the visible symptom; the underlying disease is a design decision that was never adequately pressure-tested. This paper defines that problem, then describes how IMPACT Oncology, Atheris Group’s integrated oncology drug-development offering, is built to address it.

We have deliberately kept our central claim narrow: de-risking oncology development requires expert practicing-oncologist and clinical trialist judgment applied at the point at which decisions are made, accelerated by proprietary intelligence, rather than delegated to it. The technology synthesizes precedent and surfaces risk at a speed traditional consulting cannot match, with qualified experts deciding what this means. That combination is the offering. Our aim across the following sections is to de-risk, accelerate, and strengthen oncology programs, and to be candid about where the limits of any such approach lie.

Why Oncology Trials Struggle at the Design Table

Oncology drug development failures are rarely the result of a single catastrophic error. More often they trace back to a design decision that looked reasonable in isolation, was locked into a protocol before it was fully stress-tested, and then propagated through the study design, statistical analysis plan, regulatory strategy, and label ambition until it could no longer be corrected without an amendment, a delay, or a missed endpoint. We have seen this pattern in every post-amendment debrief we have run. The trigger is almost never exotic. It is an eligibility criterion that was broader than the biology justified, an endpoint chosen for speed rather than for what regulators would accept as evidence of benefit, or a statistical hierarchy that spent its alpha on the wrong comparison. The cost is measured in time and in trial failure, not merely in rework.

That benchmark is worth sitting with. An amendment is not a routine administrative act. It is a signal that a question which should have been resolved before activation was instead resolved after patients had already been enrolled against the original design. Four forces make these questions harder to get right in oncology than in almost any other therapeutic area.

Escalating design complexity

The modern oncology protocol is a more complex object than it was a decade ago. Biomarker-enriched populations, basket and umbrella architectures, adaptive dose-finding, and seamless phase transitions have become common rather than exceptional. Novel endpoints, such as circulating tumor DNA (ctDNA) dynamics, minimal residual disease, patient-reported outcomes, and health related quality of life, carry regulatory expectations that are still being written. Sponsors can justify each of these choices on its own scientific merits; the difficulty is that they interact. A biomarker enrichment strategy changes the population, which changes the event rate, which changes the power calculation, which changes the analysis timing, which changes what the statistical hierarchy can support. A design is a system, and systems fail at the seams where decisions interact. Decisions that are sound in isolation can, in a complex system, cause cascading damage to trial execution and results.

Endpoint and statistical-hierarchy fragility

Nowhere is this more acute than in the endpoint and statistical structure. Co-primary endpoints, hierarchical testing, and alpha allocation with recycling across a family of comparisons are now standard machinery in registrational oncology trials, and they leave little margin for error. Alpha spent early on an interim analysis or a progression free survival (PFS) endpoint is alpha unavailable to a later overall-survival (OS) comparison; a testing hierarchy that gates OS behind PFS can strand a real survival signal that fails to reach significance because the gatekeeper did not. Utilising & allocating alpha spend to a PFS endpoint to test for a mechanism of action that does not improve PFS, but routinely improves OS, e.g. immune checkpoint inhibition in solid tumours, will hamstring a study irreversibly, missing a potentially revolutionary drug. The ICH E9(R1) estimands framework has raised the bar further, requiring sponsors to specify precisely what treatment effect they intend to estimate, including how intercurrent events such as treatment discontinuation or subsequent therapy are handled.³ An estimand mis-specified at design becomes an analysis that answers a question no one asked. These are not abstractions; they are the exact decisions that determine whether a trial testing a drug with a clinically meaningful benefit to patients can be read as truly positive.

Evolving and jurisdiction-specific regulatory expectations

Regulatory expectations are moving, and they are not always in lockstep across varying regions. Agencies have signaled heightened scrutiny of externally controlled and single-arm designs, with FDA draft guidance setting out the considerations sponsors should address when a concurrent randomized control is not used.^4,5 For products reaching market through accelerated approval, expectations around confirmatory trials, including that they be well underway at the time of approval, have sharpened, with randomized designs now explicitly preferred.^4,6 And requirements to plan for representative enrollment have taken statutory and guidance form, though the specifics remain in flux.⁷ We flag these as evolving and, in places, jurisdiction-specific rather than settled; a design judged sound against one agency’s current thinking may draw questions from another, and guidance that is in draft today may read differently tomorrow. This is precisely the kind of moving target that punishes a protocol frozen too early.

Cross-functional misalignment

The final force is organizational rather than scientific. Clinical designs a trial; regulatory later flags a pathway problem; medical affairs realizes the label and evidence-generation strategy was never aligned with either; meanwhile, experienced clinical input often arrives too late to change the design cheaply. Each function is competent; the misalignment lives in the spaces between them, and it compounds downstream. A protocol is where clinical, regulatory, and medical strategy either converge or quietly diverge. When they diverge, the divergence is usually discovered at an amendment, an information request, or an oncology drug advisory committee, at which point it is expensive or impossible to fix.

Table 1. The recurring drivers of oncology protocol amendment and late-stage failure.

The common thread is timing. Every one of these problems is cheaper to solve before activation than after, and each becomes an amendment, with an average implementation time of 260 days,² when it is not. Much of the information needed to catch these issues early is proprietary rather than public, and what is public is scattered across registries, the literature, and regulatory precedent. The question is whether that information is brought into focus, by the right people, at the point the decision is actually being made.

The IMPACT Oncology Methodology: Intelligence in Service of Judgment

IMPACT Oncology is not a platform or a product. It is an integrated consulting service offering, in which senior specialists with direct oncology program experience are accountable for every recommendation, where proprietary intelligence technology is used to make those specialists faster and more thorough. This is not to replace their judgment, but to augment their insights, accelerate their efficiency and output, and expand their impact and scope. The distinction matters, and we will be precise about it, because a sophisticated audience is right to be skeptical of any claim that software can adjudicate an oncology design decision. It cannot, and we do not ask it to. What follows is how the methodology is assembled.

A clinical intelligence layer built for oncology protocol review

At the technical core is a clinical intelligence layer built specifically for oncology protocol review; not a general-purpose model given an oncology prompt. A protocol, its amendments, and the associated regulatory and clinical record are ingested and parsed into structured fields rather than treated as undifferentiated text: eligibility criteria, endpoint definitions, the statistical hierarchy, safety-monitoring rules, and version history are each extracted into their own representation. Each client’s material is indexed separately and in isolation; nothing is pooled across engagements or used to inform another client’s work. A modular review engine then examines the protocol along defined axes (an eligibility module, an endpoint module, a safety module, etc.) and returns findings that are classified by severity so that a reviewer’s attention goes first to what matters most. The intelligence layer also generates a protocol risk score, a summary index of design risk used to prioritize reviewer attention; its limits are set out at the end of this section. Every finding is traceable. It resolves to a retrievable source (a specific protocol section, a registry record, or a passage of regulatory guidance) so that a reviewer can see not only what was flagged but on what basis. A flag that cannot be traced to a source is not a finding; it is a guess, and we treat it as one.

Landscape benchmarking against the primary record

A design decision is only as good as the precedent it is measured against. The intelligence layer benchmarks a protocol’s choices against international trial registries and the published literature, assembled from primary records, which we define as the accumulated memory of what has been trialled in an indication, what has read out, and how programs were modified in response. Public registries and bibliographic databases together catalog hundreds of thousands of interventional studies and tens of millions of citations, and continuous access to them means precedent can be surfaced in days rather than weeks. We are candid about the limits of this source, because our reviewers rely on it and a careful reader should know them: registry coverage is incomplete, entries vary in quality and currency, and registration is not the same as conduct or completion. A registered trial may never have enrolled; a listed endpoint may have changed, with delays in what was made publicly available. Benchmarking against the registry tells you what was planned and reported, not necessarily what happened. We treat registry-derived precedent as a starting point for expert review, not as ground truth.

The KOL-informed reasoning framework

This is the part competitors cannot shortcut, and it is where the offering earns its keep. Risk in oncology is indication-specific, and only expert clinicians and clinical trialists who practice in the disease can define it. Practicing oncologists, global key opinion leaders on trial design, on the Atheris team define what “risk” means for a given indication. The intelligence layer is directed to pattern-match on the specific design choices that have drawn FDA questions, advisory-committee scrutiny, or confirmatory-trial difficulty in that indication, not on generic trial-design principles. The difference is concrete. A general tool flags that “eligibility criteria are broad.” Our reasoning framework flags that eligibility is broad, identifies why that specific breadth has generated agency questions in comparable programs in this indication, and points to how it might be narrowed without sacrificing enrollment feasibility. One is a grammar check. The other is clinical reasoning.

Human-in-the-loop accountability

Nothing reaches a client as raw model output. A named KOL practicing oncologist reviews every finding and accepts it, rejects it, or reframes it before it becomes a recommendation. This is not a courtesy step; it is the mechanism by which the offering works and the reason Atheris is willing to put its people’s names on the advice. The intelligence layer handles the evidence-intensive, time-consuming synthesis (reading a several-hundred-page, multi-version protocol against thousands of precedents), which frees the specialist to spend their time on the judgment. Judgement that requires being a principal investigator, steering committee member and at the front line delivering complex cancer care. Where an output touches statistical evaluation, regulatory strategy, or clinical decision-making, the relevant qualified expert (a biostatistician, a regulatory lead, a practicing oncologist) must validate it before it is used. Accountability sits with people, not systems. These oncologists, alongside specialists in biostatistics, regulatory affairs & pharmaceutical sciences, become an embedded, forward deployed, clinical development team, through Impact Oncology, within your organisation.

A walled-garden evidence model

Finally, the evidence the system reasons over is curated rather than open-ended. Outputs are generated against a controlled set of sources with direct citation back to those sources, an approach analogous to the curated, citation-first clinical evidence tools. Constraining the evidence base and requiring direct citation materially reduces the risk of fabricated or conflated findings (the failure mode that makes general-purpose tools untrustworthy for clinical work) and gives reviewers the confidence that what they are adjudicating is real and checkable. It is easier to trust a system that can only cite what it can retrieve.

How the work changes

Table 2. The same protocol-review work, before and with IMPACT Oncology. Time moves from synthesis to judgment.

What the methodology does not claim

A paper written to earn a skeptical reader’s trust has to state its bounds, and we regard the following as credibility rather than weakness. The protocol risk score produced by the intelligence layer is a heuristic triage index designed to direct reviewer attention; it is not a validated instrument, and it should not be read as one. There is, at present, no completed prospective validation study of the system’s outputs; claims of predictive performance would be premature and we do not make them. The system is decision support for qualified reviewers; it does not render autonomous clinical, statistical, or regulatory judgment. Formal statistical evaluation (estimand construction, power and sample-size calculation, definition of analysis populations) is currently out of scope for the intelligence layer and requires a qualified biostatistician, who is an integral part of the Impact Oncology team. And the current reviewer input reflects a specific regulatory jurisdiction and a particular tumor-group weighting; findings should be interpreted with that scope in mind and revalidated when applied elsewhere. Any output that would inform a regulatory or clinical decision must be validated by the appropriate licensed expert before use.

An Illustrative Case: A First-Line Metastatic NSCLC Immunotherapy Trial

The following is a conceptual illustration, not a performance claim. It draws on the publicly documented history of a first-line, locally advanced or metastatic non-small-cell lung cancer (NSCLC) immunotherapy trial in EGFR/ALK wild-type patients (anti–PD-L1 monotherapy and an anti–PD-L1 + anti–CTLA-4 combination, versus platinum-based standard-of-care chemotherapy). We have removed sponsor, product, and trial identifiers deliberately; the point is not the asset but the design pattern, which recurs across the immunotherapy era. We use it to show how the failure modes of Section 1 present in a real document, and how the methodology of Section 2 would engage them. We make no claim that any tool would have predicted this trial’s result.

The design’s inherent risk profile

The trial was a Phase III, randomized, open-label, three-arm study, allocating patients 1:1:1 across anti–PD-L1 monotherapy, the anti–PD-L1 + anti–CTLA-4 combination, and platinum-doublet chemotherapy, stratified by PD-L1 status and histology, with response assessed by RECIST 1.¹ under blinded independent central review (BICR).⁸ Several features of this design carry inherent risk that an experienced reviewer would flag before activation. Three arms with distinct biological rationales multiply the interpretive burden and the multiplicity problem: the monotherapy and the combination are different hypotheses, and the trial must spend its alpha across both, therefore somewhat starving either. The enrichment strategy, PD-L1 tumor-cell (TC) expression at a defined threshold as the key biomarker for selecting benefit, is a reasonable bet for anti–PD-L1 activity, but a weaker theoretical basis for predicting incremental benefit from CTLA-4 blockade, whose mechanism is not straightforwardly PD-L1-dependent. And the reliance on PFS as an efficacy pillar sits uneasily with what was already known about immuno-oncology agents: delayed separation of survival curves, continued treatment beyond radiographic progression, and the absence of established PFS-to-OS surrogacy all blunt PFS as a read on immunotherapy benefit. None of these observations requires hindsight. They are the standard questions a practicing thoracic oncologist asks of an immunotherapy design.

How the protocol evolved across amendments

The crux of the case is what happened to the endpoints between versions. The original protocol specified PFS as the sole primary endpoint, comparing the combination with chemotherapy in an all-comers population regardless of PD-L1 status. The operative later protocol was materially different: OS as the primary endpoint for the monotherapy comparison and co-primary OS and PFS for the combination comparison, assessed in the PD-L1 TC ≥ 25% enriched population, with the statistical framework rebuilt around co-primary objectives and a hierarchical alpha-splitting and recycling scheme spanning the survival analyses across the two experimental arms. This is not a cosmetic revision. It changes the population, the endpoints, the multiplicity structure, and the very question the trial is powered to answer. A reader who understands only the first version understands a different trial from the one that was actually run.

Table 3. Amendment-driven endpoint and population evolution. The trial that was analyzed was not the trial that was originally designed.

How a general-purpose tool mishandles this document

This is exactly the document on which general-purpose AI tools fail, and we tested it. Run against the full multi-version protocol (several hundred pages), a general model latched onto the earliest endpoint structure and summarized the trial as if PFS were the sole primary endpoint, missing the amendment-driven shift to co-primary OS and PFS entirely. In a separate pass, extraction failed outright: the document’s size exceeded practical context token limits, and the assessment did not complete. Both failure modes are instructive. A tool that summarizes the wrong protocol version will confidently describe a trial that no longer exists; a tool that cannot ingest the whole document will silently omit the amendments where the real risk lives. For a reviewer relying on the output, either error is worse than no output at all, because it looks like an answer.

How the IMPACT Oncology approach would engage it

The methodology of Section 2 is built for precisely this document. Amendment traceability treats version history as a first-class field, so the endpoint change is surfaced rather than flattened. Endpoint-evolution tracking makes the shift from sole-primary PFS in all-comers to co-primary PFS/OS in the enriched population an explicit, source-linked finding. Statistical-hierarchy extraction captures the alpha-splitting and recycling structure and flags it for review, while deferring the formal adequacy assessment to a qualified biostatistician, per the scope limits stated above. And oncologist adjudication puts the enrichment assumption and the reliance on PFS in front of a practicing thoracic KOL oncologist, who can weigh them against the indication’s regulatory history before they are locked in. The value is not that the software renders a verdict. It is that the right questions reach the right expert early enough to matter.

The Integrated Impact Model: Three Pillars, One Accountable Team

Everything above is an expression of a single operating model. IMPACT Oncology is a direct application of Atheris’s Integrated Impact model, delivered through three interdependent pillars: Thought Leadership, Clinical Expertise, and Human-Driven Intelligence. The pillars are not a menu; they are designed to work together, and the way they combine is critical.

Thought Leadership

Decades of expert consulting experience across clinical development, medical affairs, and regulatory strategy: end-to-end strategic and scientific advisory support from IND-enabling work through approval, with fractional leadership (chief medical officers, regulatory leads, medical monitors) available on demand. This is the strategic frame within which a protocol decision is understood as a program decision.

Clinical Expertise

Practicing Key Opinion Leader (KOL) oncologists and leading clinical trialists are embedded in the work, bringing real-time clinical insight and peer credibility at every stage: active patient-care experience informing protocol design, endpoint selection, and therapeutic strategy across major tumor types, together with the KOL and site-level relationships that strengthen investigator confidence. This is the pillar that defines what risk means in a given indication, and it remains the primary differentiator of the offering.

Human-Driven Intelligence

The proprietary intelligence technology described in Section 2: rapid protocol scanning, source-traceable risk detection, and competitive and landscape intelligence, run on secure, governed infrastructure. Its role is explicit and bounded: it accelerates the consultants, and it does not replace them.

The combination is what distinguishes the model. The thought leaders and clinical experts drive and leverage the intelligence technology; the technology does not drive them. It handles the evidence-intensive synthesis that would otherwise consume senior experts’ time, which frees those experts to spend their judgment where it is required: on the design calls that determine whether a program succeeds. A firm that led with the technology would have the relationship backwards. We lead with the people, and we say so plainly: senior specialists are named and accountable for every deliverable, and the people a client meets at the start of an engagement are the people delivering the work throughout. That is how Atheris operates, not a selling point.

The commercial model follows from this. Engagements can start small (a single high-stakes protocol, a specific regulatory question, a pre-IND meeting, one inflection point) and expand as trust and scope grow, with the strategic context carrying forward without interruption. Pricing reflects scope, complexity, and expected outcomes. There is no platform charge and no license fee, because IMPACT Oncology is not a platform to be licensed; it is expert work, accelerated. The offering is particularly suited to emerging and mid-size sponsors who need senior integrated clinical, medical, and regulatory expertise at the moments it matters most, without building it permanently in-house.

A closing note

In oncology development, the right early decisions pay off; the wrong ones exact their cost slowly. Nine in 10 oncology protocols are amended at least once,¹ and implementing a single amendment takes an average of 260 days²; that time compounds into missed windows and, too often, into late-stage failures that were set in motion at the design table. The information needed to catch these problems early largely exists. What has been missing is a way to bring it to the surface, to the right people, at the point the decision is actually being made. That is the problem IMPACT Oncology was built to solve, and the standard against which we would ask to be judged. For teams facing a consequential design or regulatory decision, we would welcome the chance to pressure-test it with you.

References

Formatted in AMA style. Citation metadata should be confirmed against the source of record before publication.

1. Botto E, Smith Z, Getz K. New benchmarks on protocol amendment experience in oncology clinical trials. Ther Innov Regul Sci. 2024;58(4):645-654. doi:10.1007/s43441-024-00629-2
2. Getz K, Smith Z, Botto E, Murphy E, Dauchy A. New benchmarks on protocol amendment practices, trends and their impact on clinical trial performance. Ther Innov Regul Sci. 2024;58(3):539-548. doi:10.1007/s43441-024-00622-9
3. International Council for Harmonisation. ICH Harmonised Guideline E9(R1): Addendum on Estimands and Sensitivity Analysis in Clinical Trials to the Guideline on Statistical Principles for Clinical Trials. Adopted November 20, 2019.
4. US Food and Drug Administration. Clinical Trial Considerations to Support Accelerated Approval of Oncology Therapeutics. Draft guidance for industry. March 2023. Accessed October 8, 2026. https://www.fda.gov/media/166431/download
5. US Food and Drug Administration. Considerations for the Design and Conduct of Externally Controlled Trials for Drug and Biological Products. Draft guidance for industry. February 2023. Accessed October 8, 2026. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/considerations-design-and-conduct-externally-controlled-trials-drug-and-biological-products
6. US Food and Drug Administration. Expedited Program for Serious Conditions: Accelerated Approval of Drugs and Biologics. Draft guidance for industry. December 2024.
7. US Food and Drug Administration. Diversity Action Plans to Improve Enrollment of Participants from Underrepresented Populations in Clinical Studies. Draft guidance for industry. June 2024. Accessed October 8, 2026. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/diversity-action-plans-improve-enrollment-participants-underrepresented-populations-clinical-studies
8. Eisenhauer EA, Therasse P, Bogaerts J, et al. New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1). Eur J Cancer. 2009;45(2):228-247. doi:10.1016/j.ejca.2008.10.026