Tutorials: Adaptive Trial Design

Overview of Adaptive Design

Drug development is a sequence of complicated decision-making processes. Options are provided at each stage and decisions are dependent on the prior information and the probabilistic consequence of each action (decision) taken. This requires the trial design to be flexible such that it can be modified during the trial process. Adaptive design emerges for this reason and has become very attractive to pharmaceutical industries. An adaptive design is a design that allows modifications to some aspects of the trial after its initiation without undermining the validity and integrity of the trial. The following are the examples of modifications to a trial.

Due to great uncertainties of source information, mechanism of action, treatment effect and its variability, interaction with environment..., using a traditional mathematic or statistical approach makes it virtually impossible to model the dynamic system. Computer simulation is a powerful tool for achieving the ultimate goal. An overall process of an adaptive design is depicted in the following figure.

What can ExpDesign Studio do for Adaptive Design?
ExpDesign Studio allows you to simulate trials with adaptive designs. You can use response-adaptive-randomization to assign more patients to superior treatment groups or drop the 'loser' when an inferior group (loser) is identified. You may stop a trial prematurely to claim efficacy or futility based on the observed data. You may modify sample size based on observed treatment difference. You may conduct simulations using Bayesian or frequentist modeling approaches or a nonparametric approach.


Steps for Simulating an Adaptive Design
In the most complex situations, there are seven simple steps to setup your simulations using ExpDesign Studio.


Step 1: Trial Objective


There are 2 possible trial objectives: (1) to find the dose or treatment with the maximum response rate such as the cured rate and survival rate (1- death rate), and (2) To find the dose with a target rate, e.g., the maximum tolerated dose defined by the dose with a given toxicity rate.


The response rate or probability is defined as Pr (u >= c) where u is utility index and c is a threshold. The utility index is the weighting average of trial endpoints such as safety and efficacy. The weights and the threshold are often determined by experts in the relevant field. If only a single binary efficacy or safety response is concerned, the utility index u is either 0 for non-responders or 1 for responders, and the response rate is simply Pr(u =1).


Step 2: Global settings


Enter the number of simulations you want to run; number of subjects for each trial, the number of dose levels with corresponding doses and response rates. Please click the Arrow button to navigate among different dose levels. The (true) response rates can be estimated from information available. You can also input any response rates for a sensitivity analysis.


Step 3: Response model


The response rate can be modeled using Hyper-logistic function, Emax model or any user-defined function of at most 5 parameters (a1, a2, a3, a4, a5). You must use xx as the independent variable or dose in you model specification. It is critical to set appropriate parameter ranges for your model, since it will directly affect the accuracy and precision of the modeling. You can use the Graphic Calculator in the toolbar to assist you in determining the ranges by plot the functions. It is recommended using as few parameters as possible since it will greatly improve the precision of the modeling. You can choose a parameter as Bayesian parameter by checking the corresponding box next to the parameter. The response model will be updated whenever the response data become available.


Step 4: Randomization Rules


It is desirable to randomize more patients to superior treatment groups. This can be accomplished by increasing the probability of assigning a patient to the treatment group when the evidence of responsive rate increases in a group. You can choose (1) Randomized-Play-the-Winner, or (2) Utility offset model.


The cluster size is used when there is a delayed response, i.e., randomizing the next patient before knowing responses of previous patients. A cluster size of 1 indicates no response-delay. If desired, you can perform response-adaptive randomization at time of interim analyses by setting the cluster size to the increment of patients between two analyses. However, it is not a cluster randomization, because the basic randomization unit is an individual patient not a cluster of patients.


Step 5: Stopping Rules


It is desirable to stop trial when the efficacy or futility of the test drug becomes obvious during the trial. To stop a trial prematurely, one has to provide a threshold for the number of subjects randomized and at least one of the following:


(1) Utility rules: The difference in response rate between the most responsive group and the control (dose level 1) exceeds a threshold and the corresponding two-sided 95% na´ve confidence interval lower bound exceeds a threshold.
(2) Futility rules: The difference in response rate between the most responsive group and the control (dose level 1) is lower than a threshold and the corresponding two-sided 90% na´ve confidence interval upper bound is lower a threshold.


Step 6: Rules for Dropping Loser


In addition to the response-adaptive randomization, you can also improve the efficiency of a trial design by dropping some inferior groups (losers) during the trial. To drop a looser, you have to provide two thresholds for (1) maximum difference in response rate between any two dose levels, and (2) the corresponding two-sided 90% naÏve confidence lower bound. You may choose to retain all the treatment groups without dropping loser, or/and to retain the control group with a certain randomization rate for the purpose of statistical comparisons between the active groups and the control (dose level 1).


Step 7: N - Adjustment


Sample size determination requires anticipation of the expected treatment effect size defined as the expected treatment difference divided by its standard deviation. It is not uncommon that the initial estimation of the effect size turns out to be too large or small, which consequently leads to an underpowered or overpowered trial. Therefore, it is desirable to adjust the sample size according to the effect size for an ongoing trial.


To have the sample size adjusted, you have to pre-specify when and how the adjustment will be made by entering values in the above textboxes.


Note that M = initial sample size, N = new sample size, Eo_max = initial maximum treatment effect size compared to dose level 1, and E_max = observed maximum effect size.


Step 8: Bayesian Prior


If the response or utility is modeled using the Bayesian approach, you can choose one of three prior probability distribution for the Bayesian parameter in the response model, i.e., non-formative (uniform), truncated-normal and truncated-gamma distributions. The priors should be based information available at the time of designing the trial.


Response-Adaptive-Randomizations
Before we present any simulation examples, it is helpful to understand a few key concepts. The conventional randomization refers to any randomization procedures with a constant treatment allocation probability such as simple (or complete) randomization. Unlike the conventional randomization, response-adaptive randomization is a randomization in which the probability of allocating a patient to a treatment group is based on the response (outcome) of the previous patients. The purpose is to improve the overall response rate in the trial. ExpDesign Studio has implemented three different response-adaptive randomization algorithms: random-play-the-winner (RPW), utility-offset model and maximum utility model.


The generalized Randomized-Play-the-Winner denoted by RPW(n1, n2,..., nk; m1, m2,..., mk) can be described as follows:


(i) Place ni balls of ith color (corresponding to ith treatment) into a urn (i = 1, 2, ..., k), where k is number of treatment groups. There are N= ni balls in the urn.
(ii) Randomly choose a ball from the urn. If it is ith color, assign the next patient to the ith treatment group.
(iii) Add mk balls of ith color to the urn for each response observed in ith treatment. This creates more chances for choosing ith treatment.
(iv) Repeat Steps (ii) and (iii).


When ni = n and mi = m for all i, we simply write RPW(n, m) for RPW(n1, n2,..., nk; m1, m2,..., mk).

Utility-Offset Model (UOM)
To have a high probability of achieving target patient distribution among the treatment groups, the probability of assigning a patient to a group should be proportional to the corresponding predicted or observed response rate minus the proportion of patients that have been assigned to the group. This is called the Utility-offset Model.


Maximum Utility Model (MUM)
Maximum utility model for the adaptive-randomization always assigns the next patient to the group that has the highest response rate based on current estimation of either the observed or model-based predicted response rate.


Simulation Examples
Example 1: Designing a trial with two parallel groups


Suppose we are designing a trial with two parallel groups using response-adaptive randomization RPW(1,1). Assume the response rate is 0.2 and 0.3 for group 1 and 2, respectively. We want to compare this design with the common two-group balance design, i.e., RPW(1,0). The sample size required is 626 based on the Pearson Chisq-test with 80% power at alpha = 0.05. Using simulation with RPW(1,0), the design has 83% power and 156 responders with 626 subjects. The one-sided adjusted alpha = 0.02 for RPW(1,1). This adjustment is due to dependent sampling. The trial has 59% power with 626 subjects and 170 responses. The power is lost primary because of the unbalance caused by the dependent sampling procedures. The following are the basic steps to produce the results.


Step 1: Click the Adaptive Design button, and then choose the option Beginner in the panel for Experience Level in Adaptive Design. This will automatically set the trial objective To maximize response rate.


Step 2: Click on the Global settings option, then enter 10000 for the number of simulations: 626 for number of subjects for each trial, 2 for the number of dose levels and 0.2 and 0.3 for the response rates for the two dose levels, respectively. Note that actual doses are not important in this simulation, so you can enter any numbers for doses. You can click the arrow button to navigate among different dose levels.

Step 3: Click the option Randomization Rules.

Choose the option Randomized-Play-the-Winner and leave the cluster size = 1. Enter value of 1 for initial balls and additional balls for each response and each of the two dose levels.


Next, click the option Run Simulation. A graph will show up with response rate and percent of subjects at each dose level. When it is finished, click the Report icon in the tool bar to view the report.

Simulation Result section

Dose level: 1 2
Dose: 20 30
Number of patients: 174 452
Response Rate: 0.2 0.3
Mean Observed Rate: 0.18 0.30
StdDev of Obsd. Rate: 0.05 0.029

The average total number of subjects for each trial is 626. The total number of responses per trial is 170.4. The probability of correctly predicting the most responsive dose level is 0.984 based on observed rates. The power for testing the maximum effect comparing any dose level to the control (dose level 1) is 0.664 at a one-sided significant level (alpha) of 0.02. The powers for comparing each of the 1 dose levels to the control (Dose level 1) at a one-sided significant level (alpha) of 0.02 is 0.664.


Note that RPW(n,m) means Random-Play-the-Winner for the randomization with n initial balls for each arm and m additional balls added to the urn for each response. The adjusted alpha can be found through the trial-error method by entering the same response rate (e.g., 0.2) for the two groups and different alphas until the power for the maximum effect becomes very close to the alpha - the adjusted alpha for pairwise comparison.


Example 2: Design Permitting Sample Size Re-estimation


Power of a trial is heavily dependent on the estimated effect size. Therefore it is desirable to a trial design that allows modification of sample size at some time point during the trial. Let us re-design the trial in example 1 and allow a sample size-re-estimation and then study the robustness of the design.


The simulation can be characterized in to two stages. In the first stage you find the adjusted alpha. In the second stage, you use this adjusted alpha and sample size to determine the power.


Stage 1:


We keep every thing the same as in example 1, but in step 2, you create the null hypothesis condition by entering 0.2 for both dose levels. Then in step 7, you enter 100 in the textbox for Adjusted total sample size at information time, n, 0.1633 for Eo_max, 2 for parameter a and 1000 for the maximum sample size to be adjusted. You may also enter 100 for Cluster size in step 5. Now try different values for One-sided alpha in the Option panel until the power for the maximum effect (FWE) becomes 0.025, the family-wise-error (FWE). The adjusted alpha is 0.023 in present case. The average sample size is 960 under the null hypothesis.

Stage 2:


Change the response rate to the alternative hypothesis condition in step 2, i.e., enter 0.2 for dose level 1 and 0.3 for dose level 2. and run the simulation again by clicking the Run button. Clicking the Report icon to view the simulation results. You will see that the mechanism of sample size re-estimation leads to a bias in estimation of response rate.

Dose level: 1 2
Number of patients: 411 411
Response Rate: 0.2 0.3
Mean Observed Rate: 0.199 0.301
Std. Dev. of Obsd. Rate: 0.02 0.023

The design has 92.1% power with an average sample size of 821.5.

Now assume the initial effect sizes are not 0.2 versus 0.3 for the two treatment groups; instead, they are 0.2 and 0.28, respectively. We want to know the power of the flexible design. Therefore, we keep everything the same (Keep Eo_max 0.1633, which corresponds to the paired rates 0.2 and 0.3), but change the response rates to 0.2 and 0.28 for the two dose levels and run the simulation again. The key results are shown below.

Dose level: 1 2
Number of patients: 427 427
Response Rate: 0.2 0.28
Mean Observed Rate: 0.199 0.281
Std. Dev. of Obsd. Rate: 0.02 0.023

The design has 79.4% power with an average sample size of 855.


Given the two response rates 0.2 and 0.28, the design with a fixed sample size of 880 has a power of 79.4%. We can see that there is a saving of 25 patients by using the flexible design. If the response rates are 0.2 and 0.3, for 92.1% power, the required sample size is 828 with the fixed sample size design. The flexible design saves 6-7 subjects. Flexible design increases power when observed effect size is less than expected, while fixed sample increases the power uniformly regarding the observed effect size when the sample increases.


Example 3: Design a dose-response trial


The objective is to find the most responsive group. The dose-response trial has five dose levels and the following estimated response rates.

Dose level: 1 2 3 4 5
Dose: 15 30 50 85 110
Response Rate: 0.05 0.1 0.2 0.7 0.5

Using RPW(2,1) with a cluster size of 5, 100 subjects and the hyper-logistic model of the parameters shown in the figure, the simulation generates the following results.

Simulation Result Section

Dose level: 1 2 3 4 5
Dose: 15 30 50 85 110
Number of Patients: 12.0 13.0 15.1 35.1 24.9
Response Rate: 0.05 0.1 0.2 0.7 0.5
Mean Observed Rate: 0.05 0.09 0.18 0.69 0.49
StdDev of Obsd. Rate: 0.06 0.08 0.18 0.09 0.12

The average total number of subjects for each trial is 100. The total number of responses per trial is 41.8. The probability of correctly predicting the most responsive dose level is 0.919 based on observed rates. The power for testing the maximum effect comparing any dose level to the control (dose level 1) is 0.998 at a one-sided significant level (alpha) of 0.025. The powers for comparing each of the 4 dose levels to the control (Dose level 1) at a one-sided significant level (alpha) of 0.025 are 0.018, 0.206, 0.996, 0.919, respectively.


Example 4: Dose-escalation trial using Bayesian CRM


The trial objective is to determine the maximum tolerated dose (MTD) of a compound for oncology. We use continual reassessment method (CRM) to design the trial. The estimated toxicity profile is shown in the following table.

Dose level: 1 2 3 4 5 6
Dose: 20 30 40 50 60 70
Toxic Rate: 0.05 0.10 0.22 0.40 0.60 0.80

Using the logistic model with the parameters shown in the following figures and utility-offset model for the randomization, we obtain the simulation results shown below.




Simulation Result Section

Dose level: 1 2 3 4 5 6
Dose: 20 30 40 50 60 70
Number of Patients: 1.7 5.0 18.9 11.6 2.2 0.6
Response Rate: 0.01 0.10 0.22 0.40 0.60 0.80
Mean Observed Rate: 0.00 0.03 0.19 0.45 0.57 0.30
Mean Predicted Rate: 0.04 0.11 0.23 0.43 0.63 0.79
StdDev of Obsd. Rate: 0.01 0.07 0.16 0.29 0.41 0.44
StdDev of Pred. Rate: 0.01 0.03 0.07 0.11 0.11 0.09

The average total number of subjects for each trial is 40. The total number of responses per trial is 11. The average R-square is 0.25 with standard deviation of 0.341.

top