Clinical research methodology, made practical

Shifting Probabilities Mid-Trial

Response-adaptive randomization allows changing the allocation probability during the study by using the data accruing as the study progresses based on patient responses.

The Play-the-Winner Rule

An example of response-adaptive randomization is the randomized play-the-winner rule. If a patient response is success, the next patient will be allocated to the same treatment group.

Ethical Appeal vs Practical Limits

This method is based on the ethics of allowing more patients to be allocated to treatments that benefit the patients. However, if clinical studies take a very long time to obtain results, this method cannot be recommended.

Why Use Block Randomization

To avoid sample size imbalances in small trials, we set blocks for randomization and balance the number of subjects in each block.

The Danger of Small Block Sizes

If the block size is 2, the allocation result of the second subject in the block can be easily predicted with a high risk of observation bias.

Protecting Against Selection Bias

Therefore, the block size used should preferably be 4 or more. To reduce the risk of predictability from the use of one block size, the size may be varied.

Balancing Prognostic Factors

Stratified randomization refers to the situation where the strata are based on level of prognostic factors or covariates, and randomization is applied to each stratum.

The Risk of Too Many Strata

If several important prognostic factors affect the outcome, the number of strata would increase. In case of several strata in relation to the target sample size, strata may be empty or sparse.

Careful Selection is Crucial

To reduce this risk, the prognostic factors should be carefully selected. These prognostic factors should be considered again during the statistical analysis and at the end of the study.

What is Minimization

Minimization was first introduced as a covariate adaptive method to balance the prognostic factors. The information of previous subjects is used to allocate newly recruited subjects.

How Allocation Works

First, the total number of imbalances is calculated after virtually allocating a newly recruited subject to all groups. The subject will be allocated to the group with lowest total number of imbalances.

The Statistical Trade-off

A concern from a statistical point of view is that minimization does not satisfy randomness, which is the basic assumption of statistical inference. Analysis of covariance or permutation tests are proposed.

An assumption rarely tested

Thousands of trials claim double-blinding, but few evaluate if it succeeded. A meta-analysis showed only 45% of double-blind trials described regimen similarity, and only 26% reported allocation protection.

How the blind breaks down

Technical masking of appearance, smell, and taste is not enough. Side effects, perceived therapeutic efficacy, or informal conversations often provide clues that unmask participants and clinicians beyond chance.

Eliciting participant guesses

The degree of blinding is assessed by asking participants at the end to guess their assignment: active drug, placebo, or do not know. This creates a simple two-by-three contingency table for statistical testing.

Why unblinding threatens validity

Unblinding can introduce ascertainment bias, alter participant compliance, and even affect ancillary care and attention, potentially compromising trial validity even when measuring objective outcomes.

Complete Randomness Has a Catch

Simple randomization allocates subjects independently, like tossing a coin. While it eliminates predictability, it rarely produces equal group numbers in trials with small sample sizes.

Imbalance Destroys Statistical Power

In a 40-patient trial, equal allocation gives 80% power. A split of 25 to 15 drops power to 77%, and a 30 to 10 split slashes it to 67%, despite enrolling the exact same total sample size.

Chance of Imbalance Drops with Size

For a 45% to 55% ratio, a trial of 40 subjects has a 52.7% chance of imbalance. That risk drops to 15.7% at 200 subjects and to only 4.6% at 400 subjects.

When to Use Simple Randomization

Simple randomization is recommended primarily for large-scale trials where high enrollment naturally protects against imbalance. Smaller studies require restricted or adaptive designs.

The traditional approach

James' blinding index adapts kappa to measure disagreement. It scores between 0 (total unblinding) and 1 (complete blinding), with 0.5 representing random guessing, placing the highest weight on 'do not know' responses.

Dominated by 'Don't Know'

When 30% to 70% of participants answer 'do not know', James' index almost never rejects the null hypothesis of successful blinding, even when all participants who ventured a guess were completely unblinded.

The fatal cancelling effect

James' index pools all arms into one score. If the drug arm correctly guesses drug, but the placebo arm also guesses drug due to optimism, the opposing patterns cancel out into a false appearance of perfect blinding.

The need for arm-specific analysis

Because blinding mechanisms and expectations differ between active and control arms, evaluating blinding must be done separately for each arm rather than collapsing everything into a single overall metric.

Arm-specific evaluation

Bang and colleagues proposed calculating a separate blinding index for each arm using a multinomial model. This answers two distinct questions: Was the treatment arm blinded, and was the placebo arm blinded?

Understanding the -1 to +1 scale

The index ranges from -1 to 1. Zero represents random balance under perfect blinding. Positive values mean excess correct guesses beyond chance, while negative values indicate participants mostly guessed the other arm.

Direct proportion interpretation

The index value directly estimates the proportion of unblinded patients beyond chance. An index of 0.20 in the active arm means exactly 20% of participants correctly identified their treatment beyond random chance.

When to flag a blinding issue

Authors suggest reporting potential unblinding if the index exceeds 0.20, if statistical testing rejects the null hypothesis of random guessing, or if Bang's and James' indices yield conflicting conclusions.

The Search for Evidence on PICO

Few studies have been performed to evaluate the effectiveness of PICO or its variations in identifying relevant research articles in evidence-based practice.

Pilot Trials and Precision

A pilot randomized trial found that searches using PICO resulted in higher percentages of relevant citations, but the findings did not reach statistical significance.

Quick Searches in PubMed

Researchers investigated the PICO format usefulness for quick searches in PubMed and found that using PICO did not improve the results for quick searching.

What Systematic Reviews Show

A systematic review aimed to determine if PICO improved search quality, but could only include three articles and concluded that evidence was lacking.