Share

Clinical Trials

6. Erroneous Trial Results

In a clinical trial, the observed treatment effect regarding the safety and efficacy of a new drug may represent the ‘true’ difference between the new drug and the comparative treatment or it may not. This is to say that if the trial were to be repeated with all the available patients in the world then the outcome would either be the same as the trial (a true result) or different (making the trial result a chance event, or an erroneous false result). Understanding the possible sources of erroneous results is critical in the appreciation of clinical trials.

    Reasons for erroneous results fall into three main categories.
  • The trial may have been biased in some predictable fashion.
  • It could have been contaminated (confounded) by an unpredictable factor.
  • The result may simply have occurred by random chance.

Example 1

A cinnamon-based herbal oil reduced breast pain in women compared to evening primrose oil. Commercial oils were used for the study. The new cinnamon oil was provided free to all participants, while the primrose oil needed a prescription to be filled by the patient.

In this example, there are several sources of potential bias, including:

  • Trial not blinded;
  • New medications are appealing;
  • False safety impression;
  • Impressions based on age;
  • Patient drop out; and
  • Self-fulfilling prophecy.

The first source is not blinding the trial. This could result in bias because if the trial is not blinded, it is easy to know which oil women were on, resulting in observer bias and volunteer bias in terms of recording and reporting breast pain. New medications can be a source of bias because they are appealing and they usually attract positive attitudes from patients and, more importantly, physicians, especially those in a trial. This is often referred to as observer’s bias. Side effects of newer medications are not as extensively known or documented often giving a false impression of safety. This can be referred to as information bias. Impressions based on age can be a source of bias because younger, healthier patients are more likely to participate in the study and appreciate new products rather than the skepticism of new products that is often found in older patients. This is an example of selection bias. A confounding treatment effect can be caused by imbalances in subject distribution by treatment group. Non-blinded studies may not have balanced groups if people drop out if chosen for the prescription therapy arm. Another source is known as the self fulfilling prophecy effect. This is when physicians themselves may influence patients if they know which therapy a patient is receiving and may capture or record patient experiences during the trial with their own “pre-judgement” biases. This is also an example of observer’s bias.