Embed Size (px)
Citation preview
1
Controlling Errors in Medical Studies:
Overview李世昌
銘傳大學 應用統計與資訊學系
December 01, 2011
2
Agenda
• Statistical Designs in Medical Studies
• Selection of the Control Group • Sample Size Determination• Randomization• Statistical Analysis• Quality Assurance (QA)• Non-inferiority Trials• International Conference for
Harmonisation
3
Experimental Errors
• Controlling and Minimizing– Quality by Design– QA in Medical Studies
• Controlling the Errors in Industry and Medical Sectors– Drugs/medical devices/vaccine/…– Clinical Trials
• Why? How?
4
Pharmaceutical Industry
• Research & Development – Non-clinical Studies
•Search for treatments•Lab studies
– Pre-clinical Studies•Animal studies•Pharmacokinetic (PK) studies
– Clinical Trials•Human studies
5
Objectives versus Conclusions
• The Essence of Rational Medical Study is to Ask Important Questions and Answer them With Appropriate Studies – A study should be designed, conducted and
analysed according to sound scientific principles to achieve their objectives; and should be reported appropriately
• Statistical Approach in Design and Analysis
6
Inferential StatisticsA Population of the Random Variable of X
A Random of Sample Size of n
Descriptive Estimates andStatistical Analysis
Results and Conclusions
7
Inferential Statistics
• A Population of Subjects– A characteristic of interest, X ~ F(x; )
• A Random Sample of Size n– Each size of n in a population has an equal
probability to be selected – Θ ~ G(x; )
• Descriptive and Statistical Methodology – Graphs/charts, estimates, confidence
intervals, tests of hypothesis– Statistical Models
^
8
A Medical StudyAll Patients with a Specific Disease
A Study Group of Patients
Clinical Evidence of the Treatment
Clinical Conclusions
The Treatment of the Disease
9
Statistics versus Medical Studies
• The Human Experiment? • A Population? A Specific Disease?• The Treatments of Disease?• Clinical Response and Indices?• Staggered Entry?• Sample Size?• Randomization?• …
10
Human Experiments
• Medical Ethics?– Placebo? Standard/New Treatment? – Informed Consent Form
• Patient Benefit and Risk?– Efficacy and Safety Issues
• Clinical Evidence?
11
Complicated Issues
• A Specific Disease to Study?
• A New Treatment of a Disease– Any current standard treatments?
• How to Quantify a Clinical Benefit and Minimized Adverse Effects?– Efficacy and safety
12
Existence of Errors and Bias
• Patients• Investigational Team• The Treatment• Clinical Instrument and
Measurement• Unknown Factors• …
13
Statistics in a Medical Study
Study Objectives: Clinical indexes, efficacy variables/endpoints
Conduct Trial and Collect Data:Ethic, accurate, validate, and reliable data
The Analysis and Results:Interpretations and Conclusions
Protocol and Design:Clinical and design issues
14
The Distinction• Medical Studies
– Objectives IRB/DOH* Conduction Publication (literature review)
• Clinical Trials– Objectives IRB/DOH Conduction
NDA+/Marketing/Publication
• Statistics and Regulatory Issues (IRB, DOH, CDE ++, …)
*IRB/DOH: Institutional Review Board/Department of Health
+NDA: New Drug Application++ CDE: Center of Drug Evaluation
1515
Medical Studies and Clinical Trials
• Regulated Studies or Trials? • Clinical Trials
– A medical study sponsored by a pharmaceutical company or …
– A system of combing the variety of expertise
– New Drug Application (NDA) oriented– Declaration of Helsinki
• Trial Quality Assured?
16
Regulatory History in Medical Studies
• Declaration of Helsinki– Ethics and Science
• Medical Journals– Requirements on a submitted manuscript
• Regulatory Agency– US/Food and Drug Administration (US/FDA), …– Department of Health (DOH), Taiwan (TFDA)
• Nonprofit Organizations– NIH, CDC, NCI, …– Cancer center (MD Anderson, Mayo, Johns-Hopkins,
Sloan-Kettering, …) • International Conference on Harmonisation
(ICH)– Guidelines on Efficacy, Safety, Quality, Multi-
discipline
17
Quality Assurance
• A System of a Process of Tasks being Done– Designing, monitoring, documenting,
organizing, analyzing, and concluding
• Medical Research– Ethics + IRB + Journal Review
• Clinical Trials– Ethics + IRB + Regulations + ICH +…
18
QA: Concepts
• Quality Assurance (QA)– The systematic monitoring and evaluation
of the various aspects of a process and management to maximize the probability that minimum standards of quality are being attained by the entire process
(1) The Intended Purpose(2) Minimize the Errors and Bias(3) Systematic Approach(4) Valid and Reliable Conclusions
19
QC: Quality Control
• Statistical Quality Control (SQC)– Accuracy of specifications– Integrity and precision
• Total Quality of Management– QC + QA + SOPs– Monitoring and Auditing
20
QA: Clinical Trials
• A Well-designed Protocol• Study Conduction and Adherence• Documentation• Data Management• Analysis and Interpretations
• Regulations– International Conference on Harmonisation
(ICH)– Taiwan Food and Drug Administration (TFDA)
21
QA: Education and Resources
• Education – Trainings and Experience– Academic Education– Vocational Education and Training
• Resources– Industry sector – Government sector– Scientific expertise
22
Statistical Methodology
Point EstimateInterval EstimationTest of HypothesisStatistical Models
23
Three Basic Statistical Methods
• Point Estimation– No valid statement is made
• Interval Estimation– (1-)100% confidence of correctness – The upper and lower bounds for estimation
• Test of Hypotheses– Two hypothesis (Ho: no effect vs. Ha: effect
size)– Type I error rate – The power of test (1-)
• Practical Meanings?
24
Analysis of a Sample Data
• Is the Variable Well-defined?• How are the Sample Data
Collected From?• Whether the Sample Data
Represent the Study Population?
• What is the Appropriate Analysis?• How to Interpret the Results?• Do the Conclusion Validated?
25
A Medical Study?
• A Population of Patients?• A Group of enrolled Patients?• Are the Collected Clinical Data
representative?• How to Reach the Scientific Evidence?• Are the Clinical Conclusions Valid and
Reliable?
• Statistical Tools!!!• How to Use the Statistical Methodology?• How to Accomplish the Scientific Evidence?
2626
Design a Protocol
• Objectives– Efficacy and/or Safety– Primary/secondary variables
• Important Elements– Controlled?– Number of patients?– Randomization and blindness?– Statistical methodology?
27
Selection of a Control Group
• Purpose– Minimize the bias in assessing the
effect of test treatment
• Choice of a Control Group– Placebo or no treatment– Active control– Historical control
28
Types of Comparisons
• Superiority– Treatment A is better than
treatment B
• Bioequivalence– Treatment A is equivalent to
treatment B
• Non-inferiority– Treatment A is not inferior to
treatment B
2929
A Population and A Sample
• A Population of Patients– Objectives + inclusive criteria
• A Sample Clinical Data Sets– Number of patients
•Enrolled? Evaluable?
– Exclusive criteria•Safety issues, …
30
A Population Model
All Patients Control patients Test patients
A random samplefrom control patients
Clinical data of two groups
Statistical Analysis
+
A random sample from test patients
31
An Invoked Model
All PatientsA subgroupof patients
Control group Test group
A random sample from control group
Statistical Analysis
A random sample from test group
32
Minimise Bias/Error and Assess Efficacy
• Statistical Principles and Data Integrity – Selection of a control treatment– Sample size determination– Patients recruitment– Randomisation – Blinding – Compliance
33
Clinical Designs• Comparisons of Two Treatments (T vs. A)
– Equality• Ho: TA=0 vs. Ha: TA 0
– Superiority • Ho: TA=0 vs. Ha: TA>0
– Equivalence • Ho: TA L or TA U vs. Ha: L< TA < U• Bioequvalence (BE) studies
– Non-inferiority • Ho: |TA| M vs. Ha: |TA|< M
• Designs– Parallel (two independent samples)– Crossover (blocking samples)– Factorial (many independent samples)
34
Clinical Endpoints
• Scientific Evidence?– Valid Conclusions?– Primary or Secondary?
• Statistical Concerns– Type I error rate ()– Power– Multiplicity adjustment of
3535
Inclusions/Exclusions
• Define a Population and a Sample Data– Clinical judgment?– Might involve in
violations/deviations•Ethics and selection bias
– Sample size• Intent-to-treat (ITT) and per protocol
(PP)
36
Examples
• 1. Parallel Design– Two independent samples
• 2. Cross-over Design – Paired samples
• 3. One-way Analysis of Variance– Comparison of more than treatments
• 4. …
37
Number of Patients
Formulas and ChartsPractical Meanings
3838
Number of Patients
• Sample Size Determination– Information oftreatment
difference– Conclusion
• Intent-to-Treat/Per Protocol data set
• Inclusion/exclusion• Violation/deviation
39
Sample Size Calculation
• Primary Endpoint• Precision or power approaches
– Parameters: Type I error rate, power, variance, margin of error
– Formula or charts
• Consideration in survival trials
40
Formulas
• Precision Power ( z2/)2 (22) (z2/+z)2 (22
• n= ---------------- ; n=
--------------------- [D0 – Da]2 [D0 - Da]2
• Practical Meanings?• Survival Studies?
41
Power Approach
42
Randomization
Patient AllocationsImbalance Issues
Prognostics
43
Randomization
• Tradition– A random sample of size n from a
population– Completely randomized design of
Analysis of Variance (ANOVA)
• Clinical trials– Complete randomization– Randomization using prognostic
factors
44
Maximum Power
• Equal Sample Size for t-test• Balance Issue in Analysis of
Variance
• Simple Randomization• Bias Coin Randomization• Stratified Randomization
4545
Randomization
A Sequence of Random Numbers which a Treatment Assignment is based on – Code, date, and time-point
1. Non-Adherence – Human error– Training problem– Management problem – …
2. Examples
46
Simple Randomization
• Random Number Generator– No prognostic factor considered– Predictability– Balance of treatment groups
• Example: Treatments A and B– A sequence of random number generated by a
validated computer software– 1 8 6 2 6 3 5 8 7 0 …– Assign A if the random digit is 1-5, otherwise
assign B
• Imbalance between Treatments A and B– P[2:8] ≥ 0.05; P[40:60] ≥ 0.05; P[469:531] ≥ 0.05
47
Biased Coin Design
• At Each Treatment Assignment, Assign the Least Treatment with a Higher Probability– Say, if D(i)=|n(A)-n(B)|≥ 2, then assign
the treatment to the least number with p=2/3 or 3/5
– If D(i)=0, then use p=1/2 to assign treatment
4848
Random Permuted Blocks
• Patient No. Treatment 1001 A With a
block size of 4 1002 A 1003 B 1004 B 1005 A 1006 B 1007 A 1008 B 1009 B 1010 B 1011 A 1012 A …
49
Covariate-Adaptive Randomization
• Use of Prognostic Factors in Patient Allocation
• Zelen’s Rule• Stratified Randomization• Taves’ Minimization• Pocock-Simon’s Procedure
50
Stratified Randomization Male FemaleIII IV III IVB B A BA B A AB A B AA A B B
A A B AB A A BA B B BB B A A…
51
Tave’s Minimization
• Gender Test Control– Female 4 5– Male 5 3
• Age– 18-30 4 4– 31-45 2 4– 46-65 2 1
• Smoking– Yes 4 5– No 4 3
5252
Practical Issues
• Computer Resource to Implement• Inactive Voice Response System (IVRS)
• Randomization Code • Patient Log
– Informed consent and randomized dates– Dates and its sequence– Dates of subsequence visits – Termination date and related information
5353
QA Revisited
• Quality Assurance is a wide ranging concept which covers all matters which individually or collectively influence the quality of a clinical trial
• Regulations– GCP
• ICH E/S/Q/M Guidance's
5454
An Example of Statistical Analysis
A Non-inferiority TrialTest vs. Active Control
55
Non-Inferiority Trials
• In Active Controlled Non-Inferiority (NI) Trials– M=the effect size of active control– Ho: |A-T| M
• T has an effective size of M or more
– Ha: |A-T| < M (non-inferiority)• T is non-inferior to the control by less than M
• The Effect Size of a Treatment– Concurrent Placebo-Controlled Trials– Ho: T-P ≤ 0 – Ha: T-P > 0 (superiority to show effect size of T
group)
56
Logics
• Comparative Effectiveness– Test and Active-control groups: T and A– Estimate Effect Size of the active-control– Assumptions: assay sensitivity and
constancy
• The NI Margin (M)– M1: the entire effect of the active control
assumed to be present in the NI study – M2: the largest clinically acceptable
difference (degree of inferiority) of the test drug compared to the active control
57
A Two-Step Process in a NI study
• M1: the effect size of active control • M2: a specified portion of M1, based
upon clinical judgment
• The lower bound of a 95% confidence interval of A-P
• The upper bound of a 95% confidence interval of A-T obtained from the current study– If the upper bound < M2, then the
conclusion of non-inferiority is declared– The loss by the test product must be ruled
out
58
Example: Determination of an NI Margin
• SPORTIF V is a NI study that tested the novel anticoagulant ximelagatran against the active control warfarin. – Warfarin is a highly effective, orally active anticoagulant
for the treatment of patients with non-valvular atrial fibrillation at risk of thromboembolic complications (e.g., stroke, TIA, etc.).
– Six placebo-controlled studies of warfarin involving the treatment of patients with non-valvular atrial fibrillation, all published between the years 1989 and 1993.
– The primary results of these studies are summarized in Table 1 and provide the basis for choosing the NI margin for SPORTIF V.
• FDA: Guidance for Non-inferiority Trials (March, 2010)
59
60
61
Effect Size of an Active Control
• The Non-inferiority Margin – The 1722 relative risks in each of the
six studies were combined to M=1.378• In the SPORTIF V study
– The point estimate of the relative risk was 1.39 and the two-sided 95% CI for the relative risk was (0.91, 2.12). The upper limit (2.12) is greater than M (=1.378)
• Non-inferiority of ximelegatran to warfarin is not demonstrated
6262
QA Revisited
• Quality Assurance is a wide ranging concept which covers all matters which individually or collectively influence the quality of a clinical trial
• Data Management– ICH Guidance– GCP
6363
Data Management
Sponsor and InvestigatorQuality Control
Quality Assurance
64
Sponsor and Investigator
• A Protocol– Many Procedures– Data handling and Record keeping
• Trial Management– Investigational team– IRB– Sponsor/CRO
•Package of reports to organize and present, …
• Clinical Study Report
6565
Quality Assurance (QA)
• Audit Policy– SOPs– Independent unit or IRB
• Monitoring System – CRFs and medical records,
physician notes, documents, minutes
– CRO, data query
6666
Documentations
• Administrative Documents - Data lock/unlock, data clean, …
• Standard Operation Procedures (SOPs)
- Data management - Key-in, security, … - Computer facility
67
ICH Documents
• Es: Efficacy (E1-E15) • Ss: Safety • Qs: Quality (Q1-Q13)• Ms: Multi-discipline
68
ICH: Efficacy
• The work carried out by ICH under the Efficacy heading is concerned with the design, conduct, safety, and reporting of clinical trials. It also covers novel types of medicines derived from biotechnological processes and the use of pharmacogenetics/genomics techniques to produce better targeted medicines.
69
ICH: Safety
• ICH has produced a comprehensive set of safety guidelines to uncover potential risks like carcinogenicity, genotoxicity and reprotoxicity. A recent breakthrough has been a non-clinical testing strategy for assessing the QT interval prolongation liability: the single most important cause of drug withdrawals in recent years.
70
ICH: Quality
• Harmonisation achievements in the Quality area include pivotal milestones such as the conduct of stability studies, defining relevant thresholds for impurities testing and a more flexible approach to pharmaceutical quality based on Good Manufacturing Practice (GMP) risk management.
71
ICH: Multi-discipline
• Those are the cross-cutting topics which do not fit uniquely into one of the Quality, Safety and Efficacy categories. It includes the ICH medical terminology (MedDRA), the Common Technical Document (CTD) and the development of Electronic Standards for the Transfer of Regulatory Information (ESTRI).
72
ICH Efficacy Guidelines
• Guidelines Related to Statistical Issues – Efficacy
•E8 (General Considerations)•E10 (Choice of Control Group)•E9 (GSP) •E6 (GCP)•E3 (CSR)
73
Good Clinical Practice (GCP)
ICH E6An Article on GCP
Inspection
74
Contemporary Clinical Trials
• A comparative method of evaluating quality of international clinical studies in China: Analysis of site visit reports of the Clinical Research Operations and Monitoring Center (Chang, Xu, and Fan, 2008)
75
76
77
78
79
80
Conclusions
• The implemented China CIPRA program was at least comparable and equivalent to the US studies in GCP adherence
• The program's GCP performance was satisfactory in overall and for the selected critical GCP items. Protocol adherence was the major area that the China CIPRA program did more satisfactory than US sites; however, China and US sites both need close attention and more improvements in the areas of protocol adherence and essential documents/patient records.
