Upload
others
View
8
Download
0
Embed Size (px)
Citation preview
1
Critical appraisal of Systematic review &
Meta-analysis
: Practical points for oncologists
Kullathorn ThephamongkholJiraporn Setakornnukul
Radiation Oncologist, Siriraj hospital
2
ทําไมตอ้งสนใจsystematic review
meta-analysis
ฝันวา่ไดย้นิแพทยป์ระจําบา้นคยุกนั
• ปี 2: สอบตอ้งอา่น/อา้งผลการศกึษาแตล่ะ paper ไหม• ปี 3: ไมต่อ้งเลย ทอ่งเฉพาะ NCCN ไดเ้ลย• ปี 2: ทําไมละ่• ปี 3: NCCN อา้งไดเ้ลย เชือ่พี่• ปี 2: ตอ้งรูเ้หตผุลไหมวา่ จากหลกัฐานอะไร เพราะอะไร• ปี 3: สว่นใหญจ่าก RCTทกุอนัแหละ อยา่สนใจ อยา่เสยีเวลาจํา บางเรือ่ง RCT ไมไ่ดผ้ล แตเ่ขามเีหตผุลของเขาแหละ ถา้ NCCN บอก จะไมรั่กษาเหรอ
•ผมตืน่ขึน้มา อนจิจา….ภาวนาขอใหไ้มเ่ป็นเรือ่งจรงิ!!!3
Your motivation to look for meta-analysis ?
1. NEVER 2. Examination only3. Personal interest (e.g. for fun) 4. Problem in clinical practice5. Find reasons in the guideline
4
5
Clinical problem in practice
What is the best one you believe ?
1. Randomized controlled trial 2. Literature-based meta-analysis3. Individual-patient data (IPD) meta-analysis4. NCCN Guideline5. ASCO Guideline
6
VOTE: Do you give chemotherapy for Postop R0 NSCLC stage IB ? First round
1. YES, in Majority of patients2. NO, in minority of patients3. Y/N, in only special group T >=4 cm 4. Y/N, in at least one of high risk features5. Y/N, in only significant or many high risk
features 7
National Comprehensive Cancer Network
8
9
Cancer Care OntarioMcMaster University, the birthplace of evidence-based health-care
10
11
12
13
14
15
16
ASCO-CCO
17
Table summary: Adjuvant chemo after surgery (R0, -RT)
NCCN guideline 1.2018
CCO 2016
IPD pooled analysis LACE
JCO 2008
IPD metaLancet 2010
Cochrane 2015
ASCO-CCO guideline 2017
IA observe -
Clear benefit , not specify in which patients
Not recommended
IB Recommended in high risk group (poorly diff, vascular invasion, visceral pleura,T >= 4 cm, Nx
T >= 4 cm (would be reasonable to consider)
Official conclusion : Sig improve survival Last paragraph: Benefit didn’tvary except perhaps IAUndertaking validation
Benefit vs risk
IIA/B Recommended Recommended Recommended IIIA Recommended Recommended Recommended
18
19
VOTE: Do you give chemotherapy for Postop R0 NSCLC stage IB ? First round
1. YES, in Majority of patients2. NO, in minority of patients3. Y/N, in only special group T >=4 cm 4. Y/N, in at least one of high risk features5. Y/N, in only significant or many high risk
features 20
21
ในชวีติจรงิบางเรือ่งมีขดัแยง้กนัชดักวา่นีอ้กี
22
NCCN 1.2017 breast screening
•Women should be encouraged to be aware of their breast science this may facilitate detection of interval cancer between routine screening
23
Cochrane 2008
• Data from two large trials do NOT suggest a beneficial effect of screening by breast self-examination but do suggest increased harm in terms of increased numbers of benign lesions identified and an increased number of biopsies performed.
• At present, screening by breast self-examination or physical examination cannot be recommended.
24
•The USPSTF recommends againstclinicians teaching women howto perform breast self-examination. (Grade D recommendation)
25
• The Task Force’s charge is to provide evidence-based,population-level guidance.
• Yet, the media and politicians presented the breast cancer screening recommendations as a major departure from existing guidelines that ……
26
Important message Recommendation # Evidence
•PURE EVIDENCE
•Interpretation and consensus
•Recommendation
27
ทําไมตอ้ง critically appraise systematic reviewเชือ่ไปเลยไดไ้หม
28
Important message
•Suboptimal systematic review: harmful •Recommendation # PURE EVIDENCE
•Critical appraisal is very important
•How in practical/ easy/ clinician’s way?29
Objectives
•Understand critical process in sys review
•Practice/discussion of Critical appraisal
30
Discover (outline)
•Steps from question to synthesis
•Practical points in each step
•Solve examples of clinical problems
Big idea / Key concepts
•See difference in recommendation and find evidence/reasons behind this
Theory: CHECKLIST
33
Key steps
34
Critical appraisal tools
•McMaster & Duke checklist•ROBIS checklist•AMSTAR checklist•PRISMA reporting standard
35
1.Formulate question
2.Conduct searching
3.Studies collection
4.Studies appraisal
5.Synthesis & finding
36
FOCUS, PRE-SPECIFIED
COMPREHENSIVE, SYSTEMATIC, REPLICABLE
CLEAR CRITERIA, JUSTIFIED , TRANSPARENCY
RELIABLE TOOLS OF ASSESSMENT
MAGNITUDE (ABSOLUTE), CI, EXPLAIN DIFFERENCE, RATING
Theory: Statistical reasoning
37
38
Test of heterogeneity I2, p value (cut-off at 0.1)
39
Subgroup benefit of group 2 only ?(treatment effect difference between group 1 and 2)
40
Subgroup benefit of group 2 only ?
•LACE meta-analysis• IA HR 1.4,95%CI = 0.95-2.06• IB HR 0.93,95%CI =0.78-1.10• II HR 0.83,95%CI =0.73-0.95• III HR 0.83,95%CI =0.72-0.94
41
•Probability of interaction
•Trend test= more reliable than p value within subgroup
42
Subgroup benefit of stage II, III?No subgroup benefit of stage IB?
43
44
No subgroup benefit of stage III?(treatment effect difference between stage I,II and III?)
45
Only subgroup benefit of stage II?
46
Practical :
Find evidence/reason behind conclusion= synthesis
57
Table summary: Adjuvant chemo after surgery (R0, -RT)
NCCN guideline 1.2018
CCO 2016
IPD pooled analysis LACE
JCO 2008
IPD metaLancet 2010
Cochrane 2015
ASCO-CCO guideline 2017
IA observe -
Clear benefit , not specify in which patients
Not recommended
IB Recommended in high risk group (poorly diff, vascular invasion, visceral pleura,T >= 4 cm, Nx
T >= 4 cm (would be reasonable to consider)
Official conclusion : Sig improve survival Last paragraph: Benefit didn’tvary except perhaps IAUndertaking validation
Benefit vs risk
IIA/B Recommended Recommended Recommended IIIA Recommended Recommended Recommended
58
NCCN 1.2018NCCN guideline
1.2018CCO 2016
IPD pooled analysis LACE
JCO 2008
IPD metaLancet 2010
Cochrane 2015
ASCO-CCO guideline 2017
IA observe -
Clear benefit , not specify in which patients
Not recommended
IB Recommended in high risk group (poorly diff, vascular invasion, visceral pleura,T >= 4 cm, Nx
T >= 4 cm (would be reasonable to consider)
Official conclusion : Sig improve survival
Last paragraph: Benefit did not vary except perhaps stage IA
Benefit vs risk
IIA/B Recommended Recommended Recommended IIIA Recommended Recommended Recommended
59
NCCN: not seen reasons or reference behind this conclusion!
60
61
62
CCO 2016NCCN guideline
1.2018CCO 2016
IPD pooled analysis LACE
JCO 2008
IPD metaLancet 2010
Cochrane 2015
ASCO-CCO guideline 2017
IA observe -
Clear benefit , not specify in which patients
Not recommended
IB Recommended in high risk group (poorly diff, vascular invasion, visceral pleura,T >= 4 cm, Nx
T >= 4 cm (would be reasonable to consider)
Official conclusion : Sig improve survival
Last paragraph: Benefit did not vary except perhaps stage IA
Benefit vs risk
IIA/B Recommended Recommended Recommended IIIA Recommended Recommended Recommended
63
Synthesis CCO 2016Synthesis
CCO 2016 No meta-analyses by themselvesLACE pooled analysis, a difference in the effect oftreatment by stage was found (stage IA : HR, 1.40; 95% CI, 0.95-2.06; stage IB : HR, 0.93; 95% CI, 0.78-1.10; stage II : HR, 0.83; 95% CI, 0.73-0.95; stage III : HR, 0.83; 95% CI, 0.72-0.94; test for trend, P=0.043)
T>=4 cm 1) CALGB 9633 trial exploratory analysis HR 0.69, 90% CI 0.48-0.99, p=0.043 in T>=4 cm2) JBR.10 trial exploratory analysis HR non sig but positive interaction between tumor
size & chemotherapy, p=0.02T>=4 cm (HR, 0.66; 95% CI, 0.39-1.14)T< 4 cm (HR, 1.73; 95% CI, 0.98-3.04)
64
LACE pooled analysisNCCN guideline
1.2018CCO 2016
IPD pooled analysis LACE
JCO 2008
IPD metaLancet 2010
Cochrane 2015
ASCO-CCO guideline 2017
IA observe -
Clear benefit , not specify in which patients
Not recommended
IB Recommended in high risk group (poorly diff, vascular invasion, visceral pleura,T >= 4 cm, Nx
T >= 4 cm (would be reasonable to consider)
Official conclusion : Sig improve survival Last paragraph: Benefit didn’tvary except perhaps IAUndertaking validation
Benefit vs risk
IIA/B Recommended Recommended Recommended IIIA Recommended Recommended Recommended
65
Synthesis IPD LACE
66
• Different effect by stage was found • stage IA : HR, 1.40; 95% CI, 0.95-2.06; • stage IB : HR, 0.93; 95% CI, 0.78-1.10; • stage II : HR, 0.83; 95% CI, 0.73-0.95; • stage III : HR, 0.83; 95% CI, 0.72-0.94; • test for trend, P=0.04• test for interaction, P=0.06
Synthesis : IPD pooled analysis LACE
67
IPD meta Lancet 2010, Cochrane 2015NCCN guideline
1.2018CCO 2016
IPD pooled analysis LACE
JCO 2008
IPD metaLancet 2010
Cochrane 2015
ASCO-CCO guideline 2017
IA observe -
Clear benefit , not specify in which patients
Not recommended
IB Recommended in high risk group (poorly diff, vascular invasion, visceral pleura,T >= 4 cm, Nx
T >= 4 cm (would be reasonable to consider)
Official conclusion : Sig improve survival
Last paragraph: Benefit did not vary except perhaps stage IA
Benefit vs risk
IIA/B Recommended Recommended Recommended IIIA Recommended Recommended Recommended
68
Synthesis IPD meta 2015
69
70
Subgroup stage for all chemo
71
72
Subgroup stage in platinum only
• Absolute benefit 5yr OS
• IA 70% to 73%*• IB 55% to 60%** • II 40% to 45% • IIIA 30% to 35% • * contradictory with CI
73
Larger IB to stage II in 7th edition
Synthesis: IPD meta 2015
74
• *** All chemotherapy HR 0.86 (0.81-0.92);p<0.0001• *** Test for subgroup difference p=0.06, I2 = 51%
ASCO-CCO 2017NCCN guideline
1.2018CCO 2016
IPD pooled analysis LACE
JCO 2008
IPD metaLancet 2010
Cochrane 2015
ASCO-CCO guideline 2017
IA observe -
Clear benefit , not specify in which patients
Not recommended
IB Recommended in high risk group (poorly diff, vascular invasion, visceral pleura,T >= 4 cm, Nx
T >= 4 cm (would be reasonable to consider)
Official conclusion : Sig improve survival
Last paragraph: Benefit did not vary except perhaps stage IA
Benefit vs risk
IIA/B Recommended Recommended Recommended IIIA Recommended Recommended Recommended
75
Synthesis: ASCO-CCO guideline 2017
76
ASCO-CCO guideline 2017: IB•A pooled exploratory analysis based on two RCTs found a non-significant trend for increased chemotherapy effect on OS with larger tumor size
•Exploratory subgroup analysis of the NSCLCCG meta-analysis found no significant difference in the effect of adjuvant chemotherapy on survival by stage
77
ASCO-CCO 2017 Other Factors (outside trial)
•Tumor size : 5 year survival rate•3-4 cm : 74%, 4-5 cm : 65%
•Histo : PNI, tumor necrosis, LVI, visceral pleural invasion, mitotic index >10 mitoses/10 HPF
•High risk subtype of adenoCA: micropapillary or solid
78
Synthesis: ASCO-CCO guideline 2017• Using the LACE data• Increase 5 year OS• IB 64% to 67% • II 39% to 49% • IIIA 26% to 39% • blue= die either chemo or not
• Yellow =live without chemo
• Grey = live due to chemo
• Red = die due to chemo
79
So should we give chemo for IB?
• LACE IPD pooled analysis (pre-specified, positive interaction test, subgroup effect)stage IA/IB vs II, III or IA vs IB, II, III ?
• CALGB 6933 IB stage prelim data significant effect• CALGB 6933 IB stage mature data negative trial• CALGB 6933 & JBR.10 exploratory subgroup T>=4 cm (later
abstract negative), positive interaction test • NSCLCCG IPD meta (pre-specified, negative interaction test)
failed to prove difference of treatment in I, II, III• NSCLCCG IPD meta (exploratory, negative interaction test)
failed to prove this difference except IA?• Consider chemo in stage IB ?
81
VOTE: Do you give chemotherapy for Postop R0 NSCLC stage IB ? Second Round
1. YES, in Majority of patients2. NO, in minority of patients3. Y/N, in only special group T >=4 cm 4. Y/N, in at least one of high risk features5. Y/N, in only significant or many high risk
features 82
REASONS for VOTEYES, Majority of patients
1. From CALGB9633 abstract 2011: there is a POWERFUL TREND OS in all pts (HR=0.80; 90% CI 0.63-1.02; p=0.062, 1-tail), and 7% advantage (51% vs. 44%; p=0.087)
2. From NSCLCCG shows no treatment difference (negative interaction test) from stage II-III, absolute benefit is 5%, the same as II-III
3. Other personal reasons83
REASONS for VOTENO, Minority of patients
1. LACE IPD meta •Toxicity G3-4 =66% , toxicity G4 = 32% (G4 neutropenia 28%)
•Whereas survival benefit in IB only 3%2. Negative CALGB 6933 trial, so there is no
strong data support. We should play safe. Better safe than sorry
3. Other person reasons84
REASONS for VOTEYES, in special group T >=4 cm (page 1/2)1. CALGB 9633 JCO 2008:
Survival difference in T>=4 cm (HR, 0.69; CI, 0.48 to 0.99;P=0.043) and
CALGB 9633 abstract 2011:STRONG OS SIGNAL for those with large tumors. • T ≥4 cm, HR = 0.77 (90% CI: 0.57-1.04; p=0.079); • 10% improve in 8 yr OS (53% vs.43%; p=0.059); • T<4cm little OS diff (HR=0.89; p=0.31).
85
REASONS for VOTEYES, in special group T >=4 cm (page 2/2)
2. Positive interaction test (p=0.02) in JBR.10 trial with reversed effect•T>=4 cm (HR, 0.66; 95% CI, 0.39-1.14;p 0.13)•T< 4 cm (HR, 1.73; 95% CI, 0.98-3.04)3. Other personal reasons
86
REASONS for VOTEYES, in at least one of high risk features 1/2
1. Believe NCCN, • Even no explicit reasons but these high risk factors are reasonable and likely from reliable source
• NCCN doesn’t tell us weighting/ how many risk factors for considering chemotherapy, so at least one should be enough
87
REASONS for VOTEYES, in at least one of high risk features 2/2
2. Believe ASCO-CCO 2017, • Most update one• From NSCLCCG shows no treatment difference (negative interaction test) IB from stage II-III, absolute benefit is 5%, the same as II-III
• Give detail of literature review of these factors (size, high risk histo features, histo subtypes)
88
REASONS for VOTEYES, in only significant or many high risk features
With the ceiling of benefit around 3% (LACE), 5% (NSCLCCG) and 8% (CALGB at 4 year) and high toxicity G3-4 =66%, G4 = 32% (G4 neutropenia 28%), so it is worthwhile to pursue chemotherapy only in very high risk group
89
VOTE: Do you give chemotherapy for Postop R0 NSCLC stage IB ? FINALROUND
1. YES, in Majority of patients2. NO, in minority of patients3. Y/N, in only special group T >=4 cm 4. Y/N, in at least one of high risk features5. Y/N, in only significant or many high risk
features 90
My feeling (not conclusion) 1/3• IPD meta Cochrane 2015, including CALGB 9633 total =1000+/arm
“We split stage I disease into IA and IB for all but five trial comparisons (A04 Park1; A19 SGACLC ACTLC1; A21 SGACLC ACTLC2; A26 OLCSG2b; A32 OLCSG2a) which we had to exclude since this information was not available”• CALGB 9633 abstract 2011 : underpowered
• HR=0.80; 90% CI 0.63-1.02; p=0.062, 1-tail
91
My feeling (not conclusion) 2/3
•CALGB 9633 and IPD meta Cochrane 2015 are conflicting?
•Which one is more reliable? •Non statistical significance of interaction test = the same treatment benefit of stage IB, II, III?
92
My feeling (not conclusion) 2/3
•CALGB 9633 and IPD meta Cochrane 2015 are conflicting? NO
•Which one is more reliable? RCT vs IPD meta (stage is stratification factor?, diff. chemo in diff. stage from diff. trial?
•Non statistical significance of interaction test = the same treatment benefit of stage IB, II, III? Not necessary
93
My feeling (not conclusion) 3/3
• CALGB and IPD meta could reach the same conclusion but• Minute benefit but inadequate sample size?
Survival benefit 3% LACE, 5% NSCLCCG, 8% CALGB at 4 year • It’s not therapeutic question but prognostic/predictive question
because of heterogeneity? • subgroup question in IB: exploratory analysis in CALGB needed to
be validated before use• Prediction model to find direct evidence from high-risk patients from RCTs
(not from outside trial) • Size ? Chemotherapy regimens ? (Carbo+ Pac ?/ Cis ?)
• Decision analysis model (from RCTs data + cost) 94
Discussion & Conclusion from medical oncologists, please
95
We could learn BETTER from..•Different recommendations are challenging•Find specific evidence (RCTs, meta-analysis) /other reasons for each recommendation
•Use skill of statistical/clinical reasoning •Discuss with your friends & supervisors
97
อาจารย์ : ทําไมคณุเลอืกใหเ้คมบํีาบดัในรายนี้
• (แพทยป์ระจําบา้นปี 2) ผมอา่นจาก NCCN เดมิไม่ recommend แต่ version ใหมใ่ชคํ้าวา่ “consider” แลว้ครับ และ Category 1 ดว้ย ผมคยุกบัเพือ่นแลว้ตคีวาม “CONSIDER” คอืวา่สามารถใหไ้ดค้รับ
• (แพทยป์ระจําบา้นปี 3) หนูไมไ่ดอ้า่น NCCN อยา่งเดยีวคะ่ แตก่ลบัไปดดูว้ยวา่เขาอา้ง evidence อะไร พบวา่เขาอา้ง benefit ของบาง subgroup ของ RCT ตพีมิพใ์น JCO คะ่ และเป็นหลกัฐานทีเ่ชือ่ไดว้า่ควรใหค้ะ่
• (FELLOW) ถงึแม ้NCCN “consider” และมี RCT evidence แตเ่ป็นexploratory/unplanned subgroup ครับ ผมดู paper แลว้ RCT ที่ NCCN นํามาอา้ง มขีอ้น่าสงสยัหลายประการครับ และยงัมี subgroup จาก IPD meta-analysis ทีผ่ลขดัแยง้กบั NCCN ครับ ขออนุญาตนํามาอภปิรายกนักอ่นตดัสนิใจครับ
98
Credits:
99