Embed Size (px)
Citation preview
Metodisk kvalitetsvurdering av systematisk oversikt
Rigmor C BergKurs H mars 2019
ndash 12th century knowledge syntheses in field of philosophy
ndash 17th century statistical techniques for synthesizing literature common in astronomy
ndash 1904 meta-analysis of studies on typhoid vaccination
ndash 1970s advancements in meta-analytic statistical techniques
Oppsummering av forskning har lang tradisjon
Systematisk skjevhet kan foslashre til feil konklusjon
Hva er en systematisk forskningoversikt
ndash En oversikt over den beste tilgjengelige kunnskap som finnes i verden om et bestemt problem
ndash Laget paring en systematisk eksplisitt og etterproslashvbar maringte
ndash Beskriver ikke bare en og en puslespillbit men hele bildet (slik vi ser det i dag)
bull rdquoSummererrdquo noe av litteraturflommen
bull Klargjoslashr hva vi vet og ikke vet
bull Avdekker kunnskapshull -- vise vei for videre forskning
bull En syntese av data fra flere studier kan bidra til nok statistisk styrke til aring avsloslashre om et tiltak er effektivt eller ei og dermed generere ldquonyrdquo kunnskap
(Lagravebbeet al 1987 Haselkorn et al 1994 Cochrane Colloboration Handbook 1996)
bull rdquoThis side uprdquo effective in preventing SIDS
rdquoThis side uprdquo reduced SIDS
In 15 years number of SIDS reduced by
80
The development is a result of the
message that infants should sleep on their
backs not their stomack or side
In Norway in the 1980s there was an
epidemic of SIDS In 1989 142 children lt1
years old died in SIDS Today the number
is ca 20 per year
Aftenposten 290404
Int J Epidemiol 2005 Aug34(4)874-87 Epub 2005 Apr 20Infant sleeping position and the sudden infant death syndrome systematic review of observational studies and historical review of recommendations from 1940 to 2002 Gilbert R1 Salanti G Harden M See S
bull Klar tittel og formaringl bull Omfattende referert soslashkestrategi bull Klare kriterier for inklusjon og eksklusjonbull Vurdering av kvalitet paring inkluderte studierbull Protokollfoslashring av karakteristiske trekk for hvert av de
inkluderte studienebull Sammenstilling av resultatene fra inkluderte studier (narrativ
eller meta-analyse)
(Cochrane Colloboration Handbook 2011)
Hva kjennetegner systematiske forskningsoversiktsprosesser
bull Transparens
bull Systematiske metoder for aring redusere risiko for feil naringr man skal finne velge ut vurdere og sammenfatte forskning
Vurdering av metodisk kvalitet
bull 3 key questions when read a study
1 Can I trust the results Ie are the results valid (Internal validity)
2 What do the results tell me (Statistics)
3 Can I apply the results to my context patient (External validity)
bull Check lists are useful tools to assess validity
Sjekklister for SRer
bull AMSTAR (A Measurement Tool to Assess Systematic Reviews Canada)
bull CASP (Critical Appraisal Skills Programme UK)
bull Usersrsquo Guides (JAMAJournal of American Medical Association US)
bull CEBM Checklist (Center for evidence-based Med)
bull JBI checklist for SRs
Is the question clearly and explicitly stated (PICO
appropriate inclusion and exclusion criteria)
Are the sources and methods to identify studies
appropriate At least 3 databases MeSH and textwords
Was there duplicate selection of studies Two
independent screeners inclusion form
Was there duplicate assessment of bias of the studies
Two independent assessors appropriate instrument
Was more than one person involved in data extraction
synthesis and GRADE Two assessors data extraction
form appropriate synthesis appropriate grading
Were the included studies adequately described
ndash 12th century knowledge syntheses in field of philosophy
ndash 17th century statistical techniques for synthesizing literature common in astronomy
ndash 1904 meta-analysis of studies on typhoid vaccination
ndash 1970s advancements in meta-analytic statistical techniques
Oppsummering av forskning har lang tradisjon
Systematisk skjevhet kan foslashre til feil konklusjon
Hva er en systematisk forskningoversikt
ndash En oversikt over den beste tilgjengelige kunnskap som finnes i verden om et bestemt problem
ndash Laget paring en systematisk eksplisitt og etterproslashvbar maringte
ndash Beskriver ikke bare en og en puslespillbit men hele bildet (slik vi ser det i dag)
bull rdquoSummererrdquo noe av litteraturflommen
bull Klargjoslashr hva vi vet og ikke vet
bull Avdekker kunnskapshull -- vise vei for videre forskning
bull En syntese av data fra flere studier kan bidra til nok statistisk styrke til aring avsloslashre om et tiltak er effektivt eller ei og dermed generere ldquonyrdquo kunnskap
(Lagravebbeet al 1987 Haselkorn et al 1994 Cochrane Colloboration Handbook 1996)
bull rdquoThis side uprdquo effective in preventing SIDS
rdquoThis side uprdquo reduced SIDS
In 15 years number of SIDS reduced by
80
The development is a result of the
message that infants should sleep on their
backs not their stomack or side
In Norway in the 1980s there was an
epidemic of SIDS In 1989 142 children lt1
years old died in SIDS Today the number
is ca 20 per year
Aftenposten 290404
Int J Epidemiol 2005 Aug34(4)874-87 Epub 2005 Apr 20Infant sleeping position and the sudden infant death syndrome systematic review of observational studies and historical review of recommendations from 1940 to 2002 Gilbert R1 Salanti G Harden M See S
bull Klar tittel og formaringl bull Omfattende referert soslashkestrategi bull Klare kriterier for inklusjon og eksklusjonbull Vurdering av kvalitet paring inkluderte studierbull Protokollfoslashring av karakteristiske trekk for hvert av de
inkluderte studienebull Sammenstilling av resultatene fra inkluderte studier (narrativ
eller meta-analyse)
(Cochrane Colloboration Handbook 2011)
Hva kjennetegner systematiske forskningsoversiktsprosesser
bull Transparens
bull Systematiske metoder for aring redusere risiko for feil naringr man skal finne velge ut vurdere og sammenfatte forskning
Vurdering av metodisk kvalitet
bull 3 key questions when read a study
1 Can I trust the results Ie are the results valid (Internal validity)
2 What do the results tell me (Statistics)
3 Can I apply the results to my context patient (External validity)
bull Check lists are useful tools to assess validity
Sjekklister for SRer
bull AMSTAR (A Measurement Tool to Assess Systematic Reviews Canada)
bull CASP (Critical Appraisal Skills Programme UK)
bull Usersrsquo Guides (JAMAJournal of American Medical Association US)
bull CEBM Checklist (Center for evidence-based Med)
bull JBI checklist for SRs
Is the question clearly and explicitly stated (PICO
appropriate inclusion and exclusion criteria)
Are the sources and methods to identify studies
appropriate At least 3 databases MeSH and textwords
Was there duplicate selection of studies Two
independent screeners inclusion form
Was there duplicate assessment of bias of the studies
Two independent assessors appropriate instrument
Was more than one person involved in data extraction
synthesis and GRADE Two assessors data extraction
form appropriate synthesis appropriate grading
Were the included studies adequately described
Systematisk skjevhet kan foslashre til feil konklusjon
Hva er en systematisk forskningoversikt
ndash En oversikt over den beste tilgjengelige kunnskap som finnes i verden om et bestemt problem
ndash Laget paring en systematisk eksplisitt og etterproslashvbar maringte
ndash Beskriver ikke bare en og en puslespillbit men hele bildet (slik vi ser det i dag)
bull rdquoSummererrdquo noe av litteraturflommen
bull Klargjoslashr hva vi vet og ikke vet
bull Avdekker kunnskapshull -- vise vei for videre forskning
bull En syntese av data fra flere studier kan bidra til nok statistisk styrke til aring avsloslashre om et tiltak er effektivt eller ei og dermed generere ldquonyrdquo kunnskap
(Lagravebbeet al 1987 Haselkorn et al 1994 Cochrane Colloboration Handbook 1996)
bull rdquoThis side uprdquo effective in preventing SIDS
rdquoThis side uprdquo reduced SIDS
In 15 years number of SIDS reduced by
80
The development is a result of the
message that infants should sleep on their
backs not their stomack or side
In Norway in the 1980s there was an
epidemic of SIDS In 1989 142 children lt1
years old died in SIDS Today the number
is ca 20 per year
Aftenposten 290404
Int J Epidemiol 2005 Aug34(4)874-87 Epub 2005 Apr 20Infant sleeping position and the sudden infant death syndrome systematic review of observational studies and historical review of recommendations from 1940 to 2002 Gilbert R1 Salanti G Harden M See S
bull Klar tittel og formaringl bull Omfattende referert soslashkestrategi bull Klare kriterier for inklusjon og eksklusjonbull Vurdering av kvalitet paring inkluderte studierbull Protokollfoslashring av karakteristiske trekk for hvert av de
inkluderte studienebull Sammenstilling av resultatene fra inkluderte studier (narrativ
eller meta-analyse)
(Cochrane Colloboration Handbook 2011)
Hva kjennetegner systematiske forskningsoversiktsprosesser
bull Transparens
bull Systematiske metoder for aring redusere risiko for feil naringr man skal finne velge ut vurdere og sammenfatte forskning
Vurdering av metodisk kvalitet
bull 3 key questions when read a study
1 Can I trust the results Ie are the results valid (Internal validity)
2 What do the results tell me (Statistics)
3 Can I apply the results to my context patient (External validity)
bull Check lists are useful tools to assess validity
Sjekklister for SRer
bull AMSTAR (A Measurement Tool to Assess Systematic Reviews Canada)
bull CASP (Critical Appraisal Skills Programme UK)
bull Usersrsquo Guides (JAMAJournal of American Medical Association US)
bull CEBM Checklist (Center for evidence-based Med)
bull JBI checklist for SRs
Is the question clearly and explicitly stated (PICO
appropriate inclusion and exclusion criteria)
Are the sources and methods to identify studies
appropriate At least 3 databases MeSH and textwords
Was there duplicate selection of studies Two
independent screeners inclusion form
Was there duplicate assessment of bias of the studies
Two independent assessors appropriate instrument
Was more than one person involved in data extraction
synthesis and GRADE Two assessors data extraction
form appropriate synthesis appropriate grading
Were the included studies adequately described
Hva er en systematisk forskningoversikt
ndash En oversikt over den beste tilgjengelige kunnskap som finnes i verden om et bestemt problem
ndash Laget paring en systematisk eksplisitt og etterproslashvbar maringte
ndash Beskriver ikke bare en og en puslespillbit men hele bildet (slik vi ser det i dag)
bull rdquoSummererrdquo noe av litteraturflommen
bull Klargjoslashr hva vi vet og ikke vet
bull Avdekker kunnskapshull -- vise vei for videre forskning
bull En syntese av data fra flere studier kan bidra til nok statistisk styrke til aring avsloslashre om et tiltak er effektivt eller ei og dermed generere ldquonyrdquo kunnskap
(Lagravebbeet al 1987 Haselkorn et al 1994 Cochrane Colloboration Handbook 1996)
bull rdquoThis side uprdquo effective in preventing SIDS
rdquoThis side uprdquo reduced SIDS
In 15 years number of SIDS reduced by
80
The development is a result of the
message that infants should sleep on their
backs not their stomack or side
In Norway in the 1980s there was an
epidemic of SIDS In 1989 142 children lt1
years old died in SIDS Today the number
is ca 20 per year
Aftenposten 290404
Int J Epidemiol 2005 Aug34(4)874-87 Epub 2005 Apr 20Infant sleeping position and the sudden infant death syndrome systematic review of observational studies and historical review of recommendations from 1940 to 2002 Gilbert R1 Salanti G Harden M See S
bull Klar tittel og formaringl bull Omfattende referert soslashkestrategi bull Klare kriterier for inklusjon og eksklusjonbull Vurdering av kvalitet paring inkluderte studierbull Protokollfoslashring av karakteristiske trekk for hvert av de
inkluderte studienebull Sammenstilling av resultatene fra inkluderte studier (narrativ
eller meta-analyse)
(Cochrane Colloboration Handbook 2011)
Hva kjennetegner systematiske forskningsoversiktsprosesser
bull Transparens
bull Systematiske metoder for aring redusere risiko for feil naringr man skal finne velge ut vurdere og sammenfatte forskning
Vurdering av metodisk kvalitet
bull 3 key questions when read a study
1 Can I trust the results Ie are the results valid (Internal validity)
2 What do the results tell me (Statistics)
3 Can I apply the results to my context patient (External validity)
bull Check lists are useful tools to assess validity
Sjekklister for SRer
bull AMSTAR (A Measurement Tool to Assess Systematic Reviews Canada)
bull CASP (Critical Appraisal Skills Programme UK)
bull Usersrsquo Guides (JAMAJournal of American Medical Association US)
bull CEBM Checklist (Center for evidence-based Med)
bull JBI checklist for SRs
Is the question clearly and explicitly stated (PICO
appropriate inclusion and exclusion criteria)
Are the sources and methods to identify studies
appropriate At least 3 databases MeSH and textwords
Was there duplicate selection of studies Two
independent screeners inclusion form
Was there duplicate assessment of bias of the studies
Two independent assessors appropriate instrument
Was more than one person involved in data extraction
synthesis and GRADE Two assessors data extraction
form appropriate synthesis appropriate grading
Were the included studies adequately described
bull rdquoSummererrdquo noe av litteraturflommen
bull Klargjoslashr hva vi vet og ikke vet
bull Avdekker kunnskapshull -- vise vei for videre forskning
bull En syntese av data fra flere studier kan bidra til nok statistisk styrke til aring avsloslashre om et tiltak er effektivt eller ei og dermed generere ldquonyrdquo kunnskap
(Lagravebbeet al 1987 Haselkorn et al 1994 Cochrane Colloboration Handbook 1996)
bull rdquoThis side uprdquo effective in preventing SIDS
rdquoThis side uprdquo reduced SIDS
In 15 years number of SIDS reduced by
80
The development is a result of the
message that infants should sleep on their
backs not their stomack or side
In Norway in the 1980s there was an
epidemic of SIDS In 1989 142 children lt1
years old died in SIDS Today the number
is ca 20 per year
Aftenposten 290404
Int J Epidemiol 2005 Aug34(4)874-87 Epub 2005 Apr 20Infant sleeping position and the sudden infant death syndrome systematic review of observational studies and historical review of recommendations from 1940 to 2002 Gilbert R1 Salanti G Harden M See S
bull Klar tittel og formaringl bull Omfattende referert soslashkestrategi bull Klare kriterier for inklusjon og eksklusjonbull Vurdering av kvalitet paring inkluderte studierbull Protokollfoslashring av karakteristiske trekk for hvert av de
inkluderte studienebull Sammenstilling av resultatene fra inkluderte studier (narrativ
eller meta-analyse)
(Cochrane Colloboration Handbook 2011)
Hva kjennetegner systematiske forskningsoversiktsprosesser
bull Transparens
bull Systematiske metoder for aring redusere risiko for feil naringr man skal finne velge ut vurdere og sammenfatte forskning
Vurdering av metodisk kvalitet
bull 3 key questions when read a study
1 Can I trust the results Ie are the results valid (Internal validity)
2 What do the results tell me (Statistics)
3 Can I apply the results to my context patient (External validity)
bull Check lists are useful tools to assess validity
Sjekklister for SRer
bull AMSTAR (A Measurement Tool to Assess Systematic Reviews Canada)
bull CASP (Critical Appraisal Skills Programme UK)
bull Usersrsquo Guides (JAMAJournal of American Medical Association US)
bull CEBM Checklist (Center for evidence-based Med)
bull JBI checklist for SRs
Is the question clearly and explicitly stated (PICO
appropriate inclusion and exclusion criteria)
Are the sources and methods to identify studies
appropriate At least 3 databases MeSH and textwords
Was there duplicate selection of studies Two
independent screeners inclusion form
Was there duplicate assessment of bias of the studies
Two independent assessors appropriate instrument
Was more than one person involved in data extraction
synthesis and GRADE Two assessors data extraction
form appropriate synthesis appropriate grading
Were the included studies adequately described
bull rdquoThis side uprdquo effective in preventing SIDS
rdquoThis side uprdquo reduced SIDS
In 15 years number of SIDS reduced by
80
The development is a result of the
message that infants should sleep on their
backs not their stomack or side
In Norway in the 1980s there was an
epidemic of SIDS In 1989 142 children lt1
years old died in SIDS Today the number
is ca 20 per year
Aftenposten 290404
Int J Epidemiol 2005 Aug34(4)874-87 Epub 2005 Apr 20Infant sleeping position and the sudden infant death syndrome systematic review of observational studies and historical review of recommendations from 1940 to 2002 Gilbert R1 Salanti G Harden M See S
bull Klar tittel og formaringl bull Omfattende referert soslashkestrategi bull Klare kriterier for inklusjon og eksklusjonbull Vurdering av kvalitet paring inkluderte studierbull Protokollfoslashring av karakteristiske trekk for hvert av de
inkluderte studienebull Sammenstilling av resultatene fra inkluderte studier (narrativ
eller meta-analyse)
(Cochrane Colloboration Handbook 2011)
Hva kjennetegner systematiske forskningsoversiktsprosesser
bull Transparens
bull Systematiske metoder for aring redusere risiko for feil naringr man skal finne velge ut vurdere og sammenfatte forskning
Vurdering av metodisk kvalitet
bull 3 key questions when read a study
1 Can I trust the results Ie are the results valid (Internal validity)
2 What do the results tell me (Statistics)
3 Can I apply the results to my context patient (External validity)
bull Check lists are useful tools to assess validity
Sjekklister for SRer
bull AMSTAR (A Measurement Tool to Assess Systematic Reviews Canada)
bull CASP (Critical Appraisal Skills Programme UK)
bull Usersrsquo Guides (JAMAJournal of American Medical Association US)
bull CEBM Checklist (Center for evidence-based Med)
bull JBI checklist for SRs
Is the question clearly and explicitly stated (PICO
appropriate inclusion and exclusion criteria)
Are the sources and methods to identify studies
appropriate At least 3 databases MeSH and textwords
Was there duplicate selection of studies Two
independent screeners inclusion form
Was there duplicate assessment of bias of the studies
Two independent assessors appropriate instrument
Was more than one person involved in data extraction
synthesis and GRADE Two assessors data extraction
form appropriate synthesis appropriate grading
Were the included studies adequately described
bull Klar tittel og formaringl bull Omfattende referert soslashkestrategi bull Klare kriterier for inklusjon og eksklusjonbull Vurdering av kvalitet paring inkluderte studierbull Protokollfoslashring av karakteristiske trekk for hvert av de
inkluderte studienebull Sammenstilling av resultatene fra inkluderte studier (narrativ
eller meta-analyse)
(Cochrane Colloboration Handbook 2011)
Hva kjennetegner systematiske forskningsoversiktsprosesser
bull Transparens
bull Systematiske metoder for aring redusere risiko for feil naringr man skal finne velge ut vurdere og sammenfatte forskning
Vurdering av metodisk kvalitet
bull 3 key questions when read a study
1 Can I trust the results Ie are the results valid (Internal validity)
2 What do the results tell me (Statistics)
3 Can I apply the results to my context patient (External validity)
bull Check lists are useful tools to assess validity
Sjekklister for SRer
bull AMSTAR (A Measurement Tool to Assess Systematic Reviews Canada)
bull CASP (Critical Appraisal Skills Programme UK)
bull Usersrsquo Guides (JAMAJournal of American Medical Association US)
bull CEBM Checklist (Center for evidence-based Med)
bull JBI checklist for SRs
Is the question clearly and explicitly stated (PICO
appropriate inclusion and exclusion criteria)
Are the sources and methods to identify studies
appropriate At least 3 databases MeSH and textwords
Was there duplicate selection of studies Two
independent screeners inclusion form
Was there duplicate assessment of bias of the studies
Two independent assessors appropriate instrument
Was more than one person involved in data extraction
synthesis and GRADE Two assessors data extraction
form appropriate synthesis appropriate grading
Were the included studies adequately described
Hva kjennetegner systematiske forskningsoversiktsprosesser
bull Transparens
bull Systematiske metoder for aring redusere risiko for feil naringr man skal finne velge ut vurdere og sammenfatte forskning
Vurdering av metodisk kvalitet
bull 3 key questions when read a study
1 Can I trust the results Ie are the results valid (Internal validity)
2 What do the results tell me (Statistics)
3 Can I apply the results to my context patient (External validity)
bull Check lists are useful tools to assess validity
Sjekklister for SRer
bull AMSTAR (A Measurement Tool to Assess Systematic Reviews Canada)
bull CASP (Critical Appraisal Skills Programme UK)
bull Usersrsquo Guides (JAMAJournal of American Medical Association US)
bull CEBM Checklist (Center for evidence-based Med)
bull JBI checklist for SRs
Is the question clearly and explicitly stated (PICO
appropriate inclusion and exclusion criteria)
Are the sources and methods to identify studies
appropriate At least 3 databases MeSH and textwords
Was there duplicate selection of studies Two
independent screeners inclusion form
Was there duplicate assessment of bias of the studies
Two independent assessors appropriate instrument
Was more than one person involved in data extraction
synthesis and GRADE Two assessors data extraction
form appropriate synthesis appropriate grading
Were the included studies adequately described
Vurdering av metodisk kvalitet
bull 3 key questions when read a study
1 Can I trust the results Ie are the results valid (Internal validity)
2 What do the results tell me (Statistics)
3 Can I apply the results to my context patient (External validity)
bull Check lists are useful tools to assess validity
Sjekklister for SRer
bull AMSTAR (A Measurement Tool to Assess Systematic Reviews Canada)
bull CASP (Critical Appraisal Skills Programme UK)
bull Usersrsquo Guides (JAMAJournal of American Medical Association US)
bull CEBM Checklist (Center for evidence-based Med)
bull JBI checklist for SRs
Is the question clearly and explicitly stated (PICO
appropriate inclusion and exclusion criteria)
Are the sources and methods to identify studies
appropriate At least 3 databases MeSH and textwords
Was there duplicate selection of studies Two
independent screeners inclusion form
Was there duplicate assessment of bias of the studies
Two independent assessors appropriate instrument
Was more than one person involved in data extraction
synthesis and GRADE Two assessors data extraction
form appropriate synthesis appropriate grading
Were the included studies adequately described
Sjekklister for SRer
bull AMSTAR (A Measurement Tool to Assess Systematic Reviews Canada)
bull CASP (Critical Appraisal Skills Programme UK)
bull Usersrsquo Guides (JAMAJournal of American Medical Association US)
bull CEBM Checklist (Center for evidence-based Med)
bull JBI checklist for SRs
Is the question clearly and explicitly stated (PICO
appropriate inclusion and exclusion criteria)
Are the sources and methods to identify studies
appropriate At least 3 databases MeSH and textwords
Was there duplicate selection of studies Two
independent screeners inclusion form
Was there duplicate assessment of bias of the studies
Two independent assessors appropriate instrument
Was more than one person involved in data extraction
synthesis and GRADE Two assessors data extraction
form appropriate synthesis appropriate grading
Were the included studies adequately described
Is the question clearly and explicitly stated (PICO
appropriate inclusion and exclusion criteria)
Are the sources and methods to identify studies
appropriate At least 3 databases MeSH and textwords
Was there duplicate selection of studies Two
independent screeners inclusion form
Was there duplicate assessment of bias of the studies
Two independent assessors appropriate instrument
Was more than one person involved in data extraction
synthesis and GRADE Two assessors data extraction
form appropriate synthesis appropriate grading
Were the included studies adequately described
