22
  Weight of Evidence: a review of concept and methods Douglas L. Weed reviewed by: N. Basilico, D. Laniado Risk Analysis, Vol. 25, No. 6, 2005 

Weight of Evidence_BasilicoLaniado

Embed Size (px)

DESCRIPTION

Weight of Evidence

Citation preview

  • WeightofEvidence:areviewofconceptandmethods

    DouglasL.Weed

    reviewedby:N.Basilico,D.LaniadoRiskAnalysis,Vol.25,No.6,2005

  • Introduction

    animportantissueintheprocessofscientificdiscovery:experimentumcrucis(crucialexperiment)

    itisadeterminativeexperimentthatcreatesnewscientificconsensus examplescanbefoundinphysics,chemistryandengineering

  • Introduction

    publichealth:difficulttofindcrucialexperiments ingeneralasingleexperimentcannotresolveacriticalissueonthecauseofadisease

    severalformsofevidenceindirectlyconnectedtothephenomenon

    someformofaggregationisneededinordertoevaluateacausalhypothesis

  • Introduction

    WOE:takingintoaccountdifferentkindsofscientificevidenceinassessingthevalidityofacausalhypothesis

    itisthemethodcurrentlyusedbyregulatoryagenciesinriskassessmentprocess

  • ObjectiveandMethodology objective:findoutthemenaningoftheexpression"weightofevidence"inrecentliterature

    computerizedPubMedlibrarysearch selectionof92papersinwhichtheterm"weightofevidence"(WOE)wasused allpublicationsfrom2003to2004(71) allpublicationsfrom1994to2002inwhich"WOE"appearedinthetitle(21)

  • UsesofWOE

    threedifferentusesoftheexpressionWOEhavebeenidentified: methaphorical methodological(withseveralsubcategories) theoretical

  • MetaphoricalUse

    mostcommonuse notreferredtoanyparticularmethod themethodissupposedtobeimplicitlyunderstandable

    lackoftransparency

  • MethodologicalUse

    strenghtofevidence useonlyasubsetoftheevidence(onlypositiveandstatisticallyrelevantoronlyunbiasedevidence)

    weightofevidence takeintoaccountalltheevidence

  • MethodologicalUse

    useofestabilishedinterpretativemethodologies systematicnarrativereviews qualitycriteriafortoxicologicstudies epidemiology'scausalcriteria metaanalisys mixedepidemiologytoxicologymethods

  • SystematicNarrativeReviews

    describingthestateofscience makingresearchrecommendationstoauthorsabouthowtodoclaimsofcausality,orpreventiverecommendations

    fosteringclarityandtrasparencyindescribinginterpretativemethods

  • QualityCriteriaforToxicologicStudies

    classifyingtoxicologicalstudiesaccordingtotheirquality: reliablewithoutrestriction reliablewithrestriction notreliable notassignable

    first2classes:includedintheintepretativeprocess last2classes:includedwithacasebycaseapproach

  • Epidemiology'sCausalCriteria

    givenabodyofstatisticallysignificantepidemiologicalevidenceandsomelabbasedevidence howcanbeestabilishedifthereisacausalrelationship? Ourobservationsrevealanassociationbetweentwovariables[...]Whataspectsofthatassociationshouldweespeciallyconsiderbeforedecidingthatthemostlikelyinterpretationofitiscausation?(A.B.Hill,1965)

  • Epidemiology'sCausalCriteria

    plausibility coherence experimentation analogy

    strenght consistency specificity temporality biologicalgradient(doseresponse)

    Hillidentified9differentcriteria:

  • MetaAnalysis

    quantitativeapproachforsummarizingevidencefromseveralhumanpopulationstudies

    contributionofeachstudyisweightedbytheinversevarianceoftheeffectestimate

    notsufficienttostatecausalitybutitcanprovideareproducibleweightedaverageoftheestimateofeffectacrossseveralstudies

  • MixedEpidemiologyToxicologymethods

    methodstocombineepidemiologicalevidenceandlabbasedbiologicalevidence

    whatisbiologicalplausibility?1.amechanismcanbehypothesized

    2.somelabbasedevidencesupportsthehypothesis

    3.sufficientevidencetoshowhowthefactorinfluencesaknowndiseasemechanism

  • QuantitativeWeightingSchemes

    identifyingasetofattributes examples:strenghtofassociation,temporalrepresentativeness,useofastandard

    assigningaweighttoeachelementaccordingtotheoverallsatisfactionofattributes

  • WOEintheory

    WOEapproachusedtounderstandhowvisualpatternsareperceivedasregularphenomena(figuralgoodness)

    canexplainwhysymmetryisperceivedasmoreregularthanrepetition

    incognitivescience(E.M.Pothos,R.Ward,2000)

  • WOEintheory

    inlaw>aconceptualframeworkforaWOEapproachbasedon4features: relevance reliability sufficiency standardofproof

  • Problems&Proposals

    lackoftransparency encourageauthorstodescribetheinterpretativemethodsused

    multiplicityofWOEdefinitionsandapplications movingtowardstheuseoffamiliarandwellestabilishedcriteria

    differentkindsofweights roleofjudgement

  • Critics

    Lettertotheeditor(I.Linkov,2006) noncomprehensivesearchmethods lackofcriticalappraisalofincludedstudies missedstudies notastateofthesciencereview

  • Discussion

    clearformalizationofthemethodologiescanhelptoreducethesubjectivityofjudgements

    wemustbeawarethatnointerpretativemethodcanabstractfromexpertjudgement furtherinvestigationisneededinordertoclearlydefinetheroleofexpertjudgementinthedifferentusesofWOEmethodologies

  • References Weed,D.L.(2005).Weightofevidence:areviewofconceptandmethod.RiskAnalysis,25(6),

    15451557.

    Hill,A.B.(1965).Theenvironmentanddisease:Associationorcausation?JournaloftheRoyalSocietyofMedicine,58,29530

    Pothos,E.M.,andWard,R.(2000).Simmetry,repetition,andfiguralgoodness:Aninvestigationoftheweightofevidencetheory.Cognition,75(3),B6578

    Krimsky,S.(2005).Theweightofscientificevidenceinpolicyandlaw.AmericanJournalofPublicHealth,95(S1),S129S136

    Linkov,I.(2006).WeightofEvidence:WhatistheStateoftheScience?RiskAnalysis,26(3),573575.

    Weed,D.L.(1997).Ontheuseofcausalcriteria.InternationalJournalofEpidemiology,26,11371141.

    Weed,D.L.,andHursting,S.D.(1998).Biologicplausibilityincausalinference:Currentmethodandpractice.AmericanJournalofEpidemiology,147,415425.