Upload
duongtuyen
View
216
Download
0
Embed Size (px)
Citation preview
PRINOI avril 2012 Page 1
AMELIORATION DES PRATIQUES
IMPACT DES INDICATEURS DU TABLEAU DE BORD DES INFECTIONS NOSOCOMIALES
Journeacutee Preacutevention du Risque Infectieux Nosocomial de lrsquoOceacutean Indien
Vendredi 27 avril 2012
Didier LEPELLETIER
PRINOI avril 2012 Page 2
Les indicateurs
Existence drsquoindicateurs de moyens drsquoorganisation et de reacutesultats
Quels objectifs pour un indicateur laquo Benchmarking raquo
Professionnels ameacutelioration des pratiques Deacutecideurs paiement agrave la performance
Diffusion publique Droit des usagers Outil de seacutelection des ES () Equilibre entre pression politique validiteacute scientifique et outil de
communication (compreacutehensibiliteacute)
PRINOI avril 2012 Page 3
PRINOI avril 2012 Page 4
Critegraveres drsquoeacutevaluation drsquoun indicateur (1)
Pertinence Indicateur pour lequel il existe des possibiliteacutes drsquoameacutelioration
Faisabiliteacute
Charge de travail suppleacutementaire acceptable pour les professionnels des ES
Accessibiliteacute des donneacutees (par exemple dans le SIH)
PRINOI avril 2012 Page 5
Critegraveres drsquoeacutevaluation drsquoun indicateur (2)
Qualiteacutes meacutetrologiques validiteacute (sensibiliteacute speacutecificiteacute VPP VPN) fiabiliteacute
reproductibiliteacute standardisation possibiliteacute de comparaison dans le temps etou dans lrsquoespace
Utiliteacute pour les professionnels
en vue drsquoune ameacutelioration des pratiques ou de benchmarking
Utiliteacute pour les usagers
transparence arguments pour un choix raisonneacute des ES par les usagers
PRINOI avril 2012 Page 6
Recommandations HICPAC 2005 (USA)
Indicateurs recommandeacutes (impact important possibiliteacute de preacutevention)
Bacteacuterieacutemies primaires en soins intensifs confirmeacutees microbiologiquement
ISO agrave la suite de certains types drsquointervention chirurgicale
Non recommandeacutes
Infections urinaires lieacutees au sondage veacutesical (moins drsquoimpact moins de possibiliteacute de preacutevention)
Pneumopathies associeacutees agrave la ventilation meacutecanique (impact important mais deacutetectabiliteacute faible-deacutefinition CDC relativement subjective)
Guidance on public reporting of healthcare-associated infections
recommendations of the Healthcare Infection Control Practices Advisory Committee McKibben L Horan T Tokars JI Fowler G Cardo DM Pearson ML Brennan PJ
Heathcare Infection Control Practices Advisory Committee Am J Infect Control 2005 May33(4)217-26
PRINOI avril 2012 Page 7
Technical Advisory Committee Maryland Health-Care Commission
laquo Outcomes raquo scores les plus eacuteleveacutes Bacteacuterieacutemies associeacutees aux CVC (2630)
ISO apregraves pontage coronarien (2230)
ISO apregraves prothegravese totale de hanche ou de genou (2130)
Autres laquo Outcomes raquo MRSAVRE en reacuteanimation
pneumopathie acquise sous ventilation meacutecanique infections urinaires sur sonde chirurgie sein hysteacuterectomie
Public reporting of health care-associated infections (HAIs) approach to choosing HAI measures Passaretti CL et al Infect Control Hosp Epidemiol 201132768-74
PRINOI avril 2012 Page 8
En Europe
Rapport obligatoire aupregraves des tutelles mais pas de mise agrave disposition du public Taux drsquoincidence des bacteacuterieacutemies
associeacutees aux CVC Taux drsquoincidence des pneumopathies
associeacutees agrave la ventilation meacutecanique Ratio drsquoexposition aux dispositifs
invasifs Volontariat (reacuteseau KISS)
Taux de colonisation agrave SARM Taux drsquoincidence des infections agrave
Clostridium difficile Taux drsquoincidence des ISO Taux drsquoincidence des bacteacuterieacutemies
associeacutees aux CVC Taux drsquoincidence des infections
urinaires associeacutees au sondage veacutesical Taux drsquoincidence des pneumopathies
associeacutees agrave la ventilation meacutecanique
Rapporteacutes publiquement Taux drsquoincidence des
bacteacuterieacutemies dues agrave des pathogegravenes speacutecifiques autres que SARM (SASM Ecoli)
Taux drsquoincidence des bacteacuterieacutemies agrave SARM
Taux drsquoincidence des infections agrave Clostridium difficile
Nb MRSA et ICD dans les 12 derniegraveres semaines
Nb semaines sans MRSA
Variable Taux drsquoincidence des ISO Taux de preacutevalence de la
reacutesistance microbienne
ALLEMAGNE ANGLETERRE
Haustein et al Lancet ID 2011
PRINOI avril 2012 Page 9
Calcul et expression de lrsquoindicateur
QUANTITATIF Taux de preacutevalence IAS Incidence IAS (taux ou
densiteacute) Taux de preacutelegravevements
cliniques Taux brut ou taux stratifieacute RSI= Nb observeacute IAS
Nb attendu IAS
QUALITATIF Preacutesence drsquoau moins 1 type
drsquoIAS (ouinon) pendant une peacuteriode donneacutee (hellip agrave deacutefinir)
Intervalle (nb semaines) sans IAS
Taux diffeacuterent du taux de reacutefeacuterence choisi (gt1 ou lt 1)
RSI gt1 ou lt1 SEMI-QUANTITATIF Taux compris dans un intervalle
deacutefini par un(des) quartile(s) drsquoune distribution de reacutefeacuterence (P50 P75 IQ hellip)
PRINOI avril 2012 Page 10
Indicateurs IAS expeacuteriences internationales Groupe drsquoexperts
(France 2003)
Pertinence Faisabiliteacute Qualiteacute
meacutetrologique
Utilisation
agrave lrsquoexteacuterieur de
lrsquoES
Passaretti et coll
(Maryland USA ICHE 2011)
Impact
Improvability
Feasibility
Frequency
Feasibility
(lsquoSystem for collecting
data already in placersquo)
Inclusiveness
Functionality
Mc Kibben et coll
(HICPAC USA AJIC 2006)
Severity
Preventability
Detectability
Accuracy
HELICS-IPSE
(Europe)
Clinical relevance Simplicity
Acceptability
Timeliness
Completeness
Validity
Sensitivity
Readibility
Pas de distingo par les usagers vs par les professionnels
PRINOI avril 2012 Page 11
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 12
DI bacteacuterieacutemies primaires 1000 jours de catheacuteteacuterisme veineux central
59
98
53
32
09 09 12 14
04 02
13 11
0
5
10
15
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
NNIS Cardiothoracic ICU
REA-RAISIN
Reacuteanimation CTCV (HGRL CHU Nantes)
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
1 Proceacutedure pose entretien des CVC et harmonisation de la deacutefinition
3 Audit drsquoobservation
4 V2 mode opeacuteratoire pose CVC
et feuille de suivi hors USI
2 Surveillance prospective et publication de lrsquoindicateur
5 V2 mode opeacuteratoire Reacutefection pansement et changement lignes
PRINOI avril 2012 Page 13
PRINOI avril 2012 Page 14
PRINOI avril 2012 Page 15
Objectifs quantifieacutes Ameacuteliorer la preacutevention des infections associeacutees
aux actes invasifs
Source Circulaire du 26 aoucirct 2009
PRINOI avril 2012 Page 16
PRINOI avril 2012 Page 17
DI infections urinaires 1000 jours de sondage urinaire
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
Proceacutedure sondage urinaire Mateacuteriel de
diuregravese horaire preacute-connecteacute
Formation des internes agrave chaque semestre au BO
PRINOI avril 2012 Page 18
Evolution des indicateurs du tableau de bord
Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires
Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques
Nouvelle organisation des ES et ARS (loi HPST)
Deacutefinition drsquoobjectifs quantifieacutes pour 2012
Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)
PRINOI avril 2012 Page 19
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 20
Nouveau rocircle des instances dans la politique de lutte contre les IN
Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins
ICALIN 2
ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50
PRINOI avril 2012 Page 21
Nouveauteacutes ICALIN 2 Protection du personnel
proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)
Suivi de la consommation des PHA et restitution au moins 1 fois par an
Check-list SU CVP CCI CVC CSC
Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique
Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau
Reacutesultats des surveillances preacutesenteacutes en CME
Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave
Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS
PCC PCAG sonde urinaire CVPCVCCCICSC
PRINOI avril 2012 Page 22
Indicateur ICABMR
PRINOI avril 2012 Page 23
Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008
PRINOI avril 2012 Page 24
Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008
PRINOI avril 2012 Page 25
Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline
au sein de lrsquoespegravece SA
PRINOI avril 2012 Page 26
Source httpwwwrivmnlearssdatabase
PRINOI avril 2012 Page 27
Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM
ne le sont pas sur les EBLSE
Que nous disent les indicateurs de surveillance
PRINOI avril 2012 Page 28
DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)
02018018
012007003
016
019024
026033
036039
05048043
0
01
02
03
04
05
06
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
DI EBLSE 1000 JH DI SARM 1000 JH
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 2
Les indicateurs
Existence drsquoindicateurs de moyens drsquoorganisation et de reacutesultats
Quels objectifs pour un indicateur laquo Benchmarking raquo
Professionnels ameacutelioration des pratiques Deacutecideurs paiement agrave la performance
Diffusion publique Droit des usagers Outil de seacutelection des ES () Equilibre entre pression politique validiteacute scientifique et outil de
communication (compreacutehensibiliteacute)
PRINOI avril 2012 Page 3
PRINOI avril 2012 Page 4
Critegraveres drsquoeacutevaluation drsquoun indicateur (1)
Pertinence Indicateur pour lequel il existe des possibiliteacutes drsquoameacutelioration
Faisabiliteacute
Charge de travail suppleacutementaire acceptable pour les professionnels des ES
Accessibiliteacute des donneacutees (par exemple dans le SIH)
PRINOI avril 2012 Page 5
Critegraveres drsquoeacutevaluation drsquoun indicateur (2)
Qualiteacutes meacutetrologiques validiteacute (sensibiliteacute speacutecificiteacute VPP VPN) fiabiliteacute
reproductibiliteacute standardisation possibiliteacute de comparaison dans le temps etou dans lrsquoespace
Utiliteacute pour les professionnels
en vue drsquoune ameacutelioration des pratiques ou de benchmarking
Utiliteacute pour les usagers
transparence arguments pour un choix raisonneacute des ES par les usagers
PRINOI avril 2012 Page 6
Recommandations HICPAC 2005 (USA)
Indicateurs recommandeacutes (impact important possibiliteacute de preacutevention)
Bacteacuterieacutemies primaires en soins intensifs confirmeacutees microbiologiquement
ISO agrave la suite de certains types drsquointervention chirurgicale
Non recommandeacutes
Infections urinaires lieacutees au sondage veacutesical (moins drsquoimpact moins de possibiliteacute de preacutevention)
Pneumopathies associeacutees agrave la ventilation meacutecanique (impact important mais deacutetectabiliteacute faible-deacutefinition CDC relativement subjective)
Guidance on public reporting of healthcare-associated infections
recommendations of the Healthcare Infection Control Practices Advisory Committee McKibben L Horan T Tokars JI Fowler G Cardo DM Pearson ML Brennan PJ
Heathcare Infection Control Practices Advisory Committee Am J Infect Control 2005 May33(4)217-26
PRINOI avril 2012 Page 7
Technical Advisory Committee Maryland Health-Care Commission
laquo Outcomes raquo scores les plus eacuteleveacutes Bacteacuterieacutemies associeacutees aux CVC (2630)
ISO apregraves pontage coronarien (2230)
ISO apregraves prothegravese totale de hanche ou de genou (2130)
Autres laquo Outcomes raquo MRSAVRE en reacuteanimation
pneumopathie acquise sous ventilation meacutecanique infections urinaires sur sonde chirurgie sein hysteacuterectomie
Public reporting of health care-associated infections (HAIs) approach to choosing HAI measures Passaretti CL et al Infect Control Hosp Epidemiol 201132768-74
PRINOI avril 2012 Page 8
En Europe
Rapport obligatoire aupregraves des tutelles mais pas de mise agrave disposition du public Taux drsquoincidence des bacteacuterieacutemies
associeacutees aux CVC Taux drsquoincidence des pneumopathies
associeacutees agrave la ventilation meacutecanique Ratio drsquoexposition aux dispositifs
invasifs Volontariat (reacuteseau KISS)
Taux de colonisation agrave SARM Taux drsquoincidence des infections agrave
Clostridium difficile Taux drsquoincidence des ISO Taux drsquoincidence des bacteacuterieacutemies
associeacutees aux CVC Taux drsquoincidence des infections
urinaires associeacutees au sondage veacutesical Taux drsquoincidence des pneumopathies
associeacutees agrave la ventilation meacutecanique
Rapporteacutes publiquement Taux drsquoincidence des
bacteacuterieacutemies dues agrave des pathogegravenes speacutecifiques autres que SARM (SASM Ecoli)
Taux drsquoincidence des bacteacuterieacutemies agrave SARM
Taux drsquoincidence des infections agrave Clostridium difficile
Nb MRSA et ICD dans les 12 derniegraveres semaines
Nb semaines sans MRSA
Variable Taux drsquoincidence des ISO Taux de preacutevalence de la
reacutesistance microbienne
ALLEMAGNE ANGLETERRE
Haustein et al Lancet ID 2011
PRINOI avril 2012 Page 9
Calcul et expression de lrsquoindicateur
QUANTITATIF Taux de preacutevalence IAS Incidence IAS (taux ou
densiteacute) Taux de preacutelegravevements
cliniques Taux brut ou taux stratifieacute RSI= Nb observeacute IAS
Nb attendu IAS
QUALITATIF Preacutesence drsquoau moins 1 type
drsquoIAS (ouinon) pendant une peacuteriode donneacutee (hellip agrave deacutefinir)
Intervalle (nb semaines) sans IAS
Taux diffeacuterent du taux de reacutefeacuterence choisi (gt1 ou lt 1)
RSI gt1 ou lt1 SEMI-QUANTITATIF Taux compris dans un intervalle
deacutefini par un(des) quartile(s) drsquoune distribution de reacutefeacuterence (P50 P75 IQ hellip)
PRINOI avril 2012 Page 10
Indicateurs IAS expeacuteriences internationales Groupe drsquoexperts
(France 2003)
Pertinence Faisabiliteacute Qualiteacute
meacutetrologique
Utilisation
agrave lrsquoexteacuterieur de
lrsquoES
Passaretti et coll
(Maryland USA ICHE 2011)
Impact
Improvability
Feasibility
Frequency
Feasibility
(lsquoSystem for collecting
data already in placersquo)
Inclusiveness
Functionality
Mc Kibben et coll
(HICPAC USA AJIC 2006)
Severity
Preventability
Detectability
Accuracy
HELICS-IPSE
(Europe)
Clinical relevance Simplicity
Acceptability
Timeliness
Completeness
Validity
Sensitivity
Readibility
Pas de distingo par les usagers vs par les professionnels
PRINOI avril 2012 Page 11
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 12
DI bacteacuterieacutemies primaires 1000 jours de catheacuteteacuterisme veineux central
59
98
53
32
09 09 12 14
04 02
13 11
0
5
10
15
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
NNIS Cardiothoracic ICU
REA-RAISIN
Reacuteanimation CTCV (HGRL CHU Nantes)
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
1 Proceacutedure pose entretien des CVC et harmonisation de la deacutefinition
3 Audit drsquoobservation
4 V2 mode opeacuteratoire pose CVC
et feuille de suivi hors USI
2 Surveillance prospective et publication de lrsquoindicateur
5 V2 mode opeacuteratoire Reacutefection pansement et changement lignes
PRINOI avril 2012 Page 13
PRINOI avril 2012 Page 14
PRINOI avril 2012 Page 15
Objectifs quantifieacutes Ameacuteliorer la preacutevention des infections associeacutees
aux actes invasifs
Source Circulaire du 26 aoucirct 2009
PRINOI avril 2012 Page 16
PRINOI avril 2012 Page 17
DI infections urinaires 1000 jours de sondage urinaire
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
Proceacutedure sondage urinaire Mateacuteriel de
diuregravese horaire preacute-connecteacute
Formation des internes agrave chaque semestre au BO
PRINOI avril 2012 Page 18
Evolution des indicateurs du tableau de bord
Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires
Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques
Nouvelle organisation des ES et ARS (loi HPST)
Deacutefinition drsquoobjectifs quantifieacutes pour 2012
Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)
PRINOI avril 2012 Page 19
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 20
Nouveau rocircle des instances dans la politique de lutte contre les IN
Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins
ICALIN 2
ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50
PRINOI avril 2012 Page 21
Nouveauteacutes ICALIN 2 Protection du personnel
proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)
Suivi de la consommation des PHA et restitution au moins 1 fois par an
Check-list SU CVP CCI CVC CSC
Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique
Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau
Reacutesultats des surveillances preacutesenteacutes en CME
Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave
Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS
PCC PCAG sonde urinaire CVPCVCCCICSC
PRINOI avril 2012 Page 22
Indicateur ICABMR
PRINOI avril 2012 Page 23
Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008
PRINOI avril 2012 Page 24
Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008
PRINOI avril 2012 Page 25
Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline
au sein de lrsquoespegravece SA
PRINOI avril 2012 Page 26
Source httpwwwrivmnlearssdatabase
PRINOI avril 2012 Page 27
Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM
ne le sont pas sur les EBLSE
Que nous disent les indicateurs de surveillance
PRINOI avril 2012 Page 28
DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)
02018018
012007003
016
019024
026033
036039
05048043
0
01
02
03
04
05
06
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
DI EBLSE 1000 JH DI SARM 1000 JH
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 3
PRINOI avril 2012 Page 4
Critegraveres drsquoeacutevaluation drsquoun indicateur (1)
Pertinence Indicateur pour lequel il existe des possibiliteacutes drsquoameacutelioration
Faisabiliteacute
Charge de travail suppleacutementaire acceptable pour les professionnels des ES
Accessibiliteacute des donneacutees (par exemple dans le SIH)
PRINOI avril 2012 Page 5
Critegraveres drsquoeacutevaluation drsquoun indicateur (2)
Qualiteacutes meacutetrologiques validiteacute (sensibiliteacute speacutecificiteacute VPP VPN) fiabiliteacute
reproductibiliteacute standardisation possibiliteacute de comparaison dans le temps etou dans lrsquoespace
Utiliteacute pour les professionnels
en vue drsquoune ameacutelioration des pratiques ou de benchmarking
Utiliteacute pour les usagers
transparence arguments pour un choix raisonneacute des ES par les usagers
PRINOI avril 2012 Page 6
Recommandations HICPAC 2005 (USA)
Indicateurs recommandeacutes (impact important possibiliteacute de preacutevention)
Bacteacuterieacutemies primaires en soins intensifs confirmeacutees microbiologiquement
ISO agrave la suite de certains types drsquointervention chirurgicale
Non recommandeacutes
Infections urinaires lieacutees au sondage veacutesical (moins drsquoimpact moins de possibiliteacute de preacutevention)
Pneumopathies associeacutees agrave la ventilation meacutecanique (impact important mais deacutetectabiliteacute faible-deacutefinition CDC relativement subjective)
Guidance on public reporting of healthcare-associated infections
recommendations of the Healthcare Infection Control Practices Advisory Committee McKibben L Horan T Tokars JI Fowler G Cardo DM Pearson ML Brennan PJ
Heathcare Infection Control Practices Advisory Committee Am J Infect Control 2005 May33(4)217-26
PRINOI avril 2012 Page 7
Technical Advisory Committee Maryland Health-Care Commission
laquo Outcomes raquo scores les plus eacuteleveacutes Bacteacuterieacutemies associeacutees aux CVC (2630)
ISO apregraves pontage coronarien (2230)
ISO apregraves prothegravese totale de hanche ou de genou (2130)
Autres laquo Outcomes raquo MRSAVRE en reacuteanimation
pneumopathie acquise sous ventilation meacutecanique infections urinaires sur sonde chirurgie sein hysteacuterectomie
Public reporting of health care-associated infections (HAIs) approach to choosing HAI measures Passaretti CL et al Infect Control Hosp Epidemiol 201132768-74
PRINOI avril 2012 Page 8
En Europe
Rapport obligatoire aupregraves des tutelles mais pas de mise agrave disposition du public Taux drsquoincidence des bacteacuterieacutemies
associeacutees aux CVC Taux drsquoincidence des pneumopathies
associeacutees agrave la ventilation meacutecanique Ratio drsquoexposition aux dispositifs
invasifs Volontariat (reacuteseau KISS)
Taux de colonisation agrave SARM Taux drsquoincidence des infections agrave
Clostridium difficile Taux drsquoincidence des ISO Taux drsquoincidence des bacteacuterieacutemies
associeacutees aux CVC Taux drsquoincidence des infections
urinaires associeacutees au sondage veacutesical Taux drsquoincidence des pneumopathies
associeacutees agrave la ventilation meacutecanique
Rapporteacutes publiquement Taux drsquoincidence des
bacteacuterieacutemies dues agrave des pathogegravenes speacutecifiques autres que SARM (SASM Ecoli)
Taux drsquoincidence des bacteacuterieacutemies agrave SARM
Taux drsquoincidence des infections agrave Clostridium difficile
Nb MRSA et ICD dans les 12 derniegraveres semaines
Nb semaines sans MRSA
Variable Taux drsquoincidence des ISO Taux de preacutevalence de la
reacutesistance microbienne
ALLEMAGNE ANGLETERRE
Haustein et al Lancet ID 2011
PRINOI avril 2012 Page 9
Calcul et expression de lrsquoindicateur
QUANTITATIF Taux de preacutevalence IAS Incidence IAS (taux ou
densiteacute) Taux de preacutelegravevements
cliniques Taux brut ou taux stratifieacute RSI= Nb observeacute IAS
Nb attendu IAS
QUALITATIF Preacutesence drsquoau moins 1 type
drsquoIAS (ouinon) pendant une peacuteriode donneacutee (hellip agrave deacutefinir)
Intervalle (nb semaines) sans IAS
Taux diffeacuterent du taux de reacutefeacuterence choisi (gt1 ou lt 1)
RSI gt1 ou lt1 SEMI-QUANTITATIF Taux compris dans un intervalle
deacutefini par un(des) quartile(s) drsquoune distribution de reacutefeacuterence (P50 P75 IQ hellip)
PRINOI avril 2012 Page 10
Indicateurs IAS expeacuteriences internationales Groupe drsquoexperts
(France 2003)
Pertinence Faisabiliteacute Qualiteacute
meacutetrologique
Utilisation
agrave lrsquoexteacuterieur de
lrsquoES
Passaretti et coll
(Maryland USA ICHE 2011)
Impact
Improvability
Feasibility
Frequency
Feasibility
(lsquoSystem for collecting
data already in placersquo)
Inclusiveness
Functionality
Mc Kibben et coll
(HICPAC USA AJIC 2006)
Severity
Preventability
Detectability
Accuracy
HELICS-IPSE
(Europe)
Clinical relevance Simplicity
Acceptability
Timeliness
Completeness
Validity
Sensitivity
Readibility
Pas de distingo par les usagers vs par les professionnels
PRINOI avril 2012 Page 11
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 12
DI bacteacuterieacutemies primaires 1000 jours de catheacuteteacuterisme veineux central
59
98
53
32
09 09 12 14
04 02
13 11
0
5
10
15
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
NNIS Cardiothoracic ICU
REA-RAISIN
Reacuteanimation CTCV (HGRL CHU Nantes)
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
1 Proceacutedure pose entretien des CVC et harmonisation de la deacutefinition
3 Audit drsquoobservation
4 V2 mode opeacuteratoire pose CVC
et feuille de suivi hors USI
2 Surveillance prospective et publication de lrsquoindicateur
5 V2 mode opeacuteratoire Reacutefection pansement et changement lignes
PRINOI avril 2012 Page 13
PRINOI avril 2012 Page 14
PRINOI avril 2012 Page 15
Objectifs quantifieacutes Ameacuteliorer la preacutevention des infections associeacutees
aux actes invasifs
Source Circulaire du 26 aoucirct 2009
PRINOI avril 2012 Page 16
PRINOI avril 2012 Page 17
DI infections urinaires 1000 jours de sondage urinaire
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
Proceacutedure sondage urinaire Mateacuteriel de
diuregravese horaire preacute-connecteacute
Formation des internes agrave chaque semestre au BO
PRINOI avril 2012 Page 18
Evolution des indicateurs du tableau de bord
Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires
Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques
Nouvelle organisation des ES et ARS (loi HPST)
Deacutefinition drsquoobjectifs quantifieacutes pour 2012
Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)
PRINOI avril 2012 Page 19
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 20
Nouveau rocircle des instances dans la politique de lutte contre les IN
Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins
ICALIN 2
ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50
PRINOI avril 2012 Page 21
Nouveauteacutes ICALIN 2 Protection du personnel
proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)
Suivi de la consommation des PHA et restitution au moins 1 fois par an
Check-list SU CVP CCI CVC CSC
Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique
Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau
Reacutesultats des surveillances preacutesenteacutes en CME
Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave
Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS
PCC PCAG sonde urinaire CVPCVCCCICSC
PRINOI avril 2012 Page 22
Indicateur ICABMR
PRINOI avril 2012 Page 23
Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008
PRINOI avril 2012 Page 24
Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008
PRINOI avril 2012 Page 25
Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline
au sein de lrsquoespegravece SA
PRINOI avril 2012 Page 26
Source httpwwwrivmnlearssdatabase
PRINOI avril 2012 Page 27
Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM
ne le sont pas sur les EBLSE
Que nous disent les indicateurs de surveillance
PRINOI avril 2012 Page 28
DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)
02018018
012007003
016
019024
026033
036039
05048043
0
01
02
03
04
05
06
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
DI EBLSE 1000 JH DI SARM 1000 JH
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 4
Critegraveres drsquoeacutevaluation drsquoun indicateur (1)
Pertinence Indicateur pour lequel il existe des possibiliteacutes drsquoameacutelioration
Faisabiliteacute
Charge de travail suppleacutementaire acceptable pour les professionnels des ES
Accessibiliteacute des donneacutees (par exemple dans le SIH)
PRINOI avril 2012 Page 5
Critegraveres drsquoeacutevaluation drsquoun indicateur (2)
Qualiteacutes meacutetrologiques validiteacute (sensibiliteacute speacutecificiteacute VPP VPN) fiabiliteacute
reproductibiliteacute standardisation possibiliteacute de comparaison dans le temps etou dans lrsquoespace
Utiliteacute pour les professionnels
en vue drsquoune ameacutelioration des pratiques ou de benchmarking
Utiliteacute pour les usagers
transparence arguments pour un choix raisonneacute des ES par les usagers
PRINOI avril 2012 Page 6
Recommandations HICPAC 2005 (USA)
Indicateurs recommandeacutes (impact important possibiliteacute de preacutevention)
Bacteacuterieacutemies primaires en soins intensifs confirmeacutees microbiologiquement
ISO agrave la suite de certains types drsquointervention chirurgicale
Non recommandeacutes
Infections urinaires lieacutees au sondage veacutesical (moins drsquoimpact moins de possibiliteacute de preacutevention)
Pneumopathies associeacutees agrave la ventilation meacutecanique (impact important mais deacutetectabiliteacute faible-deacutefinition CDC relativement subjective)
Guidance on public reporting of healthcare-associated infections
recommendations of the Healthcare Infection Control Practices Advisory Committee McKibben L Horan T Tokars JI Fowler G Cardo DM Pearson ML Brennan PJ
Heathcare Infection Control Practices Advisory Committee Am J Infect Control 2005 May33(4)217-26
PRINOI avril 2012 Page 7
Technical Advisory Committee Maryland Health-Care Commission
laquo Outcomes raquo scores les plus eacuteleveacutes Bacteacuterieacutemies associeacutees aux CVC (2630)
ISO apregraves pontage coronarien (2230)
ISO apregraves prothegravese totale de hanche ou de genou (2130)
Autres laquo Outcomes raquo MRSAVRE en reacuteanimation
pneumopathie acquise sous ventilation meacutecanique infections urinaires sur sonde chirurgie sein hysteacuterectomie
Public reporting of health care-associated infections (HAIs) approach to choosing HAI measures Passaretti CL et al Infect Control Hosp Epidemiol 201132768-74
PRINOI avril 2012 Page 8
En Europe
Rapport obligatoire aupregraves des tutelles mais pas de mise agrave disposition du public Taux drsquoincidence des bacteacuterieacutemies
associeacutees aux CVC Taux drsquoincidence des pneumopathies
associeacutees agrave la ventilation meacutecanique Ratio drsquoexposition aux dispositifs
invasifs Volontariat (reacuteseau KISS)
Taux de colonisation agrave SARM Taux drsquoincidence des infections agrave
Clostridium difficile Taux drsquoincidence des ISO Taux drsquoincidence des bacteacuterieacutemies
associeacutees aux CVC Taux drsquoincidence des infections
urinaires associeacutees au sondage veacutesical Taux drsquoincidence des pneumopathies
associeacutees agrave la ventilation meacutecanique
Rapporteacutes publiquement Taux drsquoincidence des
bacteacuterieacutemies dues agrave des pathogegravenes speacutecifiques autres que SARM (SASM Ecoli)
Taux drsquoincidence des bacteacuterieacutemies agrave SARM
Taux drsquoincidence des infections agrave Clostridium difficile
Nb MRSA et ICD dans les 12 derniegraveres semaines
Nb semaines sans MRSA
Variable Taux drsquoincidence des ISO Taux de preacutevalence de la
reacutesistance microbienne
ALLEMAGNE ANGLETERRE
Haustein et al Lancet ID 2011
PRINOI avril 2012 Page 9
Calcul et expression de lrsquoindicateur
QUANTITATIF Taux de preacutevalence IAS Incidence IAS (taux ou
densiteacute) Taux de preacutelegravevements
cliniques Taux brut ou taux stratifieacute RSI= Nb observeacute IAS
Nb attendu IAS
QUALITATIF Preacutesence drsquoau moins 1 type
drsquoIAS (ouinon) pendant une peacuteriode donneacutee (hellip agrave deacutefinir)
Intervalle (nb semaines) sans IAS
Taux diffeacuterent du taux de reacutefeacuterence choisi (gt1 ou lt 1)
RSI gt1 ou lt1 SEMI-QUANTITATIF Taux compris dans un intervalle
deacutefini par un(des) quartile(s) drsquoune distribution de reacutefeacuterence (P50 P75 IQ hellip)
PRINOI avril 2012 Page 10
Indicateurs IAS expeacuteriences internationales Groupe drsquoexperts
(France 2003)
Pertinence Faisabiliteacute Qualiteacute
meacutetrologique
Utilisation
agrave lrsquoexteacuterieur de
lrsquoES
Passaretti et coll
(Maryland USA ICHE 2011)
Impact
Improvability
Feasibility
Frequency
Feasibility
(lsquoSystem for collecting
data already in placersquo)
Inclusiveness
Functionality
Mc Kibben et coll
(HICPAC USA AJIC 2006)
Severity
Preventability
Detectability
Accuracy
HELICS-IPSE
(Europe)
Clinical relevance Simplicity
Acceptability
Timeliness
Completeness
Validity
Sensitivity
Readibility
Pas de distingo par les usagers vs par les professionnels
PRINOI avril 2012 Page 11
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 12
DI bacteacuterieacutemies primaires 1000 jours de catheacuteteacuterisme veineux central
59
98
53
32
09 09 12 14
04 02
13 11
0
5
10
15
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
NNIS Cardiothoracic ICU
REA-RAISIN
Reacuteanimation CTCV (HGRL CHU Nantes)
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
1 Proceacutedure pose entretien des CVC et harmonisation de la deacutefinition
3 Audit drsquoobservation
4 V2 mode opeacuteratoire pose CVC
et feuille de suivi hors USI
2 Surveillance prospective et publication de lrsquoindicateur
5 V2 mode opeacuteratoire Reacutefection pansement et changement lignes
PRINOI avril 2012 Page 13
PRINOI avril 2012 Page 14
PRINOI avril 2012 Page 15
Objectifs quantifieacutes Ameacuteliorer la preacutevention des infections associeacutees
aux actes invasifs
Source Circulaire du 26 aoucirct 2009
PRINOI avril 2012 Page 16
PRINOI avril 2012 Page 17
DI infections urinaires 1000 jours de sondage urinaire
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
Proceacutedure sondage urinaire Mateacuteriel de
diuregravese horaire preacute-connecteacute
Formation des internes agrave chaque semestre au BO
PRINOI avril 2012 Page 18
Evolution des indicateurs du tableau de bord
Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires
Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques
Nouvelle organisation des ES et ARS (loi HPST)
Deacutefinition drsquoobjectifs quantifieacutes pour 2012
Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)
PRINOI avril 2012 Page 19
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 20
Nouveau rocircle des instances dans la politique de lutte contre les IN
Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins
ICALIN 2
ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50
PRINOI avril 2012 Page 21
Nouveauteacutes ICALIN 2 Protection du personnel
proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)
Suivi de la consommation des PHA et restitution au moins 1 fois par an
Check-list SU CVP CCI CVC CSC
Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique
Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau
Reacutesultats des surveillances preacutesenteacutes en CME
Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave
Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS
PCC PCAG sonde urinaire CVPCVCCCICSC
PRINOI avril 2012 Page 22
Indicateur ICABMR
PRINOI avril 2012 Page 23
Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008
PRINOI avril 2012 Page 24
Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008
PRINOI avril 2012 Page 25
Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline
au sein de lrsquoespegravece SA
PRINOI avril 2012 Page 26
Source httpwwwrivmnlearssdatabase
PRINOI avril 2012 Page 27
Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM
ne le sont pas sur les EBLSE
Que nous disent les indicateurs de surveillance
PRINOI avril 2012 Page 28
DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)
02018018
012007003
016
019024
026033
036039
05048043
0
01
02
03
04
05
06
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
DI EBLSE 1000 JH DI SARM 1000 JH
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 5
Critegraveres drsquoeacutevaluation drsquoun indicateur (2)
Qualiteacutes meacutetrologiques validiteacute (sensibiliteacute speacutecificiteacute VPP VPN) fiabiliteacute
reproductibiliteacute standardisation possibiliteacute de comparaison dans le temps etou dans lrsquoespace
Utiliteacute pour les professionnels
en vue drsquoune ameacutelioration des pratiques ou de benchmarking
Utiliteacute pour les usagers
transparence arguments pour un choix raisonneacute des ES par les usagers
PRINOI avril 2012 Page 6
Recommandations HICPAC 2005 (USA)
Indicateurs recommandeacutes (impact important possibiliteacute de preacutevention)
Bacteacuterieacutemies primaires en soins intensifs confirmeacutees microbiologiquement
ISO agrave la suite de certains types drsquointervention chirurgicale
Non recommandeacutes
Infections urinaires lieacutees au sondage veacutesical (moins drsquoimpact moins de possibiliteacute de preacutevention)
Pneumopathies associeacutees agrave la ventilation meacutecanique (impact important mais deacutetectabiliteacute faible-deacutefinition CDC relativement subjective)
Guidance on public reporting of healthcare-associated infections
recommendations of the Healthcare Infection Control Practices Advisory Committee McKibben L Horan T Tokars JI Fowler G Cardo DM Pearson ML Brennan PJ
Heathcare Infection Control Practices Advisory Committee Am J Infect Control 2005 May33(4)217-26
PRINOI avril 2012 Page 7
Technical Advisory Committee Maryland Health-Care Commission
laquo Outcomes raquo scores les plus eacuteleveacutes Bacteacuterieacutemies associeacutees aux CVC (2630)
ISO apregraves pontage coronarien (2230)
ISO apregraves prothegravese totale de hanche ou de genou (2130)
Autres laquo Outcomes raquo MRSAVRE en reacuteanimation
pneumopathie acquise sous ventilation meacutecanique infections urinaires sur sonde chirurgie sein hysteacuterectomie
Public reporting of health care-associated infections (HAIs) approach to choosing HAI measures Passaretti CL et al Infect Control Hosp Epidemiol 201132768-74
PRINOI avril 2012 Page 8
En Europe
Rapport obligatoire aupregraves des tutelles mais pas de mise agrave disposition du public Taux drsquoincidence des bacteacuterieacutemies
associeacutees aux CVC Taux drsquoincidence des pneumopathies
associeacutees agrave la ventilation meacutecanique Ratio drsquoexposition aux dispositifs
invasifs Volontariat (reacuteseau KISS)
Taux de colonisation agrave SARM Taux drsquoincidence des infections agrave
Clostridium difficile Taux drsquoincidence des ISO Taux drsquoincidence des bacteacuterieacutemies
associeacutees aux CVC Taux drsquoincidence des infections
urinaires associeacutees au sondage veacutesical Taux drsquoincidence des pneumopathies
associeacutees agrave la ventilation meacutecanique
Rapporteacutes publiquement Taux drsquoincidence des
bacteacuterieacutemies dues agrave des pathogegravenes speacutecifiques autres que SARM (SASM Ecoli)
Taux drsquoincidence des bacteacuterieacutemies agrave SARM
Taux drsquoincidence des infections agrave Clostridium difficile
Nb MRSA et ICD dans les 12 derniegraveres semaines
Nb semaines sans MRSA
Variable Taux drsquoincidence des ISO Taux de preacutevalence de la
reacutesistance microbienne
ALLEMAGNE ANGLETERRE
Haustein et al Lancet ID 2011
PRINOI avril 2012 Page 9
Calcul et expression de lrsquoindicateur
QUANTITATIF Taux de preacutevalence IAS Incidence IAS (taux ou
densiteacute) Taux de preacutelegravevements
cliniques Taux brut ou taux stratifieacute RSI= Nb observeacute IAS
Nb attendu IAS
QUALITATIF Preacutesence drsquoau moins 1 type
drsquoIAS (ouinon) pendant une peacuteriode donneacutee (hellip agrave deacutefinir)
Intervalle (nb semaines) sans IAS
Taux diffeacuterent du taux de reacutefeacuterence choisi (gt1 ou lt 1)
RSI gt1 ou lt1 SEMI-QUANTITATIF Taux compris dans un intervalle
deacutefini par un(des) quartile(s) drsquoune distribution de reacutefeacuterence (P50 P75 IQ hellip)
PRINOI avril 2012 Page 10
Indicateurs IAS expeacuteriences internationales Groupe drsquoexperts
(France 2003)
Pertinence Faisabiliteacute Qualiteacute
meacutetrologique
Utilisation
agrave lrsquoexteacuterieur de
lrsquoES
Passaretti et coll
(Maryland USA ICHE 2011)
Impact
Improvability
Feasibility
Frequency
Feasibility
(lsquoSystem for collecting
data already in placersquo)
Inclusiveness
Functionality
Mc Kibben et coll
(HICPAC USA AJIC 2006)
Severity
Preventability
Detectability
Accuracy
HELICS-IPSE
(Europe)
Clinical relevance Simplicity
Acceptability
Timeliness
Completeness
Validity
Sensitivity
Readibility
Pas de distingo par les usagers vs par les professionnels
PRINOI avril 2012 Page 11
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 12
DI bacteacuterieacutemies primaires 1000 jours de catheacuteteacuterisme veineux central
59
98
53
32
09 09 12 14
04 02
13 11
0
5
10
15
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
NNIS Cardiothoracic ICU
REA-RAISIN
Reacuteanimation CTCV (HGRL CHU Nantes)
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
1 Proceacutedure pose entretien des CVC et harmonisation de la deacutefinition
3 Audit drsquoobservation
4 V2 mode opeacuteratoire pose CVC
et feuille de suivi hors USI
2 Surveillance prospective et publication de lrsquoindicateur
5 V2 mode opeacuteratoire Reacutefection pansement et changement lignes
PRINOI avril 2012 Page 13
PRINOI avril 2012 Page 14
PRINOI avril 2012 Page 15
Objectifs quantifieacutes Ameacuteliorer la preacutevention des infections associeacutees
aux actes invasifs
Source Circulaire du 26 aoucirct 2009
PRINOI avril 2012 Page 16
PRINOI avril 2012 Page 17
DI infections urinaires 1000 jours de sondage urinaire
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
Proceacutedure sondage urinaire Mateacuteriel de
diuregravese horaire preacute-connecteacute
Formation des internes agrave chaque semestre au BO
PRINOI avril 2012 Page 18
Evolution des indicateurs du tableau de bord
Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires
Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques
Nouvelle organisation des ES et ARS (loi HPST)
Deacutefinition drsquoobjectifs quantifieacutes pour 2012
Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)
PRINOI avril 2012 Page 19
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 20
Nouveau rocircle des instances dans la politique de lutte contre les IN
Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins
ICALIN 2
ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50
PRINOI avril 2012 Page 21
Nouveauteacutes ICALIN 2 Protection du personnel
proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)
Suivi de la consommation des PHA et restitution au moins 1 fois par an
Check-list SU CVP CCI CVC CSC
Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique
Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau
Reacutesultats des surveillances preacutesenteacutes en CME
Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave
Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS
PCC PCAG sonde urinaire CVPCVCCCICSC
PRINOI avril 2012 Page 22
Indicateur ICABMR
PRINOI avril 2012 Page 23
Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008
PRINOI avril 2012 Page 24
Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008
PRINOI avril 2012 Page 25
Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline
au sein de lrsquoespegravece SA
PRINOI avril 2012 Page 26
Source httpwwwrivmnlearssdatabase
PRINOI avril 2012 Page 27
Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM
ne le sont pas sur les EBLSE
Que nous disent les indicateurs de surveillance
PRINOI avril 2012 Page 28
DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)
02018018
012007003
016
019024
026033
036039
05048043
0
01
02
03
04
05
06
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
DI EBLSE 1000 JH DI SARM 1000 JH
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 6
Recommandations HICPAC 2005 (USA)
Indicateurs recommandeacutes (impact important possibiliteacute de preacutevention)
Bacteacuterieacutemies primaires en soins intensifs confirmeacutees microbiologiquement
ISO agrave la suite de certains types drsquointervention chirurgicale
Non recommandeacutes
Infections urinaires lieacutees au sondage veacutesical (moins drsquoimpact moins de possibiliteacute de preacutevention)
Pneumopathies associeacutees agrave la ventilation meacutecanique (impact important mais deacutetectabiliteacute faible-deacutefinition CDC relativement subjective)
Guidance on public reporting of healthcare-associated infections
recommendations of the Healthcare Infection Control Practices Advisory Committee McKibben L Horan T Tokars JI Fowler G Cardo DM Pearson ML Brennan PJ
Heathcare Infection Control Practices Advisory Committee Am J Infect Control 2005 May33(4)217-26
PRINOI avril 2012 Page 7
Technical Advisory Committee Maryland Health-Care Commission
laquo Outcomes raquo scores les plus eacuteleveacutes Bacteacuterieacutemies associeacutees aux CVC (2630)
ISO apregraves pontage coronarien (2230)
ISO apregraves prothegravese totale de hanche ou de genou (2130)
Autres laquo Outcomes raquo MRSAVRE en reacuteanimation
pneumopathie acquise sous ventilation meacutecanique infections urinaires sur sonde chirurgie sein hysteacuterectomie
Public reporting of health care-associated infections (HAIs) approach to choosing HAI measures Passaretti CL et al Infect Control Hosp Epidemiol 201132768-74
PRINOI avril 2012 Page 8
En Europe
Rapport obligatoire aupregraves des tutelles mais pas de mise agrave disposition du public Taux drsquoincidence des bacteacuterieacutemies
associeacutees aux CVC Taux drsquoincidence des pneumopathies
associeacutees agrave la ventilation meacutecanique Ratio drsquoexposition aux dispositifs
invasifs Volontariat (reacuteseau KISS)
Taux de colonisation agrave SARM Taux drsquoincidence des infections agrave
Clostridium difficile Taux drsquoincidence des ISO Taux drsquoincidence des bacteacuterieacutemies
associeacutees aux CVC Taux drsquoincidence des infections
urinaires associeacutees au sondage veacutesical Taux drsquoincidence des pneumopathies
associeacutees agrave la ventilation meacutecanique
Rapporteacutes publiquement Taux drsquoincidence des
bacteacuterieacutemies dues agrave des pathogegravenes speacutecifiques autres que SARM (SASM Ecoli)
Taux drsquoincidence des bacteacuterieacutemies agrave SARM
Taux drsquoincidence des infections agrave Clostridium difficile
Nb MRSA et ICD dans les 12 derniegraveres semaines
Nb semaines sans MRSA
Variable Taux drsquoincidence des ISO Taux de preacutevalence de la
reacutesistance microbienne
ALLEMAGNE ANGLETERRE
Haustein et al Lancet ID 2011
PRINOI avril 2012 Page 9
Calcul et expression de lrsquoindicateur
QUANTITATIF Taux de preacutevalence IAS Incidence IAS (taux ou
densiteacute) Taux de preacutelegravevements
cliniques Taux brut ou taux stratifieacute RSI= Nb observeacute IAS
Nb attendu IAS
QUALITATIF Preacutesence drsquoau moins 1 type
drsquoIAS (ouinon) pendant une peacuteriode donneacutee (hellip agrave deacutefinir)
Intervalle (nb semaines) sans IAS
Taux diffeacuterent du taux de reacutefeacuterence choisi (gt1 ou lt 1)
RSI gt1 ou lt1 SEMI-QUANTITATIF Taux compris dans un intervalle
deacutefini par un(des) quartile(s) drsquoune distribution de reacutefeacuterence (P50 P75 IQ hellip)
PRINOI avril 2012 Page 10
Indicateurs IAS expeacuteriences internationales Groupe drsquoexperts
(France 2003)
Pertinence Faisabiliteacute Qualiteacute
meacutetrologique
Utilisation
agrave lrsquoexteacuterieur de
lrsquoES
Passaretti et coll
(Maryland USA ICHE 2011)
Impact
Improvability
Feasibility
Frequency
Feasibility
(lsquoSystem for collecting
data already in placersquo)
Inclusiveness
Functionality
Mc Kibben et coll
(HICPAC USA AJIC 2006)
Severity
Preventability
Detectability
Accuracy
HELICS-IPSE
(Europe)
Clinical relevance Simplicity
Acceptability
Timeliness
Completeness
Validity
Sensitivity
Readibility
Pas de distingo par les usagers vs par les professionnels
PRINOI avril 2012 Page 11
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 12
DI bacteacuterieacutemies primaires 1000 jours de catheacuteteacuterisme veineux central
59
98
53
32
09 09 12 14
04 02
13 11
0
5
10
15
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
NNIS Cardiothoracic ICU
REA-RAISIN
Reacuteanimation CTCV (HGRL CHU Nantes)
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
1 Proceacutedure pose entretien des CVC et harmonisation de la deacutefinition
3 Audit drsquoobservation
4 V2 mode opeacuteratoire pose CVC
et feuille de suivi hors USI
2 Surveillance prospective et publication de lrsquoindicateur
5 V2 mode opeacuteratoire Reacutefection pansement et changement lignes
PRINOI avril 2012 Page 13
PRINOI avril 2012 Page 14
PRINOI avril 2012 Page 15
Objectifs quantifieacutes Ameacuteliorer la preacutevention des infections associeacutees
aux actes invasifs
Source Circulaire du 26 aoucirct 2009
PRINOI avril 2012 Page 16
PRINOI avril 2012 Page 17
DI infections urinaires 1000 jours de sondage urinaire
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
Proceacutedure sondage urinaire Mateacuteriel de
diuregravese horaire preacute-connecteacute
Formation des internes agrave chaque semestre au BO
PRINOI avril 2012 Page 18
Evolution des indicateurs du tableau de bord
Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires
Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques
Nouvelle organisation des ES et ARS (loi HPST)
Deacutefinition drsquoobjectifs quantifieacutes pour 2012
Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)
PRINOI avril 2012 Page 19
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 20
Nouveau rocircle des instances dans la politique de lutte contre les IN
Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins
ICALIN 2
ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50
PRINOI avril 2012 Page 21
Nouveauteacutes ICALIN 2 Protection du personnel
proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)
Suivi de la consommation des PHA et restitution au moins 1 fois par an
Check-list SU CVP CCI CVC CSC
Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique
Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau
Reacutesultats des surveillances preacutesenteacutes en CME
Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave
Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS
PCC PCAG sonde urinaire CVPCVCCCICSC
PRINOI avril 2012 Page 22
Indicateur ICABMR
PRINOI avril 2012 Page 23
Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008
PRINOI avril 2012 Page 24
Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008
PRINOI avril 2012 Page 25
Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline
au sein de lrsquoespegravece SA
PRINOI avril 2012 Page 26
Source httpwwwrivmnlearssdatabase
PRINOI avril 2012 Page 27
Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM
ne le sont pas sur les EBLSE
Que nous disent les indicateurs de surveillance
PRINOI avril 2012 Page 28
DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)
02018018
012007003
016
019024
026033
036039
05048043
0
01
02
03
04
05
06
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
DI EBLSE 1000 JH DI SARM 1000 JH
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 7
Technical Advisory Committee Maryland Health-Care Commission
laquo Outcomes raquo scores les plus eacuteleveacutes Bacteacuterieacutemies associeacutees aux CVC (2630)
ISO apregraves pontage coronarien (2230)
ISO apregraves prothegravese totale de hanche ou de genou (2130)
Autres laquo Outcomes raquo MRSAVRE en reacuteanimation
pneumopathie acquise sous ventilation meacutecanique infections urinaires sur sonde chirurgie sein hysteacuterectomie
Public reporting of health care-associated infections (HAIs) approach to choosing HAI measures Passaretti CL et al Infect Control Hosp Epidemiol 201132768-74
PRINOI avril 2012 Page 8
En Europe
Rapport obligatoire aupregraves des tutelles mais pas de mise agrave disposition du public Taux drsquoincidence des bacteacuterieacutemies
associeacutees aux CVC Taux drsquoincidence des pneumopathies
associeacutees agrave la ventilation meacutecanique Ratio drsquoexposition aux dispositifs
invasifs Volontariat (reacuteseau KISS)
Taux de colonisation agrave SARM Taux drsquoincidence des infections agrave
Clostridium difficile Taux drsquoincidence des ISO Taux drsquoincidence des bacteacuterieacutemies
associeacutees aux CVC Taux drsquoincidence des infections
urinaires associeacutees au sondage veacutesical Taux drsquoincidence des pneumopathies
associeacutees agrave la ventilation meacutecanique
Rapporteacutes publiquement Taux drsquoincidence des
bacteacuterieacutemies dues agrave des pathogegravenes speacutecifiques autres que SARM (SASM Ecoli)
Taux drsquoincidence des bacteacuterieacutemies agrave SARM
Taux drsquoincidence des infections agrave Clostridium difficile
Nb MRSA et ICD dans les 12 derniegraveres semaines
Nb semaines sans MRSA
Variable Taux drsquoincidence des ISO Taux de preacutevalence de la
reacutesistance microbienne
ALLEMAGNE ANGLETERRE
Haustein et al Lancet ID 2011
PRINOI avril 2012 Page 9
Calcul et expression de lrsquoindicateur
QUANTITATIF Taux de preacutevalence IAS Incidence IAS (taux ou
densiteacute) Taux de preacutelegravevements
cliniques Taux brut ou taux stratifieacute RSI= Nb observeacute IAS
Nb attendu IAS
QUALITATIF Preacutesence drsquoau moins 1 type
drsquoIAS (ouinon) pendant une peacuteriode donneacutee (hellip agrave deacutefinir)
Intervalle (nb semaines) sans IAS
Taux diffeacuterent du taux de reacutefeacuterence choisi (gt1 ou lt 1)
RSI gt1 ou lt1 SEMI-QUANTITATIF Taux compris dans un intervalle
deacutefini par un(des) quartile(s) drsquoune distribution de reacutefeacuterence (P50 P75 IQ hellip)
PRINOI avril 2012 Page 10
Indicateurs IAS expeacuteriences internationales Groupe drsquoexperts
(France 2003)
Pertinence Faisabiliteacute Qualiteacute
meacutetrologique
Utilisation
agrave lrsquoexteacuterieur de
lrsquoES
Passaretti et coll
(Maryland USA ICHE 2011)
Impact
Improvability
Feasibility
Frequency
Feasibility
(lsquoSystem for collecting
data already in placersquo)
Inclusiveness
Functionality
Mc Kibben et coll
(HICPAC USA AJIC 2006)
Severity
Preventability
Detectability
Accuracy
HELICS-IPSE
(Europe)
Clinical relevance Simplicity
Acceptability
Timeliness
Completeness
Validity
Sensitivity
Readibility
Pas de distingo par les usagers vs par les professionnels
PRINOI avril 2012 Page 11
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 12
DI bacteacuterieacutemies primaires 1000 jours de catheacuteteacuterisme veineux central
59
98
53
32
09 09 12 14
04 02
13 11
0
5
10
15
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
NNIS Cardiothoracic ICU
REA-RAISIN
Reacuteanimation CTCV (HGRL CHU Nantes)
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
1 Proceacutedure pose entretien des CVC et harmonisation de la deacutefinition
3 Audit drsquoobservation
4 V2 mode opeacuteratoire pose CVC
et feuille de suivi hors USI
2 Surveillance prospective et publication de lrsquoindicateur
5 V2 mode opeacuteratoire Reacutefection pansement et changement lignes
PRINOI avril 2012 Page 13
PRINOI avril 2012 Page 14
PRINOI avril 2012 Page 15
Objectifs quantifieacutes Ameacuteliorer la preacutevention des infections associeacutees
aux actes invasifs
Source Circulaire du 26 aoucirct 2009
PRINOI avril 2012 Page 16
PRINOI avril 2012 Page 17
DI infections urinaires 1000 jours de sondage urinaire
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
Proceacutedure sondage urinaire Mateacuteriel de
diuregravese horaire preacute-connecteacute
Formation des internes agrave chaque semestre au BO
PRINOI avril 2012 Page 18
Evolution des indicateurs du tableau de bord
Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires
Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques
Nouvelle organisation des ES et ARS (loi HPST)
Deacutefinition drsquoobjectifs quantifieacutes pour 2012
Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)
PRINOI avril 2012 Page 19
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 20
Nouveau rocircle des instances dans la politique de lutte contre les IN
Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins
ICALIN 2
ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50
PRINOI avril 2012 Page 21
Nouveauteacutes ICALIN 2 Protection du personnel
proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)
Suivi de la consommation des PHA et restitution au moins 1 fois par an
Check-list SU CVP CCI CVC CSC
Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique
Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau
Reacutesultats des surveillances preacutesenteacutes en CME
Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave
Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS
PCC PCAG sonde urinaire CVPCVCCCICSC
PRINOI avril 2012 Page 22
Indicateur ICABMR
PRINOI avril 2012 Page 23
Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008
PRINOI avril 2012 Page 24
Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008
PRINOI avril 2012 Page 25
Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline
au sein de lrsquoespegravece SA
PRINOI avril 2012 Page 26
Source httpwwwrivmnlearssdatabase
PRINOI avril 2012 Page 27
Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM
ne le sont pas sur les EBLSE
Que nous disent les indicateurs de surveillance
PRINOI avril 2012 Page 28
DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)
02018018
012007003
016
019024
026033
036039
05048043
0
01
02
03
04
05
06
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
DI EBLSE 1000 JH DI SARM 1000 JH
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 8
En Europe
Rapport obligatoire aupregraves des tutelles mais pas de mise agrave disposition du public Taux drsquoincidence des bacteacuterieacutemies
associeacutees aux CVC Taux drsquoincidence des pneumopathies
associeacutees agrave la ventilation meacutecanique Ratio drsquoexposition aux dispositifs
invasifs Volontariat (reacuteseau KISS)
Taux de colonisation agrave SARM Taux drsquoincidence des infections agrave
Clostridium difficile Taux drsquoincidence des ISO Taux drsquoincidence des bacteacuterieacutemies
associeacutees aux CVC Taux drsquoincidence des infections
urinaires associeacutees au sondage veacutesical Taux drsquoincidence des pneumopathies
associeacutees agrave la ventilation meacutecanique
Rapporteacutes publiquement Taux drsquoincidence des
bacteacuterieacutemies dues agrave des pathogegravenes speacutecifiques autres que SARM (SASM Ecoli)
Taux drsquoincidence des bacteacuterieacutemies agrave SARM
Taux drsquoincidence des infections agrave Clostridium difficile
Nb MRSA et ICD dans les 12 derniegraveres semaines
Nb semaines sans MRSA
Variable Taux drsquoincidence des ISO Taux de preacutevalence de la
reacutesistance microbienne
ALLEMAGNE ANGLETERRE
Haustein et al Lancet ID 2011
PRINOI avril 2012 Page 9
Calcul et expression de lrsquoindicateur
QUANTITATIF Taux de preacutevalence IAS Incidence IAS (taux ou
densiteacute) Taux de preacutelegravevements
cliniques Taux brut ou taux stratifieacute RSI= Nb observeacute IAS
Nb attendu IAS
QUALITATIF Preacutesence drsquoau moins 1 type
drsquoIAS (ouinon) pendant une peacuteriode donneacutee (hellip agrave deacutefinir)
Intervalle (nb semaines) sans IAS
Taux diffeacuterent du taux de reacutefeacuterence choisi (gt1 ou lt 1)
RSI gt1 ou lt1 SEMI-QUANTITATIF Taux compris dans un intervalle
deacutefini par un(des) quartile(s) drsquoune distribution de reacutefeacuterence (P50 P75 IQ hellip)
PRINOI avril 2012 Page 10
Indicateurs IAS expeacuteriences internationales Groupe drsquoexperts
(France 2003)
Pertinence Faisabiliteacute Qualiteacute
meacutetrologique
Utilisation
agrave lrsquoexteacuterieur de
lrsquoES
Passaretti et coll
(Maryland USA ICHE 2011)
Impact
Improvability
Feasibility
Frequency
Feasibility
(lsquoSystem for collecting
data already in placersquo)
Inclusiveness
Functionality
Mc Kibben et coll
(HICPAC USA AJIC 2006)
Severity
Preventability
Detectability
Accuracy
HELICS-IPSE
(Europe)
Clinical relevance Simplicity
Acceptability
Timeliness
Completeness
Validity
Sensitivity
Readibility
Pas de distingo par les usagers vs par les professionnels
PRINOI avril 2012 Page 11
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 12
DI bacteacuterieacutemies primaires 1000 jours de catheacuteteacuterisme veineux central
59
98
53
32
09 09 12 14
04 02
13 11
0
5
10
15
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
NNIS Cardiothoracic ICU
REA-RAISIN
Reacuteanimation CTCV (HGRL CHU Nantes)
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
1 Proceacutedure pose entretien des CVC et harmonisation de la deacutefinition
3 Audit drsquoobservation
4 V2 mode opeacuteratoire pose CVC
et feuille de suivi hors USI
2 Surveillance prospective et publication de lrsquoindicateur
5 V2 mode opeacuteratoire Reacutefection pansement et changement lignes
PRINOI avril 2012 Page 13
PRINOI avril 2012 Page 14
PRINOI avril 2012 Page 15
Objectifs quantifieacutes Ameacuteliorer la preacutevention des infections associeacutees
aux actes invasifs
Source Circulaire du 26 aoucirct 2009
PRINOI avril 2012 Page 16
PRINOI avril 2012 Page 17
DI infections urinaires 1000 jours de sondage urinaire
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
Proceacutedure sondage urinaire Mateacuteriel de
diuregravese horaire preacute-connecteacute
Formation des internes agrave chaque semestre au BO
PRINOI avril 2012 Page 18
Evolution des indicateurs du tableau de bord
Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires
Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques
Nouvelle organisation des ES et ARS (loi HPST)
Deacutefinition drsquoobjectifs quantifieacutes pour 2012
Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)
PRINOI avril 2012 Page 19
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 20
Nouveau rocircle des instances dans la politique de lutte contre les IN
Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins
ICALIN 2
ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50
PRINOI avril 2012 Page 21
Nouveauteacutes ICALIN 2 Protection du personnel
proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)
Suivi de la consommation des PHA et restitution au moins 1 fois par an
Check-list SU CVP CCI CVC CSC
Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique
Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau
Reacutesultats des surveillances preacutesenteacutes en CME
Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave
Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS
PCC PCAG sonde urinaire CVPCVCCCICSC
PRINOI avril 2012 Page 22
Indicateur ICABMR
PRINOI avril 2012 Page 23
Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008
PRINOI avril 2012 Page 24
Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008
PRINOI avril 2012 Page 25
Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline
au sein de lrsquoespegravece SA
PRINOI avril 2012 Page 26
Source httpwwwrivmnlearssdatabase
PRINOI avril 2012 Page 27
Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM
ne le sont pas sur les EBLSE
Que nous disent les indicateurs de surveillance
PRINOI avril 2012 Page 28
DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)
02018018
012007003
016
019024
026033
036039
05048043
0
01
02
03
04
05
06
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
DI EBLSE 1000 JH DI SARM 1000 JH
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 9
Calcul et expression de lrsquoindicateur
QUANTITATIF Taux de preacutevalence IAS Incidence IAS (taux ou
densiteacute) Taux de preacutelegravevements
cliniques Taux brut ou taux stratifieacute RSI= Nb observeacute IAS
Nb attendu IAS
QUALITATIF Preacutesence drsquoau moins 1 type
drsquoIAS (ouinon) pendant une peacuteriode donneacutee (hellip agrave deacutefinir)
Intervalle (nb semaines) sans IAS
Taux diffeacuterent du taux de reacutefeacuterence choisi (gt1 ou lt 1)
RSI gt1 ou lt1 SEMI-QUANTITATIF Taux compris dans un intervalle
deacutefini par un(des) quartile(s) drsquoune distribution de reacutefeacuterence (P50 P75 IQ hellip)
PRINOI avril 2012 Page 10
Indicateurs IAS expeacuteriences internationales Groupe drsquoexperts
(France 2003)
Pertinence Faisabiliteacute Qualiteacute
meacutetrologique
Utilisation
agrave lrsquoexteacuterieur de
lrsquoES
Passaretti et coll
(Maryland USA ICHE 2011)
Impact
Improvability
Feasibility
Frequency
Feasibility
(lsquoSystem for collecting
data already in placersquo)
Inclusiveness
Functionality
Mc Kibben et coll
(HICPAC USA AJIC 2006)
Severity
Preventability
Detectability
Accuracy
HELICS-IPSE
(Europe)
Clinical relevance Simplicity
Acceptability
Timeliness
Completeness
Validity
Sensitivity
Readibility
Pas de distingo par les usagers vs par les professionnels
PRINOI avril 2012 Page 11
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 12
DI bacteacuterieacutemies primaires 1000 jours de catheacuteteacuterisme veineux central
59
98
53
32
09 09 12 14
04 02
13 11
0
5
10
15
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
NNIS Cardiothoracic ICU
REA-RAISIN
Reacuteanimation CTCV (HGRL CHU Nantes)
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
1 Proceacutedure pose entretien des CVC et harmonisation de la deacutefinition
3 Audit drsquoobservation
4 V2 mode opeacuteratoire pose CVC
et feuille de suivi hors USI
2 Surveillance prospective et publication de lrsquoindicateur
5 V2 mode opeacuteratoire Reacutefection pansement et changement lignes
PRINOI avril 2012 Page 13
PRINOI avril 2012 Page 14
PRINOI avril 2012 Page 15
Objectifs quantifieacutes Ameacuteliorer la preacutevention des infections associeacutees
aux actes invasifs
Source Circulaire du 26 aoucirct 2009
PRINOI avril 2012 Page 16
PRINOI avril 2012 Page 17
DI infections urinaires 1000 jours de sondage urinaire
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
Proceacutedure sondage urinaire Mateacuteriel de
diuregravese horaire preacute-connecteacute
Formation des internes agrave chaque semestre au BO
PRINOI avril 2012 Page 18
Evolution des indicateurs du tableau de bord
Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires
Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques
Nouvelle organisation des ES et ARS (loi HPST)
Deacutefinition drsquoobjectifs quantifieacutes pour 2012
Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)
PRINOI avril 2012 Page 19
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 20
Nouveau rocircle des instances dans la politique de lutte contre les IN
Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins
ICALIN 2
ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50
PRINOI avril 2012 Page 21
Nouveauteacutes ICALIN 2 Protection du personnel
proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)
Suivi de la consommation des PHA et restitution au moins 1 fois par an
Check-list SU CVP CCI CVC CSC
Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique
Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau
Reacutesultats des surveillances preacutesenteacutes en CME
Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave
Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS
PCC PCAG sonde urinaire CVPCVCCCICSC
PRINOI avril 2012 Page 22
Indicateur ICABMR
PRINOI avril 2012 Page 23
Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008
PRINOI avril 2012 Page 24
Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008
PRINOI avril 2012 Page 25
Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline
au sein de lrsquoespegravece SA
PRINOI avril 2012 Page 26
Source httpwwwrivmnlearssdatabase
PRINOI avril 2012 Page 27
Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM
ne le sont pas sur les EBLSE
Que nous disent les indicateurs de surveillance
PRINOI avril 2012 Page 28
DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)
02018018
012007003
016
019024
026033
036039
05048043
0
01
02
03
04
05
06
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
DI EBLSE 1000 JH DI SARM 1000 JH
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 10
Indicateurs IAS expeacuteriences internationales Groupe drsquoexperts
(France 2003)
Pertinence Faisabiliteacute Qualiteacute
meacutetrologique
Utilisation
agrave lrsquoexteacuterieur de
lrsquoES
Passaretti et coll
(Maryland USA ICHE 2011)
Impact
Improvability
Feasibility
Frequency
Feasibility
(lsquoSystem for collecting
data already in placersquo)
Inclusiveness
Functionality
Mc Kibben et coll
(HICPAC USA AJIC 2006)
Severity
Preventability
Detectability
Accuracy
HELICS-IPSE
(Europe)
Clinical relevance Simplicity
Acceptability
Timeliness
Completeness
Validity
Sensitivity
Readibility
Pas de distingo par les usagers vs par les professionnels
PRINOI avril 2012 Page 11
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 12
DI bacteacuterieacutemies primaires 1000 jours de catheacuteteacuterisme veineux central
59
98
53
32
09 09 12 14
04 02
13 11
0
5
10
15
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
NNIS Cardiothoracic ICU
REA-RAISIN
Reacuteanimation CTCV (HGRL CHU Nantes)
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
1 Proceacutedure pose entretien des CVC et harmonisation de la deacutefinition
3 Audit drsquoobservation
4 V2 mode opeacuteratoire pose CVC
et feuille de suivi hors USI
2 Surveillance prospective et publication de lrsquoindicateur
5 V2 mode opeacuteratoire Reacutefection pansement et changement lignes
PRINOI avril 2012 Page 13
PRINOI avril 2012 Page 14
PRINOI avril 2012 Page 15
Objectifs quantifieacutes Ameacuteliorer la preacutevention des infections associeacutees
aux actes invasifs
Source Circulaire du 26 aoucirct 2009
PRINOI avril 2012 Page 16
PRINOI avril 2012 Page 17
DI infections urinaires 1000 jours de sondage urinaire
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
Proceacutedure sondage urinaire Mateacuteriel de
diuregravese horaire preacute-connecteacute
Formation des internes agrave chaque semestre au BO
PRINOI avril 2012 Page 18
Evolution des indicateurs du tableau de bord
Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires
Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques
Nouvelle organisation des ES et ARS (loi HPST)
Deacutefinition drsquoobjectifs quantifieacutes pour 2012
Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)
PRINOI avril 2012 Page 19
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 20
Nouveau rocircle des instances dans la politique de lutte contre les IN
Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins
ICALIN 2
ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50
PRINOI avril 2012 Page 21
Nouveauteacutes ICALIN 2 Protection du personnel
proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)
Suivi de la consommation des PHA et restitution au moins 1 fois par an
Check-list SU CVP CCI CVC CSC
Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique
Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau
Reacutesultats des surveillances preacutesenteacutes en CME
Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave
Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS
PCC PCAG sonde urinaire CVPCVCCCICSC
PRINOI avril 2012 Page 22
Indicateur ICABMR
PRINOI avril 2012 Page 23
Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008
PRINOI avril 2012 Page 24
Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008
PRINOI avril 2012 Page 25
Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline
au sein de lrsquoespegravece SA
PRINOI avril 2012 Page 26
Source httpwwwrivmnlearssdatabase
PRINOI avril 2012 Page 27
Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM
ne le sont pas sur les EBLSE
Que nous disent les indicateurs de surveillance
PRINOI avril 2012 Page 28
DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)
02018018
012007003
016
019024
026033
036039
05048043
0
01
02
03
04
05
06
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
DI EBLSE 1000 JH DI SARM 1000 JH
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 11
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 12
DI bacteacuterieacutemies primaires 1000 jours de catheacuteteacuterisme veineux central
59
98
53
32
09 09 12 14
04 02
13 11
0
5
10
15
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
NNIS Cardiothoracic ICU
REA-RAISIN
Reacuteanimation CTCV (HGRL CHU Nantes)
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
1 Proceacutedure pose entretien des CVC et harmonisation de la deacutefinition
3 Audit drsquoobservation
4 V2 mode opeacuteratoire pose CVC
et feuille de suivi hors USI
2 Surveillance prospective et publication de lrsquoindicateur
5 V2 mode opeacuteratoire Reacutefection pansement et changement lignes
PRINOI avril 2012 Page 13
PRINOI avril 2012 Page 14
PRINOI avril 2012 Page 15
Objectifs quantifieacutes Ameacuteliorer la preacutevention des infections associeacutees
aux actes invasifs
Source Circulaire du 26 aoucirct 2009
PRINOI avril 2012 Page 16
PRINOI avril 2012 Page 17
DI infections urinaires 1000 jours de sondage urinaire
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
Proceacutedure sondage urinaire Mateacuteriel de
diuregravese horaire preacute-connecteacute
Formation des internes agrave chaque semestre au BO
PRINOI avril 2012 Page 18
Evolution des indicateurs du tableau de bord
Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires
Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques
Nouvelle organisation des ES et ARS (loi HPST)
Deacutefinition drsquoobjectifs quantifieacutes pour 2012
Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)
PRINOI avril 2012 Page 19
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 20
Nouveau rocircle des instances dans la politique de lutte contre les IN
Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins
ICALIN 2
ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50
PRINOI avril 2012 Page 21
Nouveauteacutes ICALIN 2 Protection du personnel
proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)
Suivi de la consommation des PHA et restitution au moins 1 fois par an
Check-list SU CVP CCI CVC CSC
Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique
Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau
Reacutesultats des surveillances preacutesenteacutes en CME
Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave
Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS
PCC PCAG sonde urinaire CVPCVCCCICSC
PRINOI avril 2012 Page 22
Indicateur ICABMR
PRINOI avril 2012 Page 23
Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008
PRINOI avril 2012 Page 24
Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008
PRINOI avril 2012 Page 25
Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline
au sein de lrsquoespegravece SA
PRINOI avril 2012 Page 26
Source httpwwwrivmnlearssdatabase
PRINOI avril 2012 Page 27
Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM
ne le sont pas sur les EBLSE
Que nous disent les indicateurs de surveillance
PRINOI avril 2012 Page 28
DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)
02018018
012007003
016
019024
026033
036039
05048043
0
01
02
03
04
05
06
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
DI EBLSE 1000 JH DI SARM 1000 JH
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 12
DI bacteacuterieacutemies primaires 1000 jours de catheacuteteacuterisme veineux central
59
98
53
32
09 09 12 14
04 02
13 11
0
5
10
15
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
NNIS Cardiothoracic ICU
REA-RAISIN
Reacuteanimation CTCV (HGRL CHU Nantes)
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
1 Proceacutedure pose entretien des CVC et harmonisation de la deacutefinition
3 Audit drsquoobservation
4 V2 mode opeacuteratoire pose CVC
et feuille de suivi hors USI
2 Surveillance prospective et publication de lrsquoindicateur
5 V2 mode opeacuteratoire Reacutefection pansement et changement lignes
PRINOI avril 2012 Page 13
PRINOI avril 2012 Page 14
PRINOI avril 2012 Page 15
Objectifs quantifieacutes Ameacuteliorer la preacutevention des infections associeacutees
aux actes invasifs
Source Circulaire du 26 aoucirct 2009
PRINOI avril 2012 Page 16
PRINOI avril 2012 Page 17
DI infections urinaires 1000 jours de sondage urinaire
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
Proceacutedure sondage urinaire Mateacuteriel de
diuregravese horaire preacute-connecteacute
Formation des internes agrave chaque semestre au BO
PRINOI avril 2012 Page 18
Evolution des indicateurs du tableau de bord
Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires
Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques
Nouvelle organisation des ES et ARS (loi HPST)
Deacutefinition drsquoobjectifs quantifieacutes pour 2012
Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)
PRINOI avril 2012 Page 19
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 20
Nouveau rocircle des instances dans la politique de lutte contre les IN
Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins
ICALIN 2
ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50
PRINOI avril 2012 Page 21
Nouveauteacutes ICALIN 2 Protection du personnel
proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)
Suivi de la consommation des PHA et restitution au moins 1 fois par an
Check-list SU CVP CCI CVC CSC
Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique
Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau
Reacutesultats des surveillances preacutesenteacutes en CME
Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave
Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS
PCC PCAG sonde urinaire CVPCVCCCICSC
PRINOI avril 2012 Page 22
Indicateur ICABMR
PRINOI avril 2012 Page 23
Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008
PRINOI avril 2012 Page 24
Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008
PRINOI avril 2012 Page 25
Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline
au sein de lrsquoespegravece SA
PRINOI avril 2012 Page 26
Source httpwwwrivmnlearssdatabase
PRINOI avril 2012 Page 27
Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM
ne le sont pas sur les EBLSE
Que nous disent les indicateurs de surveillance
PRINOI avril 2012 Page 28
DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)
02018018
012007003
016
019024
026033
036039
05048043
0
01
02
03
04
05
06
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
DI EBLSE 1000 JH DI SARM 1000 JH
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 13
PRINOI avril 2012 Page 14
PRINOI avril 2012 Page 15
Objectifs quantifieacutes Ameacuteliorer la preacutevention des infections associeacutees
aux actes invasifs
Source Circulaire du 26 aoucirct 2009
PRINOI avril 2012 Page 16
PRINOI avril 2012 Page 17
DI infections urinaires 1000 jours de sondage urinaire
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
Proceacutedure sondage urinaire Mateacuteriel de
diuregravese horaire preacute-connecteacute
Formation des internes agrave chaque semestre au BO
PRINOI avril 2012 Page 18
Evolution des indicateurs du tableau de bord
Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires
Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques
Nouvelle organisation des ES et ARS (loi HPST)
Deacutefinition drsquoobjectifs quantifieacutes pour 2012
Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)
PRINOI avril 2012 Page 19
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 20
Nouveau rocircle des instances dans la politique de lutte contre les IN
Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins
ICALIN 2
ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50
PRINOI avril 2012 Page 21
Nouveauteacutes ICALIN 2 Protection du personnel
proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)
Suivi de la consommation des PHA et restitution au moins 1 fois par an
Check-list SU CVP CCI CVC CSC
Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique
Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau
Reacutesultats des surveillances preacutesenteacutes en CME
Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave
Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS
PCC PCAG sonde urinaire CVPCVCCCICSC
PRINOI avril 2012 Page 22
Indicateur ICABMR
PRINOI avril 2012 Page 23
Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008
PRINOI avril 2012 Page 24
Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008
PRINOI avril 2012 Page 25
Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline
au sein de lrsquoespegravece SA
PRINOI avril 2012 Page 26
Source httpwwwrivmnlearssdatabase
PRINOI avril 2012 Page 27
Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM
ne le sont pas sur les EBLSE
Que nous disent les indicateurs de surveillance
PRINOI avril 2012 Page 28
DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)
02018018
012007003
016
019024
026033
036039
05048043
0
01
02
03
04
05
06
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
DI EBLSE 1000 JH DI SARM 1000 JH
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 14
PRINOI avril 2012 Page 15
Objectifs quantifieacutes Ameacuteliorer la preacutevention des infections associeacutees
aux actes invasifs
Source Circulaire du 26 aoucirct 2009
PRINOI avril 2012 Page 16
PRINOI avril 2012 Page 17
DI infections urinaires 1000 jours de sondage urinaire
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
Proceacutedure sondage urinaire Mateacuteriel de
diuregravese horaire preacute-connecteacute
Formation des internes agrave chaque semestre au BO
PRINOI avril 2012 Page 18
Evolution des indicateurs du tableau de bord
Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires
Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques
Nouvelle organisation des ES et ARS (loi HPST)
Deacutefinition drsquoobjectifs quantifieacutes pour 2012
Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)
PRINOI avril 2012 Page 19
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 20
Nouveau rocircle des instances dans la politique de lutte contre les IN
Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins
ICALIN 2
ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50
PRINOI avril 2012 Page 21
Nouveauteacutes ICALIN 2 Protection du personnel
proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)
Suivi de la consommation des PHA et restitution au moins 1 fois par an
Check-list SU CVP CCI CVC CSC
Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique
Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau
Reacutesultats des surveillances preacutesenteacutes en CME
Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave
Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS
PCC PCAG sonde urinaire CVPCVCCCICSC
PRINOI avril 2012 Page 22
Indicateur ICABMR
PRINOI avril 2012 Page 23
Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008
PRINOI avril 2012 Page 24
Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008
PRINOI avril 2012 Page 25
Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline
au sein de lrsquoespegravece SA
PRINOI avril 2012 Page 26
Source httpwwwrivmnlearssdatabase
PRINOI avril 2012 Page 27
Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM
ne le sont pas sur les EBLSE
Que nous disent les indicateurs de surveillance
PRINOI avril 2012 Page 28
DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)
02018018
012007003
016
019024
026033
036039
05048043
0
01
02
03
04
05
06
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
DI EBLSE 1000 JH DI SARM 1000 JH
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 15
Objectifs quantifieacutes Ameacuteliorer la preacutevention des infections associeacutees
aux actes invasifs
Source Circulaire du 26 aoucirct 2009
PRINOI avril 2012 Page 16
PRINOI avril 2012 Page 17
DI infections urinaires 1000 jours de sondage urinaire
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
Proceacutedure sondage urinaire Mateacuteriel de
diuregravese horaire preacute-connecteacute
Formation des internes agrave chaque semestre au BO
PRINOI avril 2012 Page 18
Evolution des indicateurs du tableau de bord
Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires
Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques
Nouvelle organisation des ES et ARS (loi HPST)
Deacutefinition drsquoobjectifs quantifieacutes pour 2012
Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)
PRINOI avril 2012 Page 19
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 20
Nouveau rocircle des instances dans la politique de lutte contre les IN
Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins
ICALIN 2
ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50
PRINOI avril 2012 Page 21
Nouveauteacutes ICALIN 2 Protection du personnel
proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)
Suivi de la consommation des PHA et restitution au moins 1 fois par an
Check-list SU CVP CCI CVC CSC
Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique
Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau
Reacutesultats des surveillances preacutesenteacutes en CME
Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave
Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS
PCC PCAG sonde urinaire CVPCVCCCICSC
PRINOI avril 2012 Page 22
Indicateur ICABMR
PRINOI avril 2012 Page 23
Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008
PRINOI avril 2012 Page 24
Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008
PRINOI avril 2012 Page 25
Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline
au sein de lrsquoespegravece SA
PRINOI avril 2012 Page 26
Source httpwwwrivmnlearssdatabase
PRINOI avril 2012 Page 27
Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM
ne le sont pas sur les EBLSE
Que nous disent les indicateurs de surveillance
PRINOI avril 2012 Page 28
DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)
02018018
012007003
016
019024
026033
036039
05048043
0
01
02
03
04
05
06
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
DI EBLSE 1000 JH DI SARM 1000 JH
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 16
PRINOI avril 2012 Page 17
DI infections urinaires 1000 jours de sondage urinaire
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
Proceacutedure sondage urinaire Mateacuteriel de
diuregravese horaire preacute-connecteacute
Formation des internes agrave chaque semestre au BO
PRINOI avril 2012 Page 18
Evolution des indicateurs du tableau de bord
Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires
Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques
Nouvelle organisation des ES et ARS (loi HPST)
Deacutefinition drsquoobjectifs quantifieacutes pour 2012
Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)
PRINOI avril 2012 Page 19
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 20
Nouveau rocircle des instances dans la politique de lutte contre les IN
Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins
ICALIN 2
ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50
PRINOI avril 2012 Page 21
Nouveauteacutes ICALIN 2 Protection du personnel
proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)
Suivi de la consommation des PHA et restitution au moins 1 fois par an
Check-list SU CVP CCI CVC CSC
Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique
Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau
Reacutesultats des surveillances preacutesenteacutes en CME
Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave
Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS
PCC PCAG sonde urinaire CVPCVCCCICSC
PRINOI avril 2012 Page 22
Indicateur ICABMR
PRINOI avril 2012 Page 23
Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008
PRINOI avril 2012 Page 24
Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008
PRINOI avril 2012 Page 25
Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline
au sein de lrsquoespegravece SA
PRINOI avril 2012 Page 26
Source httpwwwrivmnlearssdatabase
PRINOI avril 2012 Page 27
Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM
ne le sont pas sur les EBLSE
Que nous disent les indicateurs de surveillance
PRINOI avril 2012 Page 28
DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)
02018018
012007003
016
019024
026033
036039
05048043
0
01
02
03
04
05
06
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
DI EBLSE 1000 JH DI SARM 1000 JH
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 17
DI infections urinaires 1000 jours de sondage urinaire
National Healthcare Safety Network (NHSN) report data summary for 2006 through 2008 issued December 2009
Proceacutedure sondage urinaire Mateacuteriel de
diuregravese horaire preacute-connecteacute
Formation des internes agrave chaque semestre au BO
PRINOI avril 2012 Page 18
Evolution des indicateurs du tableau de bord
Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires
Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques
Nouvelle organisation des ES et ARS (loi HPST)
Deacutefinition drsquoobjectifs quantifieacutes pour 2012
Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)
PRINOI avril 2012 Page 19
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 20
Nouveau rocircle des instances dans la politique de lutte contre les IN
Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins
ICALIN 2
ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50
PRINOI avril 2012 Page 21
Nouveauteacutes ICALIN 2 Protection du personnel
proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)
Suivi de la consommation des PHA et restitution au moins 1 fois par an
Check-list SU CVP CCI CVC CSC
Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique
Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau
Reacutesultats des surveillances preacutesenteacutes en CME
Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave
Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS
PCC PCAG sonde urinaire CVPCVCCCICSC
PRINOI avril 2012 Page 22
Indicateur ICABMR
PRINOI avril 2012 Page 23
Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008
PRINOI avril 2012 Page 24
Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008
PRINOI avril 2012 Page 25
Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline
au sein de lrsquoespegravece SA
PRINOI avril 2012 Page 26
Source httpwwwrivmnlearssdatabase
PRINOI avril 2012 Page 27
Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM
ne le sont pas sur les EBLSE
Que nous disent les indicateurs de surveillance
PRINOI avril 2012 Page 28
DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)
02018018
012007003
016
019024
026033
036039
05048043
0
01
02
03
04
05
06
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
DI EBLSE 1000 JH DI SARM 1000 JH
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 18
Evolution des indicateurs du tableau de bord
Programme de preacutevention des infections associeacutees aux soins et sa deacuteclinaison pour les ES (PROPIN 2009-2013) Progresser sur des domaines prioritaires
Maicirctrise du risque infectieux lieacute aux dispositifs invasifs Maicirctrise de la reacutesistance bacteacuterienne aux antibiotiques
Nouvelle organisation des ES et ARS (loi HPST)
Deacutefinition drsquoobjectifs quantifieacutes pour 2012
Indicateurs de reacutesultats (travail en cours du HCSP sur saisine de la DGS)
PRINOI avril 2012 Page 19
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 20
Nouveau rocircle des instances dans la politique de lutte contre les IN
Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins
ICALIN 2
ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50
PRINOI avril 2012 Page 21
Nouveauteacutes ICALIN 2 Protection du personnel
proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)
Suivi de la consommation des PHA et restitution au moins 1 fois par an
Check-list SU CVP CCI CVC CSC
Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique
Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau
Reacutesultats des surveillances preacutesenteacutes en CME
Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave
Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS
PCC PCAG sonde urinaire CVPCVCCCICSC
PRINOI avril 2012 Page 22
Indicateur ICABMR
PRINOI avril 2012 Page 23
Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008
PRINOI avril 2012 Page 24
Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008
PRINOI avril 2012 Page 25
Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline
au sein de lrsquoespegravece SA
PRINOI avril 2012 Page 26
Source httpwwwrivmnlearssdatabase
PRINOI avril 2012 Page 27
Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM
ne le sont pas sur les EBLSE
Que nous disent les indicateurs de surveillance
PRINOI avril 2012 Page 28
DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)
02018018
012007003
016
019024
026033
036039
05048043
0
01
02
03
04
05
06
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
DI EBLSE 1000 JH DI SARM 1000 JH
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 19
Tableau de bord des IN
Indicateurs 1e geacuteneacuteration fondamentaux drsquoune preacutevention des IN
Mise en place en 2006
ICALIN ICSHA SURVISO ICATB Indice de SARM Score agreacutegeacute
Indicateurs de 2e geacuteneacuteration
Nouvelle eacutetape drsquoameacutelioration des pratiques
Mise en place en 2012
ICALIN2 ICHSHA2 ICALISO ICABMR Indice SARM Score agreacutegeacute
Am
eacutelio
ratio
n de
s ES
PRINOI avril 2012 Page 20
Nouveau rocircle des instances dans la politique de lutte contre les IN
Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins
ICALIN 2
ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50
PRINOI avril 2012 Page 21
Nouveauteacutes ICALIN 2 Protection du personnel
proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)
Suivi de la consommation des PHA et restitution au moins 1 fois par an
Check-list SU CVP CCI CVC CSC
Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique
Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau
Reacutesultats des surveillances preacutesenteacutes en CME
Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave
Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS
PCC PCAG sonde urinaire CVPCVCCCICSC
PRINOI avril 2012 Page 22
Indicateur ICABMR
PRINOI avril 2012 Page 23
Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008
PRINOI avril 2012 Page 24
Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008
PRINOI avril 2012 Page 25
Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline
au sein de lrsquoespegravece SA
PRINOI avril 2012 Page 26
Source httpwwwrivmnlearssdatabase
PRINOI avril 2012 Page 27
Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM
ne le sont pas sur les EBLSE
Que nous disent les indicateurs de surveillance
PRINOI avril 2012 Page 28
DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)
02018018
012007003
016
019024
026033
036039
05048043
0
01
02
03
04
05
06
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
DI EBLSE 1000 JH DI SARM 1000 JH
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 20
Nouveau rocircle des instances dans la politique de lutte contre les IN
Organisation du travail entre lrsquoEOH et le coordonnateur de la gestion des risques associeacutees aux soins
ICALIN 2
ICALIN ICALIN 2 Organisation 33 20 Moyens 33 30 Actions 34 50
PRINOI avril 2012 Page 21
Nouveauteacutes ICALIN 2 Protection du personnel
proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)
Suivi de la consommation des PHA et restitution au moins 1 fois par an
Check-list SU CVP CCI CVC CSC
Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique
Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau
Reacutesultats des surveillances preacutesenteacutes en CME
Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave
Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS
PCC PCAG sonde urinaire CVPCVCCCICSC
PRINOI avril 2012 Page 22
Indicateur ICABMR
PRINOI avril 2012 Page 23
Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008
PRINOI avril 2012 Page 24
Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008
PRINOI avril 2012 Page 25
Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline
au sein de lrsquoespegravece SA
PRINOI avril 2012 Page 26
Source httpwwwrivmnlearssdatabase
PRINOI avril 2012 Page 27
Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM
ne le sont pas sur les EBLSE
Que nous disent les indicateurs de surveillance
PRINOI avril 2012 Page 28
DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)
02018018
012007003
016
019024
026033
036039
05048043
0
01
02
03
04
05
06
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
DI EBLSE 1000 JH DI SARM 1000 JH
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 21
Nouveauteacutes ICALIN 2 Protection du personnel
proceacutedure de veacuterification de lrsquoimmunisation VHB surveillance de la couverture vaccinale (grippe rougeole varicellecoqueluche)
Suivi de la consommation des PHA et restitution au moins 1 fois par an
Check-list SU CVP CCI CVC CSC
Pose condition drsquoasepsie reacuteeacutevaluation inteacuterecirct maintien surveillance clinique
Au moins 1 surveillance des IN dans le cadre drsquoun reacuteseau
Reacutesultats des surveillances preacutesenteacutes en CME
Deacutemarche drsquoanalyse des causes en cas drsquoeacuteveacutenement infectieux grave
Objectifs plus exigeants Protocoles actualiseacutes + EPP dans les 5 derniegravere anneacutees AES hygiegravene des mains PS
PCC PCAG sonde urinaire CVPCVCCCICSC
PRINOI avril 2012 Page 22
Indicateur ICABMR
PRINOI avril 2012 Page 23
Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008
PRINOI avril 2012 Page 24
Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008
PRINOI avril 2012 Page 25
Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline
au sein de lrsquoespegravece SA
PRINOI avril 2012 Page 26
Source httpwwwrivmnlearssdatabase
PRINOI avril 2012 Page 27
Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM
ne le sont pas sur les EBLSE
Que nous disent les indicateurs de surveillance
PRINOI avril 2012 Page 28
DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)
02018018
012007003
016
019024
026033
036039
05048043
0
01
02
03
04
05
06
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
DI EBLSE 1000 JH DI SARM 1000 JH
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 22
Indicateur ICABMR
PRINOI avril 2012 Page 23
Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008
PRINOI avril 2012 Page 24
Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008
PRINOI avril 2012 Page 25
Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline
au sein de lrsquoespegravece SA
PRINOI avril 2012 Page 26
Source httpwwwrivmnlearssdatabase
PRINOI avril 2012 Page 27
Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM
ne le sont pas sur les EBLSE
Que nous disent les indicateurs de surveillance
PRINOI avril 2012 Page 28
DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)
02018018
012007003
016
019024
026033
036039
05048043
0
01
02
03
04
05
06
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
DI EBLSE 1000 JH DI SARM 1000 JH
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 23
Indicateur SARM Taux drsquoincidence pour 1000 JH en 2008
PRINOI avril 2012 Page 24
Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008
PRINOI avril 2012 Page 25
Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline
au sein de lrsquoespegravece SA
PRINOI avril 2012 Page 26
Source httpwwwrivmnlearssdatabase
PRINOI avril 2012 Page 27
Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM
ne le sont pas sur les EBLSE
Que nous disent les indicateurs de surveillance
PRINOI avril 2012 Page 28
DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)
02018018
012007003
016
019024
026033
036039
05048043
0
01
02
03
04
05
06
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
DI EBLSE 1000 JH DI SARM 1000 JH
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 24
Indicateur SARM (2) Taux drsquoincidence pour 1000 JH de 2002 agrave 2008
PRINOI avril 2012 Page 25
Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline
au sein de lrsquoespegravece SA
PRINOI avril 2012 Page 26
Source httpwwwrivmnlearssdatabase
PRINOI avril 2012 Page 27
Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM
ne le sont pas sur les EBLSE
Que nous disent les indicateurs de surveillance
PRINOI avril 2012 Page 28
DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)
02018018
012007003
016
019024
026033
036039
05048043
0
01
02
03
04
05
06
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
DI EBLSE 1000 JH DI SARM 1000 JH
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 25
Indicateur SARM (3) Preacutevalence de la reacutesistance agrave la meacuteticilline
au sein de lrsquoespegravece SA
PRINOI avril 2012 Page 26
Source httpwwwrivmnlearssdatabase
PRINOI avril 2012 Page 27
Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM
ne le sont pas sur les EBLSE
Que nous disent les indicateurs de surveillance
PRINOI avril 2012 Page 28
DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)
02018018
012007003
016
019024
026033
036039
05048043
0
01
02
03
04
05
06
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
DI EBLSE 1000 JH DI SARM 1000 JH
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 26
Source httpwwwrivmnlearssdatabase
PRINOI avril 2012 Page 27
Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM
ne le sont pas sur les EBLSE
Que nous disent les indicateurs de surveillance
PRINOI avril 2012 Page 28
DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)
02018018
012007003
016
019024
026033
036039
05048043
0
01
02
03
04
05
06
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
DI EBLSE 1000 JH DI SARM 1000 JH
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 27
Pourquoi les mesures de preacuteventions qui semblent ecirctre efficaces sur le SARM
ne le sont pas sur les EBLSE
Que nous disent les indicateurs de surveillance
PRINOI avril 2012 Page 28
DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)
02018018
012007003
016
019024
026033
036039
05048043
0
01
02
03
04
05
06
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
DI EBLSE 1000 JH DI SARM 1000 JH
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 28
DI SARMEBLSE pour 1000 JH (Source CHU de Nantes)
02018018
012007003
016
019024
026033
036039
05048043
0
01
02
03
04
05
06
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
DI EBLSE 1000 JH DI SARM 1000 JH
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 29
Les BMR eacutemergentes que lrsquoon appelle les BH (EPC ERG)
Lepelletier et al Bull Acad Natl Med 2009 Lepelletier et al J Travel Med 2011 Vaux S Euro Surveill 2011
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 30
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 31
au moins 100 actes
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 32
Deacutemarche drsquoameacutelioration de la preacutevention des ISO en chirurgie cardiaque depuis 2001 (Source CHU Nantes)
Surveillance continue des infections du site opeacuteratoire (ISO)
Innovation introduction des DM impreacutegneacutes
2001
Lepelletier et al Infect Control Hosp Epidemiol 2005
Lepelletier et al Antibiotiques 2005
Lepelletier et al Arch Cardiovasc Dis 2009
Standardisation antibioprophylaxie et controcircle de la glyceacutemie peacuteri-opeacuteratoire
Audit drsquoobservation des pratiques
de preacuteparation
cutaneacutee
+
Enquecircte reacutetrospective
Taux drsquoISO
Mise en œuvre drsquoune surveillance prospective continue des
ISO
2002
Reacutedaction drsquoun mode
opeacuteratoire sur la preacuteparation
cutaneacutee du site opeacuteratoire
Application du mode
opeacuteratoire par lrsquoensemble de
lrsquoeacutequipe chirurgicale
Introduction de fils de suture impreacutegneacutes
drsquoantiseptique
2e eacutevaluation des pratiques
de la preacuteparation
cutaneacutee (audit POP)
2003 2005 2007
Introduction
drsquoune eacuteponge impreacutegneacutee
drsquoantibiotique
2009 2008
Arrecirct utilisation
eacuteponge impreacutegneacutee
2011
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 33
copy Photos DL GH Bichat-CB AP-HP 2009
Agreement among Health Care Professionals inDiagnosing Case Vignette-Based Surgical Site InfectionsDidier Lepellet ier12 Philippe Ravaud345 Gabriel Baron345 Jean-Christophe Lucet 12curren
1 Infection Control Unit Bichat-Claude Bernard Hospital Assistance Publique-Hopitaux de Paris Paris France 2 University Paris Diderot Sorbonne Paris Cite Paris France3 Hotel Dieu Centre drsquoEpidemiologie Clinique Assistance Publique des Hopitaux de Paris Paris France 4 INSERM U738 Paris France 5 University Paris DescartesSorbonne Paris Cite Paris France
Abst ract
Objective To assess agreement in diagnosing surgical site infection (SSI) among healthcare professionals involved in SSIsurveillance
Methods Case-vignette study done in 2009 in 140 healthcare professionals from seven specialties (20 in each specialtyAnesthesiologists Surgeons Public health specialists Infection control physicians Infection control nurses Infectiousdiseases specialists Microbiologists) in 29 University and 36 non-University hospitals in France We developed 40 case-vignettes based on cardiac and gastrointestinal surgery patients with suspected SSI Each participant scored six randomlyassigned case-vignettes before and after reading the SSI definition on an online secure relational database The intraclasscorrelation coefficient (ICC) was used to assess agreement regarding SSI diagnosis on a seven-point Likert scale and thekappa coefficient to assess agreement for superficial or deep SSI on a three-point scale
Results Based on a consensus SSI was present in 21 of 40 vignettes (525) Intraspecialty agreement for SSI diagnosisranged across specialties from 015 (95 confidence interval 000ndash059) (anesthesiologists and infection control nurses) to073 (032ndash090) (infectious diseases specialists) Reading the SSI definition improved agreement in the specialties with poorinitial agreement Intraspecialty agreement for superficial or deep SSI ranged from 010 (2 019ndash038) to 054 (025ndash083)(surgeons) and increased after reading the SSI definition only among the infection control nurses from 010 (2 019ndash038) to041 (2 009ndash072) Interspecialty agreement for SSI diagnosis was 036 (022ndash054) and increased to 047 (031ndash064) afterreading the SSI definition
Conclusion Among healthcare professionals evaluating case-vignettes for possible surgical site infection there was largedisagreement in diagnosis that varied both between and within specialties
Citat ion Lepelletier D Ravaud P Baron G Lucet J-C (2012) Agreement among Health Care Professionals in Diagnosing Case Vignette-Based Surgical SiteInfections PLoS ONE 7(4) e35131 doi101371journalpone0035131
Editor Daniel Morgan University of Maryland United States of America
Received November 10 2011 Accept ed March 13 2012 Published April 17 2012
Copyright szlig 2012 Lepelletier et al This is an open-access article distributed under the terms of the Creative Commons Attribution License which permitsunrestricted use distribution and reproduction in any medium provided the original author and source are credited
Funding This study wassupported by the French Ministry of Health (national grant PREQHOS0901) The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript
Compet ing Interests The authors have declared that no competing interests exist
E-mail jean-christophelucetbchaphpfr
curren Current address Bacteriology and Hygiene Department Nantes University Hospital and University of Nantes - EA 3826 UFR Medicine Nantes France
Int roduct ion
Surgical site infection (SSI) is receiving considerable interestfrom healthcare authorities the media and the public Becausethey are often considered avoidable the SSI rate has been usedfor performance assessments and benchmarking [1] andseveral countries require that healthcare facilities publish SSIrates to improve transparency and possibly quality of care andpatient safety [2] However the evidence that publishingquality indicators improves care is scant [3] Recent reportsindicate a need for improved measurement reliability [4] andmandatory public reporting remains a focus of vigorous debate[56]
Methodological issues related to benchmarking and publicreporting remain controversial If the SSI rate is to serve as aperformance indicator then valid and consistent SSI ratesmust be
obtained [2] SSI rates vary according to co-morbidities to thecontamination classand conditions of the surgical procedure Theneed for adjustment hasbeen demonstrated and most surveillancenetworksuserisk stratification [78] Another factor that influencesSSI rates is the certainty of SSI diagnosis The extent to whichdifferent healthcare professionalswill agree regarding thediagnosisof SSI depends on many factors including training experienceand the use of a common SSI definition A single-centre studyshowed variability in the SSI incidence rate according to the SSIdefinition [9]
We designed a study to assess agreement among healthcareprofessionals within and among different specialties regardingdiagnosis and superficial or deep SSI based on case-vignettesconcerning real patients We also evaluated whether the providingof NHSN criteria change the agreement estimates
PLoS ONE | wwwplosoneorg 1 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 34
and post-discharge surveillance [21] Several authorsevaluated theusefulness of surrogate indicators [2223]
Weare aware of a single study evaluating the impact of differentSSI definitions on SSI rates [9] In this study SSI rates varied bymore than 50 when small changes were made in the SSIdefinition This study has limitations however including the
single-centre design and possible observation bias due to theexpectation that SSI rates would vary according to the SSIdefinition Other studies suggest imperfect agreement acrossphysicians regarding the diagnosis of SSI In one study widedifferences in the diagnosis of SSI were noted between infectioncontrol practitioners and surgeons aswell asacrosssurgeons [24]
Table 2 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale (Int raclass correlat ion coeff icient )
Number of vignettesscored
Scoring without the SSIdefinit ion (95CI)
Number of vignet tesscored
Scoring with the SSIdefinit ion (95CI)
Intraspecialty correlat ion
Anesthesiologist 40 015 (000ndash059) 40 035 (000ndash073)
Surgeon 32 045 (000ndash081) 28 042 (009ndash080)
Public health specialist 40 056 (018ndash080) 40 029 (000ndash066)
Infection control physician 40 030 (000ndash069) 40 001 (000ndash048)
Infection control nurse 40 019 (000ndash059) 40 056 (000ndash080)
Infectious diseases specialist 40 073 (032ndash090) 40 066 (022ndash091)
Microbiologist 40 056 (019ndash081) 40 042 (000ndash071)
Interspecialty correlat ion 238 036 (022ndash054) 238 047 (031ndash064)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t002
Table 3 Assessment of surgical site infection (SSI) diagnosis for 40 vignettes (20 cardiac surgery cases and 20 gastrointestinalsurgery cases) developed based on real patients in three French university hospitals
SSI diagnosis score 7-point Likert scale categorized in 2 classes (1234 vs 567)
Scoring without the SSI definit ion (95CI)
Number of vignet tes scored Observed agreement () (95CI) Kappa coeff icient (95CI)
Intraspecialty
Anesthesiologist 40 650 (408ndash846) 015 (2 028ndash057)
Surgeon 32 688 (413ndash890) 038 (2 005ndash080)
Public health specialist 40 750 (509ndash913) 052 (020ndash084)
Infection control physician 40 650 (408ndash846) 021 (2 024ndash064)
Infection control nurse 40 550 (315ndash769) 012 (2 030ndash053)
Infectious diseases specialist 40 850 (621ndash968) 066 (030ndash100)
Microbiologist 40 800 (563ndash943) 060 (026ndash094)
Interspecialty 238 - 028 (021ndash036)
Scoring with the SSI definit ion (95CI)
Intraspecialty
Anesthesiologist 40 750 (509ndash913) 043 (001ndash085)
Surgeon 28 714 (419ndash916) 028 (2 005ndash063)
Public health specialist 40 650 (408ndash846) 030 (2 006ndash066)
Infection control physician 40 550 (315ndash769) 2 003 (2 045041)
Infection control nurse 40 650 (408ndash846) 040 (2 001ndash080)
Infectious diseases specialist 40 850 (621ndash968) 062 (025ndash100)
Microbiologist 40 700 (457ndash881) 041 (002ndash080)
Interspecialty 238 - 041 (034ndash048)
Number of vignettes scored (for intraspecialty 20 vignettes were scored twice and for interspecialty 34 vignettes were scored 7 times)missing values due a computer assignment glitchdoi101371journalpone0035131t003
Agreement in Diagnosing SSI
PLoS ONE | wwwplosoneorg 4 April 2012 | Volume 7 | Issue 4 | e35131
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 35
Discussion (1)
Les indicateurs
Neacutecessaires agrave lrsquoameacutelioration des pratiques Impact positif du 1er tableau de bord des IN
Importance des critegraveres drsquoeacutevaluation
Pertinence faisabiliteacute qualiteacutes meacutetrologiques Inteacuterecirct pour les professionnels de santeacute utilisateurs
Les mesure-t-on tous de la mecircme maniegravere (PNP ISO)
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 36
Discussion (2)
Les indicateurs
Un indicateur au maximum de son score nrsquoa plus drsquoeffet laquo moteur raquo Indicateurs de 2e geacuteneacuteration en attendant ICATB2 et les
indicateurs de reacutesultats Changement des classes de performance
Importance de deacutevelopper des indicateurs au sein des ES
par contractualisation avec les pocircles en dehors des indicateurs agrave diffusion publique
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients
PRINOI avril 2012 Page 37
Quel regard des usagers sur les indicateurs (en routine en cas de crise)
PRINOI avril 2012 Page 38
Choisir des indicateurs utiles aux patients