View
713
Download
0
Category
Preview:
DESCRIPTION
Recent Findings on Using Rich Media in Online SurveysBernie Malinoff element54Jeffrey Henning, PRCVoviciThis presentation will explore some hotly debated questions about data consistency, and what respondents are actually doing with our online surveys — spanning the most widely used 'radio buttons', to the more recently introduced 'rich media' types.Researchers concerned about declining online survey response rates have rushed to embrace "rich media" (aka Web 2.0) as a way to enliven the survey taking process and reignite respondent engagement. Yet engagement is not the same as entertainment. When can rich media foster engagement, reliability and validity and when can rich media damage it? Bernie Malinoff, CEO of element54, and Jeffrey Henning, founder of Vovici, have both independently conducted research-on-research studies comparing "traditional" online questionnaires to "rich media" online survey formats. Both have consistently seen dramatic variations in results.
Citation preview
RecentFindingsonUsingRichMediainOnlineSurveys
BernieMalinoffJeffreyHenning,PRC
PanningforRespondents
EngagedRespondentsareGolden
LosingRespondentsAlongtheWay
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Engaged
Completed
ParRcipated
Invited
Source: Vovici
ButisaGhostTownourFuture?
PaperSurveysInspiredtheWeb
MaybeTechnologyistheAnswer?
WebSurveysasEntertainment
WhatCanWeTaketotheBank?
ELEMENT54R&D
DriversofSurveyCooperaRon
RespondentCooperaRonSurveyInvite
QuesRonnaireLength/IncenRve
TopicInterest
Panel/ClientReputaRon
VisualAppeal
Typicalrespondentexperience
ResponsePa^erns/Flatlining
"Engagement&
Enjoyment"
OnscreenLayout
RespondentExpectaRons
Divergence/VarietyofNewQuesRonTypes
Sources: Spyglass Solutions/2x4 Research & Consulting
‘BlackBox’PhaseofOnlineDesign
FinalQuesRonnaire
ResearcherGivesto
Programming
ProgrammingDeliversa“Link”
Researcher/ClientReview
ProjectKick‐offBriefingPlanningSampleDraj(s)...
InterviewingCoding
TabulaRonAnalysisReport
‘RespondentEngagement’ConvergingIssues
DataConsistency
RespondentUsability
MajorR&DIniRaRves DataConsistency RespondentUsability
Balanced5CellDesign:2000TotalInterviews
RadioBu^on
Anchor@‘1’
Anchor@‘5’
Anchor@’10’
Experimental
QuesRonnaireCoverage
StudyHighlights
+8%overstatedpast7dayconsumpRon‐slidervs.radiobu^on
‐8%understatedissueimportance‐slidervs.radiobu^on
36%variancehowojenbrushteeth‐slidervs.radiobu^on
eg‐NetPromoterScore(NPS)
GivenDifferences,WhatistheBenchmark?
RadioBu^on
Default,EasierStandards
Sliders
Scales/AnchorsMa^er
DragnDrop
CauRon:ContentsHot
SurveyCompleRonTime&RespondentEnjoyment
RichMediavs.RadioBu^onQuesRons
EyesDon’tLie‐ResearchProtocol
EyeTracking
Recording
Interviewing
EyeTracking
"1.0vs.2.0"UsingEyeTrackingforSurveyUsability
... to better understand the destination, by examining the journey.
EyeTracking‐'AreasofInterest'
Inferencevs.InstrucRon
Inferencevs.InstrucRon42%didnotreadthequesRon
33%didnotreadthe"tradiRonal"
quesRon
Don’tInterruptMeWhenI’mThinking
ErrorPrevenRon&Recovery
Thisrespondentneeded5minutestocompletethisquesRonandencountered3errormessages.
ThequesRonis‐howdowetreatthedatawhichemergesfromsomeonewhotriedhisng"next",andthenaddedorchangeddatatoescape?
ErrorPrevenRon&Recovery
RespondentinterpretedtheerrormessagelocaRonasaguidetowheretoputtheanswer.RecommendaRon‐havesomewaytohighlighttheenRrerow,avoidinganyvisualcuesorbiases.
MinimumStandards‐eg:HorizontalLayout
Horizontallayoutensuresconsistentviewing,regardlessofscreensize
EqualRaRngsfor‘Engagement’Metrics
1.08.20
8.86
7.02
2.08.25
8.85
6.56
SurveyEnjoyment
EaseofUse
DesiretoRepeat
SexyQuesRons,DangerousResults?ConsideraRons
1. ThereisnoBe^er,Faster,Cheaper‐youcanonlypick2.2. Wemustreturntotheusabilityprincipleswhichguided
papersurveydesign,notCATIdesign,tomaximizeconfidenceinourdata,andbusinessdecisions.
3. TherearewaystomakeallquesRons/surveysmoreappealing,intuiRveandfuture‐relevant.
4. 'ResponsibleInnovaRon'iskey.5. Asanindustry,wemustdeviseandreinforcebest
pracRcesforonlineresearchinterfaces,likewehaveforsomanyelementsoftheresearchprocess.
TipsforMigraRngtoRichMedia• DiscreRonisthebe^erpartofvalor• SplitballottestrichmediaquesRonsagainsttradiRonalquesRonstoidenRfydifferences
• Visualchoicesintroducebias,especiallyformoreabstractchoices
TechnologyisNottheOnlyAnswer
TipsforEngagement• Keepthesurveyshort• MakethequesRonsinteresRng
• Minimizetheuseofmatrixes
• Userichmediainaconsideredmanner
CreaRngEngagingSurveysisHard
WeWelcomeYourQuesRonsBernieMalinoffPresident
element54
h^p://twi^er.com/BernieMalinoff
Bernie.malinoff@element‐54.com
(514)833‐2268
VoviciBlog
Vovici ’s Jeffrey Henning blogs about survey research, EFM, voice of the customer and online feedback.
blog.vovici.com
JeffreyHenningFounder&VP,Strategy
Vovici
h^p://twi^er.com/jhenning
Jhenning@vovici.com
(800)787‐8755
NPR Listens: How Panel Management helps NPR ‘Dial In’ on Unmet Needs
Jeanne Bliss:
EarnyourCustomersRave
Lessons from a loyalty Leader: A conversa>on with Navy Federal Credit Union
ContactInformaRonBernieMalinoff
• bernie.malinoff@element‐54.com
• 514‐833‐2268• @berniemalinoff
• h^p://www.element‐54.com
JeffreyHenning,PRC
• jhenning@vovici.com
• 781‐848‐8100x550• @jhenning
• h^p://www.vovici.com
Recommended