Embed Size (px)
DESCRIPTION
Slides from a talk that I gave for the Boston CHI meeting on April 9, 2013. Moderating usability studies and interviewing research participants is more than just reading questions off of a page - it's about having a conversation. In this talk I discuss why it's important to have a conversation with participants and how to do so in a way that encourages them to talk honestly. I also discuss how to maintain neutrality and minimize introducing bias into the conversation.
Citation preview
APRIL 9, 2013
Susan Mercer Senior Experience Researcher [email protected] @susanamercer
Boston CHI Monthly Mee>ng
STRENGTHENING DESIGN RESEARCH MODERATION SKILLS
Strengthening Moderating and Interviewing Skills
§ Once I learned the basics
§ And ran a couple of studies,
§ Now what?
§ How do I strengthen my skills?
2
Story Time
Don’t play the notes on the page…
…play the music
This applies to moderating and
interviewing too.
Don’t just read the questions from the page…
…have a conversation
3
The Art of Moderating and Interviewing
§ Not just reading questions
§ It’s also…
§ Building rapport with the participant
§ Creating a smooth conversation
§ Encouraging the participant to talk
§ Remaining objective
§ Listening effectively
§ Being human
4
What Is Our Goal?
§ To elicit honest thoughts from participants in an unbiased manner.
§ “Would you like to try one of my chocolate chip oyster and herring brownies? They’re really good!”
§ Sometimes it’s hard to be fully honest
5
The Rules of Politeness
Linguistic anthropologists have studied this phenomenon
The Rules of Politeness 1
1. Don’t Impose
2. Give Options
3. Be Friendly
6 Sources: 1 Lakoff, 1976; 2 Tannen, 2005
“Human beings are always balancing the paradoxical fact that they are simultaneously individuals and social creatures.”
-‐ Deborah Tannen
How Do We Get Beyond Politeness?
§ In other words, we’re often polite rather than honest to strangers.
§ As human beings we have a need to be socially accepted
Social Acceptance > Honesty
§ We guess what is expected and answer to avoid offending
§ It’s second nature – we’re often not aware we’re doing it
7
How Do We Get Beyond Politeness?
Build their trust
8
Politeness Trust
Strangers Good Friend
Familiarity
Building Trust in an Interview
1. Be trustworthy upfront
2. Build rapport
3. Engage in comfortable conversation
4. Stay neutral and accepting
9
1. Be Trustworthy Upfront
§ Confidentiality
§ How are you recording what is said?
§ Who is listening?
§ What will you do with the information you collect?
§ Neutrality
§ You’re not emotionally involved in the design/project
§ There are no right or wrong answers
§ Your job is to get honest opinions
§ Consent Form
§ Put it all in writing – using their language
2. Build Rapport
§ Greeting a Participant
§ Smile
§ Use their name
§ Handshake / Warm phone greeting
§ Make them feel important - sincerely
§ Small Talk – Find Common Ground
§ Safe topics: travel to office, traffic, weather
§ Avoid asking direct questions
§ Listen and look for shared experiences
A person's name is to that person the sweetest and most important sound in any language.
-‐ Dale Carnegie
Source: Carnegie, 1936.
2. Build Rapport
§ Be Empathetic
§ Apologize if they had trouble finding the office
§ Show you understand their point of view
§ “Oh, it’s raining there? It is here too. I hate rainy days.”
§ Inject Some Humor
§ Joke about yourself
§ Joke about the situation
§ Don’t joke about them
§ Short version – Show them that you are human too
3. Engage in Comfortable Conversation
Two schools of usability testing moderating techniques
§ Moderator keeps silent and says “Keep talking” 1
§ Awkward feeling for participant
§ Reminds them that they are in a “study”
§ Moderator creates conversation where participant is primary speaker 2
§ Still accomplishes goals of gathering information
§ Meets the participant’s expectations of a socially acceptable conversation
§ Moderator can still be neutral and minimize bias
13 Sources: 1 Ericsson and Simon, 1980, 2 Boren and Raney, 2000.
3. Engage in Comfortable Conversation
What is a Comfortable Conversation?
§ Conversational cues and turn-taking are expected
§ Acknowledgement tokens – “Uh huh”, etc.
§ Encourage the continuation of the other speaker’s talk
§ Usually implies that the other speaker’s prior talk is incomplete
§ Some indicate change of speakership
§ “Yeah” – more than half the time indicates changing turns in speaking
§ “Yeah. I heard that the other day and…”
§ “Oh!” - may indicate noticing something, then transitioning to another topic
§ “Oh! That reminded me…”
3. Engage in Comfortable Conversation
§ Some may introduce bias
§ “Oh!”, “Interesting” – indicating unexpected answer
§ “Yes”, “Perfect”, “Great” – indicating agreement
§ “Hmmm.”, “Really?” – indicating disagreement
§ Notice that tone is key
§ Neutral is best
§ “Mhmm”, “Uh huh”, “Continue”, “Tell me more”, “OK”
§ “Mhmm” or “Uh huh” vs. silence à interviewees saying 31% more phrases. 1
Source: 1 Matarazzo et. al., 1964
4. Stay Neutral and Accepting
§ Ask Open-ended Questions
§ Start with Who, What, When, Where, Why, How?
§ Really Listen
§ Pay attention – stay in the moment
§ Look at the participant
§ Take notes if you can
§ Be quiet - give them time to say what they need to
4. Stay Neutral and Accepting
§ Watch Your Reactions
§ Don’t show surprise
§ May make them think that they are giving a wrong answer
§ Don’t overly agree
§ May make them think that they are giving the right answer
§ Don’t be negative
§ Watch your tone – stay neutral and accepting
§ Try not to laugh
4. Stay Neutral and Accepting
§ Be yourself
§ No one is perfectly neutral
§ Recover gracefully and move on
§ “Perfect” – “That’s the level of detailed feedback we’re looking for.”
§ “Interesting!” – “I haven’t heard that perspective yet, tell me more.”
§ (something surprising) – “I can understand that.”
§ Interject some Rapport-building comments when needed
§ Quiet or uncomfortable participants
§ “I hate it when that happens.”, “I can imagine that was challenging”, etc.
§ Again, showing that you are human like them
5. Encourage them to talk
§ Be Quiet!
§ Most agreements happen immediately. Most people delay before disagreeing.1
§ If you don’t respond to their answer, it encourages them to talk more
§ People often delay speaking before disagreeing – give them time
§ Some people are uncomfortable with silence, so they will keep talking
§ The best way to stay neutral J
Source: 1 Goodwin and Heritage, 1990.
Building Trust in an Interview
1. Be trustworthy upfront
2. Build rapport
3. Engage in comfortable conversation
4. Stay neutral and accepting
5. Be Quiet
20
KEEPING YOUR SKILLS SHARP
21
1. Identify your improvement areas
§ Watch your videos
§ Have colleagues give you feedback
22
2. Learn From Others
§ Watch others moderate
§ Be a participant
§ Listen to talk radio interviews
23
3. Practice These Skills in Everyday Life
§ Be quiet and listen in everyday conversations
§ Ask open-ended questions more frequently
24
The most important thing is…
25
PRACTICE!
References
Boren, T. and Ramey, J. (2000) Thinking aloud: reconciling theory and practice. IEEE Transactions on Professional Communication, 43 (3), 261-278.
Carnegie, D. (1936) How to win friends and influence people. Simon & Shuster.
Dumas, J. and Loring, B. (2008) Moderating usability tests, Morgan Kaufman.
Dumas, J. and Redish, J. (1999) A practical guide to usability testing, Intellect Ltd.
Ericsson, K. and Simon, H. (1980) Verbal reports as data. Psychological review. 87 (3), 215-251.
Goodwin, C., & Heritage, J. (1990). “Conversation analysis.” Annual review of anthropology 19 (1990): 283-307.
Matarazzo, J.D.., Wiens, A. N., Saslow, G., Allen, B. V., & Weitman, M. (1064). Interviewer Mm-Hmm and interviewee speech durations. Psychotherapy: Theory, Research & Practice, 1(3), 109.
Tannen, D. (2005). Conversational Style: Analyzing Talk among Friends. Oxford University Press, USA.
26
Coming Soon!
New book on Moderating
• Local Authors
• Coming this Fall
• Follow @ModSurvivalUX
27
28
Questions?
