Embed Size (px)
Citation preview
Sonja Marziano Smart Chicago Collaborative
April 25, 2015
www.smartchicagocollaborative.org
CIVIC USER TESTING GROUP (CUTGROUP)
SMART CHICAGO COLLABORATIVE
A civic organization devoted to improving lives in Chicago through technology. ¡ We work on increasing access to the Internet,
improving skills for using the Internet, and develop meaningful products from data.
¡ CUTGroup is a central program because it cuts across these three areas
PREMISE OF CUTGROUP
The CUTGroup is a set of regular Chicago residents who get paid to test civic apps ¡ Over 800 Chicago residents signed up, from
all over the city – all 50 wards, all 77 community areas
¡ $5 gift card sign-up ¡ $20 gift card per test
15 tests to date that cover topics such as health, education, social services, transportation, permits, and more ¡ All different styles: focus groups, one-on-one
proctored tests, remote ¡ We publish all of our methods, questions and
results online § We also wrote a book! § www.cutgroupbook.org
CUTGROUP TESTS
THREE MAIN COMPONENTS
¡ UX Testing ¡ Digital Skills ¡ Community Engagement
UX TESTING
¡ Developers who work with us get specific, actionable feedback
¡ Both quantitative and qualitative information
¡ Stray from UX test design principles in a key way: by requiring the developer to participate in the test
DIGITAL SKILLS
¡ Each test consists of a mix of 10 – 20 people with varying digital skills
¡ Introduce to new websites and technology
¡ Developers learn how to conduct UX tests to improve their sites
COMMUNITY ENGAGEMENT
¡ Diverse groups of testers from different neighborhoods
¡ Conversations about technology between residents and developers
Developer Meeting ¡ Who are the current
users? ¡ Who do we want to
target for this CUTGroup test? § What type of screening
questions should we ask? ¡ What type of test? ¡ What do we want to
ask? ¡ What does the
developer want to get out of this CUTGroup test?
Example: mRelief ¡ Current users: Southside ¡ Target: Enrolled in social
services benefits, eligible for benefits, different devices
¡ Type: One-on-one proctoring
¡ Questions: Language, ease of use, text feature, pop-overs
¡ Wanted to know: language accessibility, user experience, importance of additional info about docs or process
PRE-TEST
¡ Find location: public computer centers and libraries
¡ Mailchimp email campaigns to § Invite testers § Learn more through screening questions § Check availability
PRE-TEST
¡ We use Wufoo forms to capture responses
¡ Also, used as a script for test questions
¡ Tools: § Screencast-o-matic § Video cameras § Laptops & iPads § Other devices § Gift cards § Candy
DURING TEST
Types of Questions
¡ Current behaviors & experiences
¡ Devices ¡ Homepage review ¡ First item clicked on ¡ Likert scale ¡ Guided/task-related
questions ¡ Quantitative and
qualitative
Example: CPS.edu ¡ Do you search online
about the schools your children attend?
¡ Review the homepage. What are 3 things you notice?
¡ Search for your child’s school. How did you search? What did you learn?
¡ In what ways would you improve the CPS website?
DURING TEST
¡ Analysis § In-depth analysis of each question § Publish raw test data
¡ Publish videos and photos from test session ¡ Blog post with overview ¡ Follow-up with Github issues
AFTER THE TEST
Example: Foodborne Chicago ¡ How do users feel about
responding to a random tweet?
¡ Does the form require too much info?
¡ Would users feel better with more CDPH or 311 visibility?
¡ Why might residents on the South side not respond as often as residents from other neighborhoods?
Answers ¡ Tweets should have 3
elements: compassion, question, and an official element
¡ Form was not too long ¡ Wanted more visibility
of connection with the City of Chicago
¡ Twitter is used as a private communications network
AFTER THE TEST
UPCOMING CUTGROUP CHANGES
¡ Outreach to Cook County ¡ Text messaging system to recruit and notify new
testers
¡ Proctoring program ¡ More post-test work, assisting with implementing
changes
¡ Extensive how to run CUTGroup
¡ Oakland & Chattanooga have started CUTGroup in their cities
www.cutgroupbook.org
CUTGROUP BOOK
¡ Sign-up here: www.cutgroup.org
¡ Are you a developer? Sign-up here: www.cutgroup.org/developers.html
BE A TESTER!
Sonja Marziano [email protected] @ssmarziano
Smart Chicago @SmartChicago www.smartchicagocollaborative.org www.cutgroup.org
THANK YOU!!!!!!
