virtual Touch keypad 12-key Chinese text input benchmark test report

Confidential

UIDesign

Rev PA1 2009-3-10 1

Soft 12-key Chinese text inputBenchmark Test

By John Chen, UX Lab Beijing

Confidential

Rev PA1 2009-3-10 2

Purpose

• To evaluate the existing soft 12-key keypad text input methods in touch handsets.• A4• TouchPal• Dayhand• Nokia 5800

• To advise on the design of Sony Ericsson’s soft 12-key Chinese text input in touch phone.

A4 Dayhand

TouchPal v3.5 Nokia S60 Touch (N5800)

Confidential

Rev PA1 2009-3-10 3

Methodology

• Task1 (Chinese only): 先到附件的嘉禾一品或者肯德基吃点东西

• Task2 (Mixed Chinese & Letters): 我在单位呢现在上不了 MSN 和 Gtalk

• Task3 (mixed Chinese & English): 晚上去哪儿 happy ？

• Task4 (mixed all): 我的邮箱是 [email protected]

12 participants (5males, age 20-30) were paid to test the 4 different 12key touch text input methods. They were asked to enter the following 4 sentences by 4 different input methods. The 4 inputs presented randomly.

These users used Pinyin in their keypad phones, and some of them had some experience with touch.

The users were asked to explore the input by themselves for about 10 min.

Tasks

Introduction

Confidential

Rev PA1 2009-3-10 4

Executive Summary

• A4 was the easiest to get started and was the most efficient and satisfying input method amongst the 4 tested soft 12key Chinese text input method.

• Nokia was the 2nd easiest one, but was only able to enter the dictionary phrases, which slowed down the texting speed a little bit.

• Touchpal (v3.5) was the most difficult one to get started due to the complex and hidden gesture operation.

• Dayhand was the 2nd difficult one due to the special sliding operation to select the candidate word, but it was easier to learn compared to TouchPal.

• All the 4 text input methods had significant difficulties in precise uppercase and lowercase text entry.

Confidential

Rev PA1 2009-3-10 5

Detailed Result – Task 1 Sumamry

• A4 was the fastest and easiest one on Chinese text entry

• Nokia was easy too for this sentence (only Chinese), but the entry speed was slower than A4 due to lack of the capability of non-dictionary phrase entry.

• Dayhand was difficult to get started due to the sliding operation, need significant longer time to get adapted.

• TouchPal: There was a big deviation on the task time. Some users were able to enter text easily, but others made a lot of mistakes on text entry due to that there were 2 backspaces on the panel.

先到附件的嘉禾一品或者肯德基吃点东西Task1

Confidential

Rev PA1 2009-3-10 6

Detailed Result – Task 1: A4

• All the test users (except one) could enter Chinese on A4 quickly and accurately.

• A4 was capable to enter more words continuously (sentence based)

• Some users entered the text by typing 5 words together, e.g. typing “wozaidanweine” for “ 我在单位呢”

• Feel safe to enter non-dictionary phrases• “It’s under my control even I made mistakes”

• “I know where to find what I need”

• Full Pinyin candidates were listed on the left side were very helpful when enter non-dictionary phrase.

先到附件的嘉禾一品或者肯德基吃点东西

Confidential

Rev PA1 2009-3-10 7

Detailed Result – Task 1: Nokia

• Most users could enter Chinese only text easily in Nokia 5800.

• Some minor issues:• Nokia Candidate pinyin highlight confusion (refer to the

video on the right)

• Nokia has no capability to enter non-dictionary phrase• You have to clear what you’ve entered if it was not in

the dictionary


Confidential

Rev PA1 2009-3-10 8

Detailed Result – Task 1: TouchPal

• Two backspaces (one for delete and the other for clear) confusion on the interface• Always delete all the texts in the editor by mistakes

• Candidate pinyin (only one letter showing on the left list)

• It was ignored by most users• “If the default predictions is not what I want, how

could I do?”


Confidential

Rev PA1 2009-3-10 9

Detailed Result – Task 1: Dayhand

• Dayhand required user to learn how to select candidate by sliding operation, which took a while to get adapted.

• “选字” (select word) not a good phrase for indicating that more candidates were available.

• No hint or instructions for the first time users on how to choose predictions by sliding down or sliding up through the word.


Confidential

Rev PA1 2009-3-10 10

Detailed Result – Task 2: Summary

Task2我在单位呢现在上不了 MSN 和 Gtalk

• All the 4 input methods were problematic in the uppercase and lowercase switch task. ANOVA test showed no significant difference amongst the 4 inputs in terms of task time.

• There were 3 users failed in TouchPal and Dayhand, and 2 users failed in A4, 1 user failed in Nokia.

• The main confusions here were:

• Switch between Chinese and English

• Switch between uppercase and lowercase

• Switch between Predictive English mode and multi-tap letters mode

Confidential

Rev PA1 2009-3-10 11

Detailed Result – Task 2 A4

• For A4, users didn’t need to switch between letters mode and predictive mode;

• 2 users took significant more time to find the uppercase and lowercase switch than others did.

• Number “1” key was used as symbols input and changed to case switch once a letter was entered

• It was always ignored by the first time users that the “1” key has changed to ‘ 大写’ (uppercase) or ‘ 小写’ (lowercase).

• It was dangerous when challenge the users habitual thinking or existing mental mode• There should be a significant indication when this

happens

我在单位呢现在上不了 MSN 和 Gtalk

Confidential

Rev PA1 2009-3-10 12

Detailed Result – Task 2 Nokia

• Nokia’s predictive English mode made a lot of troubles• Difficult to choose candidate by pressing the default

prediction• Toggling candidates by pressing star key was very

hidden

• ON/OFF predictive English mode was problematic

• ‘预想输入’ (translation for predictive input) was problematic)

• Transition from ON to OFF was not smooth

• Switch from Chinese mode to English mode sometimes problematic• Association words obstructed the switch button


Confidential

Rev PA1 2009-3-10 13

Detailed Result – Task 2 Dayhand

• Why 3 users failed?

• Didn’t find how to switch cases• Or didn’t find the single letters mode (toggling button

on the left side from predictive to letters mode)

• 2 hidden ways to enter uppercase letters

• To switch to letters mode first then to enter uppercase letters• Most users used this way to succeed in this task

• Sliding up on space key to switch cases (learned either from TouchPal or moderator’s instructions)


Confidential

Rev PA1 2009-3-10 14

Detailed Result – Task 2 TouchPal

• 3 users failed in this task due to the hidden case switch• either sliding up on space key • or sliding up on each letter

• Auto spaces and initial letter auto capitalized made big troubles in mixed Chinese and English entry


Confidential

Rev PA1 2009-3-10 15

Detailed Result – Task 3 SummaryTask3晚上去哪儿 happy?

• This task required switch between Chinese, English and symbols.

• For Nokia, users entered ‘happy’ either by multi-type mode or by predictive English mode, but the switch between Chinese and English took time because of obstruction by association words.

• For TouchPal & Dayhand, the question mark was difficult to enter.

Confidential

Rev PA1 2009-3-10 16

Detailed Result – Task 3 A4 & Dayhand

• Both A4 and Dayhand were easy to enter this sentence

• Switch on the left list was easy to understand

• Symbols on the number “1” key worked very well

晚上去哪儿 happy?

Confidential

Rev PA1 2009-3-10 17

Detailed Result – Task 3 Nokia

• Obstruction of association words:

• Backspace key was used only to delete the entered words.• Would be better if press backspace to delete

association words first

• Users had to figure out the other ways to switch• Press the space area on the edit screen• Press harsh key• Select an association word and then delete it

• Transition was bad

• When switch on or off the predictive mode• When press star key to enter symbols

晚上去哪儿 happy?

Confidential

Rev PA1 2009-3-10 18

Detailed Result – Task 3 TouchPal晚上去哪儿 happy?

• For TouchPal: Symbols were problematic.• Sliding down on the 12 key keypad

• Sliding left or right to have more symbols• Hidden, no indication

Confidential

Rev PA1 2009-3-10 19

Detailed Result – Task 4 SummaryTask4我的邮箱是 [email protected]

• A4 was fastest due to:• Simple switch between Chinese, English, symbols and

numbers• Pinyin was still predicted in English mode (e.g. xiao, hua),

which was highly appreciated by users

• Some users had difficulties in mode switch and predictive mode (English)

• Touchpal made big confusion in this task due to the hidden English letters mode and annoying auto space.

Confidential

Rev PA1 2009-3-10 20

Detailed Result – Task 4 A4 & Dayhand我的邮箱是 [email protected]

• A4 was fastest due to:

• Simple switch between Chinese, English, symbols and numbers

• In English mode, pinyin was predicted

• Dayhand:• Some users used predictive

mode to enter non-dictionary word (xiaohua) took relative long time

Confidential

Rev PA1 2009-3-10 21

Detailed Result – Task 4 TouchPal

• Touchpal was optimized for English text input, but the auto space was annoying in mixed Chinese & English entry.

• Users didn’t know how to enter non-dictionary phrase, e.g. xiaohua• Hidden letters mode (sliding up on backspace key

to switch from predictive mode to letters mode)

• Candidates list on left side was ignored by most users

• Delete the complete word was good in English mode, but not in this mix entry mode, which always resulted in clear all the entered words by mistakes.

我的邮箱是 [email protected]

Confidential

Rev PA1 2009-3-10 22

Detailed Result – Task 4 Nokia我的邮箱是 [email protected]

• Nokia switch issues again• Obstruction of switch button

• Don’t know how to off the predictive mode in English

• Not clear indicators of ON/OFF status for predictive English mode

Confidential

Rev PA1 2009-3-10 23

Satisfaction Summary

• No doubt A4 got highest rating on satisfaction due to the efficiency and simplicity.

• Nokia was slower than Dayhand in entry speed, but still got higher ratings than Dayhand, because Dayhand required the learning of sliding operation which was perceived lack of precise and controls.

• Some features optimized for English input was not suitable for mixed Chinese & Latin input, e.g. auto space and delete the complete word etc.

• Too many hidden features in TouchPal frustrated most users.

SUS (System Usability Scale) used in the test:

SUS Scores

Confidential

Rev PA1 2009-3-10 24

DESIGN RECOMMENDATIONS

Confidential

Rev PA1 2009-3-10 25

Design Recommendations - Efficiency

• Efficiency is the first principle of text input design

• Don’t waste user’s time• Any effects or controls at the cost of user’s time should be

avoided in text input

• Using Sliding operation cautiously

• Could be an option, but should not be the only way to complete a specific task, e.g. case switch, language toggle etc.

• Sliding trace line (see Dayhand) is a good hint for users (red line helped track sliding)

Confidential

Rev PA1 2009-3-10 26

Design Recommendations - Errors

• Decrease errors• Any risk of increasing errors should be avoided, e.g. ‘clear’ key is

dangerous in Chinese text input

• Sliding made more mistakes than press.

• Small touch size made more mistakes than big one.

• The accidental presses on the outside of soft keypad made error inputs on the editor (often happened on Windows notepad input).

• People made more errors on touch text input compared to that on traditional keypad phone due to the lack of tactile feedback and real touch perception.

Confidential

Rev PA1 2009-3-10 27

Design Recommendations – Keypad layout

• Keypad layout• Keep the layout simple, and make it clear at a glance

• A4 was well received on the simplicity of layout

• Soft keypad size can be customized (like TouchPal did) preferred• Minimal size for thumb usage: 7mmX7mm is acceptable for most

participants of this test

Simple Complex

Confidential

Rev PA1 2009-3-10 28

Design Recommendations – Delete key Actions

• Delete/Backspace key action (it was quite consistent among 3 of them, except Nokia 5800)

• Delete association words first

• Then delete typed pinyin

• Finally delete typed words

• Long press ‘Delete’ key to clear all the entered words

Confidential

Rev PA1 2009-3-10 29

Design Recommendations –Space key Actions

• Space key actions (it was different among the 4 different inputs)

• TouchPal & Dayhand

• Enter space when there’s no pinyin entered

• Confirm the default prediction words (like Touchpal does, enter the first candidate by press space key, which is consistent with PC input)

• Dayhand used number “1” key as separator

• Unfortunately, no users used these extra functions on space key due to• A little bit hidden features• Not willing to take risk to explore more in a timed task

Confidential

Rev PA1 2009-3-10 30

Design Recommendations –Space key Actions

• Space key actions (it was different among the 4 different inputs)

• Nokia 5800 & A4

• Enter space when there’s no pinyin entered

• used as “separator” when entering Jianpin (e.g. enter ‘b, space, and j’ for BeiJing)

• A few users tried “ 分隔符” & “ 声调”

Confidential

Rev PA1 2009-3-10 31

Design Recommendations – Latin input

• The most frustration in the test was the mixed Chinese & Latin input, especially with precise case entering.

• No need to switch between Chinese & English is the smartest way, like TouchPal v4 did.

• No need to switch between predictive mode and multi-tap mode (letters mode) is the most simple and fast way, like A4 did.

• Disable auto space when the phone language is set to Chinese. In Chinese text input, there were much fewer spaces than in English input.• Auto space not needed in mixed Chinese & English text input for most use cases

• Pinyin was predicted in English mode was strongly preferred• E.g. chen, hua etc Pinyin were predicted in English mode

• Uppercase/lowercase switch• Dedicated key for case switch• Switch case first and then start entering is most people’s mental model (Do not

conflict with it)

Confidential

Rev PA1 2009-3-10 32

Design Recommendations – Edit

• Edit between text• It was extremely difficult to place cursor precisely between the texts

need to be edited.

• All the inputs except Nokia 5800 offer the edit panel for moving cursor, copy & paste etc.

• But only a few people used them, most of them just used the most intuitive way: place the thumb between the texts, although it’s inaccurate & always made mistakes.• Maybe iPhone’s Magnifying glass solution is a more intuitive way

Confidential

Rev PA1 2009-3-10 33

Design Recommendations – Visual feedback

• Big size key (> 7mmx7mm)

• Using significant different color to indicate which was pressed is acceptable

Confidential

Rev PA1 2009-3-10 34

Design Recommendations – Visual feedback

• But for small size touch key (e.g. pinyin candidates, switch buttons on the left <7mmx7mm)

• Distinct color indicator needed

• Pop-up indicator needed (like soft keyboard in iPhone)

• It’s better to confirm by which key was released on, not by which key was pressed, which allows people to move the thumb around for precise positioning.

Confidential

Rev PA1 2009-3-10 35

Design Recommendations – Context sensitive key

• Context sensitive key

• Change the label correspondingly according to the context

• Change the color of the label correspondingly is recommended (e.g. number “1” key in A4)

• Sometimes, only label changs didn’t give enough hint to the first time users.

Confidential

Rev PA1 2009-3-10 36

Design Recommendations - Terminology

• Using understandable terminology, avoid confusion

Confidential

Rev PA1 2009-3-10 37

Design Recommendations – Advanced features

• 3 of tested inputs featured with:• Non-dictionary phrase input (user dictionary)

• Capability of creating / deleting own phrases

• Word frequency adjustment

• JianPin / Abbreviated Pinyin (clear communicated)

• Rich Smileys panel

• Edit panel (including copy & paste, select all, delete, navigation etc)

• More unique features provided by TouchPal v4.0 (refers to appendix)

Confidential

Rev PA1 2009-3-10 38

Further research

• Focusing on mobile usage context

• Considering the ergonomics influence• key size• key locations• Feedback:

• Visual feedback• Audio feedback• Tactile feedback

Confidential

Rev PA1 2009-3-10 39

Appendix – improvements of TouchPal v4 compared with v3.5 on usability issues

• Layout

• Using texts instead of icons

• Clearly differentiated “clear” and “backspace”

• Add one row of buttons on the bottom for toggling• Solve the issue that can not switch when there’re association

words on

• Dedicated Case switch key (Shift & Caps)

V3.5

V4.0

Confidential

Rev PA1 2009-3-10 40

Appendix – improvements of TouchPal v4 compared with v3.5 on usability issues

• Complex sliding actions on Space were removed

• Sliding is not the only way to change keyboard or the size

• Symbols panel

• Some advanced features were not tested• Mixed Chinese & English input without switching• Error tolerant

Confidential

Rev PA1 2009-3-10 41

Appendix – SUS (System Usability Scale)