NGOMSL Modeling

1

George Mason UniversityHuman Factors & Applied Cognitive Program

NGOMSL Modeling

Psyc645 -- Week 10

Wayne D. Gray and Thomas Mayfield

2


Next Week -- TUESDAY!!

NGOMSL#1 Produce NGOMSL methods from the trace of the VCR (my trace not yours)

Are no pops in NGOMSL. Verifies may be suitable replacements. Yours to

decide.

NGOMSL#2 Create NGOMSL methods for the ManTel task

Should be one set of methods that can be applied to both interfaces and to

1 or 2 queries

Obviously, not all methods need to be used in each analysis

Produce an NGOMSL trace for Dialog-Box 1 query and PopUp 2 queries

Readings Paul Green’s 1999 Human Factors Ergonomics Society conference article

3


Psyc645 -- Reprise of GOMS lecture 1

Levels of analysis based on Newell’s Time Scale of Human Action

Important to concept is that behavior at one level can be decomposed into

behavior at next lower level

Show what is added at various levels

Important to the argument that I am trying to make to convince you that analyses

at increasing lower levels entail decreasing degrees of generality (or increasing

degrees of specificity)

Programming VCR is a generally understood concept, but programming my VCR is

very different than programming George Lukas’ VCR. Setting Tom’s watch or Mary

Ann’s watch is very different than setting the clock in my car

Implications?

Behavior is not as unconstrained as we would like to believe. As we get more and

more specific in our description of behavior we find more and more constraints on

behavior.

4


Psyc645 - Overview of GOMS lecture 2

NGOMSL analyses Unlike KLM, NGOMSL is a serious notation for discussing

interactive behavior at levels lower than the unit task

Not simply a tool for analyzing design but a vehicle for

representing low-level performance

Today we will work through several examples

5


NGOMSL

Natural Language GOMS based on structured natural language notation and a procedure

for constructing them

models are in program form

Control Structure: Hierarchical goal stack

Serial or parallel: Serial

Level of Analysis: In principle, as necessary for your design question

6


NGOMSL - why?

More powerful than KLM. Much more useful for

analyzing large systems

More built-in cognitive theory

Provides predictions of operator sequence, execution

time, and time to learn the methods

7


NGOMSL - Overall Approach

Step 1: Perform goal/subgoal decomposition

Step 2: Develop a method to accomplish each goal List the actions/steps the user has to do (at as general and high-level as

possible for the current level of analysis)

Identify similar methods/collapse where appropriate

Step 3: Add flow of control (decides)

Step 4: Add verifies

Step 5: Add perceptuals, etc.

Step 6: Add mentals for retrieves, forgets, recalls

Step 7: Add times for each step

Step 8: Calculate total time

8


NGOMSL - Example

Car clock

Presetting radio stations simple

perverse

9


Provides predictions of methods and operators used to complete

a task. If you provide estimates of operator-duration, you can

get predictions of error-free expert performance time.

NGOMSL--Car Radio example (1)

10


set clockset mode

set hourset minute

release knobturn knobdepress knob (and keep it depressed)

press tune “-” buttonpress tune “+” button

NGOMSL-- Car Radio example (2)

Goal/subgoal hierarchy

11



Assumptions User's hands are NOT on the buttons at the beginning.

Time to move hand & arm to time change level is estimated by

2ft "reach" from Barnes (1963): 410 msec.

D = device time, time for clock to move forward one number, =

500 msec (we estimate this on the next slide)

I am assuming a 12-hr clock. Seems good assumption for a car

clock.

Radio is off at beginning and end.

Begin knowing the time you want to set the clock to.

12


Right Hand

Cognitive Operators

Visual Perception

Minimum time required to perceive clockTime=targetTime and to release the toggle lever

perceive info (X)

290

verify info (X)

50

initiate release (X)

50

release (X)

100


D = device time, time for clock to move forward one number, =

500 msec

13


set clockset mode

set hourset minute

release knobturn knobdepress knob (and keep it depressed)

press tune “-” buttonpress tune “+” button


• (Duplicate slide) (Duplicate slide)

• Note that method for accomplishing the top level goal will have three Note that method for accomplishing the top level goal will have three subgoals and one “release” physical operatorsubgoals and one “release” physical operator

14



stmt time oper

op time

tot time assumptions

Method for goal: SET-CLOCK 0.1 0.10

Step 1Accomplish goal: set Mode to set Clock 0.1 0.10

Step 2

Decide: If curHr≠targetHr Then: Accomplish goal: Change Hour display 0.1 0.10

Step 3

Decide: If curMin≠targetMin Then: Accomplish goal: Change Minute display 0.1 0.10

Step 4 Release "vol/on/off" knob 0.1 B 0.10 0.20

Step 5 Return with goal accomplished 0.1 0.10

Need methods for each of these three subgoalsNeed methods for each of these three subgoals

15



stmt time op

op time


Method for goal: set Mode to set Clock 0.1 0.10

Step 1 Recall "vol/on/off" is modeSwitch 0.1 M 0.50 0.60

retrieval from LTM or equivalent (location and perception of labels?) needed

Step 2

Decide: If location of modeSwitch is not known then Locate modeSwitch 0.1 M 1.20 1.30

For this example we assume that location is NOT known. If known then tot. time for this step would be the stmt time, 0.10 sec

Step 3Reach & Grasp "volume/on/off" knob 0.1 R 0.48 0.58

source: Barnes, 1963, reach = 410 msec, grasp = 070 msec

Step 4 Turn knob 0.1 T 0.34 0.44 source: Barnes, 1963

Step 5Home thumb to "volume/on/off" knob 0.1 H 0.40 0.50

400 msec seems too high here, might use 100 msec for button press instead.

Step 6 Press and hold knob 0.1 B 0.10 0.20Step 7 Return with goal accomplished 0.1 0.10

16



stmt time oper

op time


Method for goal: Change <time> display 0.1 0.10

Step 1

Decide: If location of setTime switch is not known then Locate setTime switch 0.1 M 1.20 1.30 assume that loc is not known

Step 2

Decide: If finger is NOT on setTime switch then: Reach to setTime switch 0.1 R 0.41 0.51

Step 3Retrieve-from-LTM that <time> = "+ or -" 0.1 M 0.50 0.60

retrieval from LTM or equivalent (location and perception of labels?) needed and takes ≈ 500 msec

Step 4 Determine current setting 0.1 P 0.32 0.42

based upon cpm-goms analysis it should take 420 msec for eye to move to clock and for user to perceive the hour.

Step 5Press and hold setTime switch to <time> 0.1 B 0.10 0.20

cpm-goms calc. is ≈250 msec, go with KLM

Step 6Hold until <curTime> = <targetTime>, (500 x #digits) W

System response time is 500 msec/digit, this is enough time for Ss to perceive and initiate keyUp response.

Step 7 Verify display <time> 0.1 M 1.20 1.30Step 8 Return with goal accomplished 0.1 0.10

17



time for SetClock goal 0.70time for setMode goal 3.82

time for setHour from 1 to 10 9.03

time to setMin from 56 to 50 31.53total 45.08

18


NGOMSL-- PreSetting a station (1)

Locate Station

Select AM/FM

Initiate seek

Identify StationTerminate

seekSet Mode to

RadioetcSet PreSETEnter PreSet

MODEEnter PreSet

ButtonExit PreSet

MODEGeneral goal hierarchy for presetting a stationPreSET Network

Show

19



Note differences between this and previous slide.

Locate Station

Select AM/FM

PreSET Network Show

Set Mode to Radio

etc

Set PreSET

Goal hierarchy for optimal design

press & hold preset button until sound returns

seek & identify

20



MODEL 1: SETTING A PRESET TO A NETWORK SHOW - OPTIMAL DESIGN

Method for goal: preset network showStep1: Decide: IF radio is not playing, THEN Accomplish goal:

set-mode-to-radioStep2: Decide: IF show N.E. intended show, THEN Accomplish

goal: locate-stationStep3: Accomplish goal: set-presetStep4: Return with goal accomplished

Method for goal: locate-stationStep1: Decide: IF band N.E. desired band, THEN Accomplish

goal: select AM/FMStep2: Accomplish goal: seek&identifyStep3: Return with goal accomplished

Method for goal: select AM/FMStep1: Reach to button for desired bandStep2: Press button for desired bandStep3: Verify band correctStep4: Return with goal accomplished

21



Method for goal: seek&identifyStep1: Reach to right side of "Seek" button Step2: Press "Seek" buttonStep3: Decide: IF show N.E. intended show, THEN Goto 2Step4: Verify show correctStep5: Return with goal accomplished

Method for goal: set-presetStep1: Reach to desired "Preset" buttonStep2: Press and hold "Preset" buttonStep3: Wait until sound returnsStep4: Remove finger from "Preset" buttonStep5: Return with goal accomplished

22


NGOMSL-- PreSetting perverse (1)

Locate StationSelect AM/FMInitiate seekExit seek

MODE

Set Mode to Radio

etcAssign new PreSETEnter

PreSet MODE

Enter PreSet ButtonExit

PreSet MODE

Goal hierarchy for perverse designPreSET Network ShowDelete old PreSETEnter delete MODE

Enter PreSet Button

Exit delete MODE

Set PreSETEnter seek mode

seek & identify

23



ssEJ

– TUNE +

PUSH CLOCK

ON OFF

FMAM

– +BASS

– +TREBLE

L R

BALANCE

L R

FADE2:41

1 2 3

4 5 6

Radio for Perverse Preset Design

SEEK MODE

ERASE PRESET

MEMORIZE PRESET

ssSEEK

24



25



Method for goal: initiate-seekStep1: Accomplish goal setMode-SEEKStep2: Accomplish goal: seek&identifyStep3: Accomplish goal: exit seek modeStep4: Return with goal accomplished

Method for goal: setMode-<mode>Step1: Reach for <mode> buttonStep2: Press <mode> buttonStep3: Verify <mode> correctStep4: Return with goal accomplished

Method for goal: seek&identify (s/a optimal)Step1: Reach to right side of "Seek" button Step2: Press "Seek" buttonStep3: Decide: IF show N.E. intended show, THEN Goto 2Step4: Verify show correctStep5: Return with goal accomplished

Method for goal: set-presetStep1: Accomplish goal: delete-previous-presetStep2: Accomplish goal: assign-new-presetStep3: Return with goal accomplished

26



Method for goal: delete-previous-presetStep1: Accomplish goal: setMode-deletePreSet-on

(setMode-<mode>)Step2: Accomplish goal: enter-preset-buttonStep3: Accomplish-goal: setMode-deletePreSet-off

(setMode-<mode>)Step4: Return with goal accomplished

Method for goal: assign-new-presetStep1: Accomplish goal: setMode-preset-on (setMode-

<mode>)Step2: Accomplish goal: enter-preset-buttonStep3: Accomplish-goal: setMode-preset-off (setMode-

<mode>)Step4: Return with goal accomplished

Method for goal: enter-preset-buttonStep1: Reach to desired "Preset" buttonStep2: Press "Preset" buttonStep3: Return with goal accomplished

27


NGOMSL--comparison of optimal and perverse designs

Optimal Perverse

number of different methods 6 11

number of methods used 6 18

number of different steps 21 38

number of steps required 21 66

28


Cost-of-Knowledge Characteristic Function

Card, S. K., Pirolli, P., & Mackinlay, J. D. (1994).

The cost-of-knowledge characteristic function:

Display evaluation for direct-walk dynamic

information visualizations. In B. Adelson, S.

Dumais, & J. Olson (Eds.), ACM CHI'94 Conference

on Human Factors in Computing Systems (Vol. 1,

pp. 238-244). New York: ACM Press.

29


CENTURY-METHOD =GOAL: DO-TASK

GOAL: GET-DATETURN-TO [MANUSCRIPT]GET-DATE

GOAL: ACCESS-DAY-CALENDARGET-YEAR . . . if necessaryGOAL: SELECT-CENTURY (1700’s)

POINT-TO (Century=1700-1790s))=> Century-display

GET-YEAR . . . if necessaryGOAL: SELECT-DECADE (1710’s)

POINT-TO (1710-1719))=> Decade display

GET-YEAR . . . if necessaryGOAL: SELECT-YEAR: (1719)

POINT-TO (1719))=> Year-display

GET-MONTH . . . if necessaryGOAL: SELECT-MONTH: (November)

POINT-TO (November))=> Month-display

GET-DAY . . . if necessaryGOAL: SELECT-WEEK: [??]

POINT-TO [23]=> Week-display

GET-DAY . . . if necessaryGOAL: SELECT-DAY: [23]

POINT-TO (23))=> Day-display

32


33


But,

Card, Pirolli, & MacKinley did not do a complete

GOMS analysis

Only did enough of an analysis to determine that the

“volume part” of the task was the repeated accesses

of menus for each component (century, decade, year,

month, week, day)

34


Cost-of-Knowledge Function

Small amounts of knowledge can be accessed

quickly with access costs increasing as amount of

knowledge accessed increases ≈6.9 s for 1 day

≈10.4 s for 7 days

≈14 s for 30 days

≈17.5 s for 365 days

≈21 s for 3,562 days

≈24.6 s for 36,525 days

A = Spiral Calendar as isA = Spiral Calendar as isB = SC w/ 2 s cycle time (was 3.5)B = SC w/ 2 s cycle time (was 3.5)C = Eliminate need to select “week” can do C = Eliminate need to select “week” can do that when select day from the month calendarthat when select day from the month calendar

D = Eliminate “week” & improve cycle timeD = Eliminate “week” & improve cycle time

36


Design Improvements?

Speed up system response time (SRT)

Time to access = 3.3 + 3.5 * Ncycles

Imply in text that it takes 1 s to point to item and 2.5 s for system to bring

up item

Eliminate some of the cycles -- seem redundant

Get rid of the week cycle

Have users bring up month and directly select day from the month

calendar rather than first selecting week and then month

Issue --

Implicit estimate of 2.5 s srt seems tediously long

Suggest need for a full NGOMSL analysis

37


NGOMSL Methods for Spiral Calendar

stmt time op t/f

op time tot time assumptions

Method for goal: Set-Date 0.1 0.10Step 1 Accomplish goal: get-date 0.1 0.10Step 2 Accomplish goal: Access-day-calendar 0.1 0.10Step 3 Return with goal accomplished 0.1 0.10

Total Time 0.40

Method for goal: get-date 0.1 0.10

Step 1 Decide: IF page not turned, then turn page 0.1 R T 0.88 0.98

source: Barnes, 1963, reach = 410 msec, grasp = 070 msec, + 400 est for flip. ** could analysis this further e.g. CPM-GOMS

Step 2 Read Date 0.1 M 1.2 1.30use M for estimate of time to read and put into memory

Step 3 Home hand to keyboard 0.1 H 0.4 0.50Step 4 Return with goal accomplished 0.1 0.10

Total Time 2.98

Top method and one subordinateTop method and one subordinate

38


NGOMSL Methods (cont’)

Method for goal: access-day-calendar 0.1 0.10Step 1 Decide: IF forgot date, then Read Date 0.1 M F 1.20 0.10Step 2 Accomplish Goal: Set-<timeUnit> 0.1 0.10

Step 3Decide: If current_day N.E. target_day, then GO TO Step 1 0.1 0.10

Step 4 Verify date 0.1 M 1.20 1.30Step 5 Return with goal accomplished 0.1 0.10

Total Time 1.80

Method for goal: Set-<timeUnit> 0.1 0.10Step 1 Point to <timeUnit> 0.1 P 1.10 1.20Step 2 ClickOn <timeUnit> 0.1 BB 0.20 0.30Step 3 Wait for System Response 0.1 W 1.00 1.10Step 4 Return with goal accomplished 0.1 0.10

Total Time 2.80

0.10 Method for goal: Set-Date0.10 Step 1 Accomplish goal: get-date0.10 Method for goal: get-date0.98 Step 1 Decide: IF page not turned, then turn page1.30 Step 2 Read Date0.50 Step 3 Home hand to keyboard0.10 Step 4 Return with goal accomplished0.10 Step 2 Accomplish goal: Access-day-calendar0.10 Method for goal: access-day-calendar0.10 Step 1 Decide: IF forgot date, then Read Date0.10 Step 2 Accomplish Goal: Set-<timeUnit>0.10 Method for goal: Set-<timeUnit>1.20 Step 1 Point to <timeUnit>0.30 Step 2 ClickOn <timeUnit>1.10 Step 3 Wait for System Response0.10 Step 4 Return with goal accomplished0.10 Step 3 Decide: If current_day N.E. target_day, then GO TO Step 10.10 Step 1 Decide: IF forgot date, then Read Date0.10 Step 2 Accomplish Goal: Set-<timeUnit>0.10 Method for goal: Set-<timeUnit>1.20 Step 1 Point to <timeUnit>0.30 Step 2 ClickOn <timeUnit>1.10 Step 3 Wait for System Response0.10 Step 4 Return with goal accomplished0.10 Step 3 Decide: If current_day N.E. target_day, then GO TO Step 10.10 Step 1 Decide: IF forgot date, then Read Date0.10 Step 2 Accomplish Goal: Set-<timeUnit>0.10 Method for goal: Set-<timeUnit>1.20 Step 1 Point to <timeUnit>0.30 Step 2 ClickOn <timeUnit>1.10 Step 3 Wait for System Response0.10 Step 4 Return with goal accomplished0.10 Step 3 Decide: If current_day N.E. target_day, then GO TO Step 11.30 Step 4 Verify date0.10 Step 5 Return with goal accomplished0.10 Step 3 Return with goal accomplished

40


How well does NGOMSL fit the data?

Pretty well by rPretty well by r22

41



0

5

10

15

20

25

30

1 2 3 4 5 6

Number of Cycles

Time in Seconds

empirical data card model NGOMSL

Pretty well compared to the equation they Pretty well compared to the equation they derived from the dataderived from the data

42



0

5

10

15

20

25

30

35

1 2 3 4 5 6

Number of Cycles

Time in Seconds

empirical data card model NGOMSL NGOMSL w/2.5 s srt

But not too well when I assume a 2.5 sec But not too well when I assume a 2.5 sec system response timesystem response time

43


What Happened?

Did the modeler cheat?

Let’s examine the assumptions of the model

Critical assumptions concern the time per cycle and

the elements that compose the cycle

Card et al., empirically derived 3.5 s per cycle

The NGOMSL model, which assumed a 1 s SRT,

predicted 3.1 s per cycle

0.10 Method for goal: Set-Date0.10 Step 1 Accomplish goal: get-date0.10 Method for goal: get-date0.98 Step 1 Decide: IF page not turned, then turn page1.30 Step 2 Read Date0.50 Step 3 Home hand to keyboard0.10 Step 4 Return with goal accomplished0.10 Step 2 Accomplish goal: Access-day-calendar0.10 Method for goal: access-day-calendar0.10 Step 1 Decide: IF forgot date, then Read Date0.10 Step 2 Accomplish Goal: Set-<timeUnit>0.10 Method for goal: Set-<timeUnit>1.20 Step 1 Point to <timeUnit>0.30 Step 2 ClickOn <timeUnit>1.10 Step 3 Wait for System Response0.10 Step 4 Return with goal accomplished0.10 Step 3 Decide: If current_day N.E. target_day, then GO TO Step 10.10 Step 1 Decide: IF forgot date, then Read Date0.10 Step 2 Accomplish Goal: Set-<timeUnit>0.10 Method for goal: Set-<timeUnit>1.20 Step 1 Point to <timeUnit>0.30 Step 2 ClickOn <timeUnit>1.10 Step 3 Wait for System Response0.10 Step 4 Return with goal accomplished0.10 Step 3 Decide: If current_day N.E. target_day, then GO TO Step 10.10 Step 1 Decide: IF forgot date, then Read Date0.10 Step 2 Accomplish Goal: Set-<timeUnit>0.10 Method for goal: Set-<timeUnit>1.20 Step 1 Point to <timeUnit>0.30 Step 2 ClickOn <timeUnit>1.10 Step 3 Wait for System Response0.10 Step 4 Return with goal accomplished0.10 Step 3 Decide: If current_day N.E. target_day, then GO TO Step 11.30 Step 4 Verify date0.10 Step 5 Return with goal accomplished0.10 Step 3 Return with goal accomplished

3.1 s3.1 s

3.1 s3.1 s

3.1 s3.1 s

45


Where is the problem?

For NGOMSL to accept 2.5 s SRT means either accepting a very bad fit, or

changing the model

But, what can be changed?

My guess is that they never measured the SRT of

their system

In any event it is incredible that they assumed that all

of the cycle time not devoted to pointing was

attributable to SRT

46


Two Morals to this Story?

First Even good researchers can be careless when they do not check

their assumptions against a model

(& yes, Card, Pirolli, and Mackinlay are VERY good researchers)

Second Before investing lots of effort into jazzing up system response

time, do a profile analysis with a complete GOMS model

47


Next Week -- TUESDAY!!

NGOMSL#1 Produce NGOMSL methods from the trace of the VCR (my trace not yours)

Are no pops in NGOMSL. Verifies may be suitable replacements. Yours to decide.

NGOMSL#2 Create NGOMSL methods for the ManTel task

Should be one set of methods that can be applied to both interfaces and to 1 or 2

queries

Obviously, not all methods need to be used in each analysis

Produce an NGOMSL trace for Dialog-Box 1 query and PopUp 2 queries

Readings Paul Green’s 1999 Human Factors Ergonomics Society conference article

48


End of NGOMSL

Documents

NGOMSL Modeling