8/7/2019 12237_KE

1/25

Chapter 13Knowledge elicitationNigel Shadbolt and Mike Burton

IntroductionExpert systems are cpnlputcr programs which a re u~tendedto s o l v e real-world problems, a i h l e ~ m gthe sarns levcl of dicuralry as h u m a n cxpcr ts .There are m a n y obstacles in such an endeavour . L7ne of t h e greacesr is rhzacquisition u f th e knowledge which human exper t r u5r in thcir problemsolving. The issue is so important to the developrncnt of k1lowlz i i~e-k3redsystem: rhat ~t hu been described as the 'bottle-neck m E x p c r r Sy i t en l jcons truc t lor i ' (Hlycs-Roth et a / . , 1983).De sp~ t eits cer~rralrole there is no comprehensive t heory of knowledgeacq! is ir ion ~ r ~ a ~ l a h l c .Many regard th e area as an a r t r a t h e r than a science. I tis n u t thc pi l rposr u l th is chapte r to inves t iga te the throrc t lca l s h o r ~ ~ i g i n g so i krlowledgr acqulsinon b u t to deliver practical advice and gu.;"",:'.,mw,>~ceo ni',,~.,per forming t h e p1.3c'255. :.4 lipExpert systemsIn th e early days of Arc~ficialI n t c hg cncc much cKxt went ~ritoa t r e p ,$ s to;:;, 0discover general principles of lilt e l l ~ g e ~ ~t bchav~oi l r . Newel1 an d ,:,$$;n's(1963) Genera l Probleni Solver e x e m p l ~ f i r rr lus apprcuch They wereinterestedin uncovermg a general problem s o l v ~ i > gsrraregy which could be used forany human task. In th e early 1970s this pos~ncr ncame to be challenged. Anew s logan came to prorninence-'in th c knowlcdge l ~ e st he power' . Al e a d ~ n gexponent of [his view wa s E d w a r d Feigenbaum of SRI. i l t - observedthat cxpens are experts b y vir tue of dornaln specific probIem solv~ngsrrateglestogether with a great deal of domain 5pecific knowledge. It w as the atternptto incorporate these variqur sorts of domain knowledge which r r s u l t e d 113[he class of programs taUcd E xper t Sysrems.

Throughout this chapter we will bc asjumkg that current cornrncrnallyava~lableexpert sys t em s o f ' t w l r c will be the implementation vehicle for t h eprograms. T h u s the fortn in rrhich the knowlcdgc wil l be i rnplcmented 1s

8/7/2019 12237_KE

2/25

Techniques :t i dc~i,

8/7/2019 12237_KE

3/25

T h ~ schap t e r will continuc by describing, 111 su3aen t drcail h r t h e reddert9 apply them, ex amp les of major KEmetl~vils We w11l t hcn mcnnon o the rt cchniques and where rhe reader can find out u l p r r abaur h e m In 1a:erszccious we wdl review asp:cts of expcrtrsr ~ n diognit~onrhdr A K C likely tod ~ r r i r l yaffect th e KE process. Rnally, u r c J c s c r r Lc t h e cr>nstruction ofprogrammes of acquisition.

.Methods of knowledge elicitatiotlThe structured interviewAlrrlost everyonc stArt; in K E b y detct-~rlitlingto u se a n interview. Theinterview is th e 1 7 7m t r ~ n l s l ~ o r ~ i yLISCJknowledge chcitation technique andtakes many fhrms. from rh r c u m p le c c ly u:~~tnrc~urcdinterview to t h e formal ly-plarlned, srrucrrired mterv iew ( F u r A full review of ~ntcrvicwtechniques sccSinclair in th i s v o l u m e ! The s r ruc tu rc J lnrcrvlcw 1s a formal ve r s i on inwhich thc h o w l c d y c e n p c t r h a s p l ~ n n e d[h e wllolc session. The s t ruccc--cdIatervlew has t h e advanragc rhar it prov~desrrr:~cturcd transcripts that' .arccajier to analyse t h ln unstructured 'cllat'. T h e rei~ovelyfqrmal 1nrt.i-vitwwhich w e havc specified here constrams the cupc r t - l~c i to r dialogue c i l thegcne r a l prir lc ip l rs of th e d o m ain . E.rpert5 d o no c woik t h r ~ u g ha p ~ r r i c c ~ l ~ rscrnar irs rr.tractcci from th e domain by thc cIicltur; r a th er t l ~ ccxpcrrs gz tx r i r c ltherr oxl.,.;n ~ i c r ~ ~ r i o sas rhc intemiew progrehscs . T h e > t r u i n i r c of a typic11inter7;lesv 15 25 follows.

1. Ask the cxpcrt to givr a bricf (IG inin) outllrlc oi thr targcr t a s k ,inilud!tlg the h l l o w i n g i n t b r m a t i o r ~ :(A) at ) outl l t le of th e task , inc luding a dcscript~onof t l ~ c pnssiblc

solur innr to th e prob len l ;( b j 1 descriyrion o f [he v ar i ab l es which a f fec t che choicc of s o l u t ~ o n s ;( c ) a l l s t ; ~ frn330r ru les which conncc r r k r r ~ r ~ a h l c sto t h o s o l u t i ons .2. T a k e each rulc cIlcltcd In rtagc 1 , dsk .nhci~i r is a p p r u p r j a t c a n d w h e n

11 is :lor. T h e aim is to revca! t h e s copc ( K ~ n c : ~ l l t ~and spccifici ty) of cachexist ing ru le , arid hopclul ly gcnerarc sLnIc n c LV r u l ~ j .3. Rcprat \rage 2 un t i l l r I \ c le lr t l i a t ~ h ccup;rr will not produce any

a d d ~ t ~ o ~ l a litiformat ion.I t is important In n s i n g this r c c h n ~ q u c1 0 b c c l c x 2nd spccific about how toperform 5tage 2 . Wc h a v e f o u n d r l u t !t 1s k~r l p f u lt u ionsrrzin th c cljcitor'si t ~ t c rv cn t i o n st u a specific sct of ptobr: , c ~ c hwith a spccific f u n c t i o n . Herris a list of probcs (P ) and r e I ~ t e dfunctions (F ) which will hclp in s c ~ g c2 .P I Why would yoit d o that 'F l C o r ~ v c r t san assertion into a r u l e .P2 How w o u l d y o u du [ha t?F2 Cr-ll~rr,irrs!E,CP : . . " - IE~T;!CS.

8/7/2019 12237_KE

4/25

W %would you do that? I s dways the case?m.R e y a k the generality o~!thc rule md may generate other ruIes..Wkpt drernatives to a re there?F4. , w em e ru les.@ ,WEif i t were not the case that .A -raw rules for when current condition d m not apply.P6 Can you telI me more about W used-to generate funher dialogue-if expert dries up .The i d a h k e is that th e e l i d to r -engages in a type of s l ~ t / f i l l e rdialogue.The requirement t h a t the elicitor hstcm out for relevant concepts and rclatiom&pcms a large c w t i v e load on the elicitor-Th e provision o f fixed hnguisr ic!&ms.!- wbi&oo ask qu&ons a b u t concepts, relations, a rtribures and

, -&,k:&e, e&mr'r job,veq.mncb &a. I t also p r o v i d e sharply.,,: .-.,v-5~,wb,i&,. " -' ,. . @i l i t a ~& p r m of extracting usable knowledge.:; ;.: Q & - b . k a h i smtca when none of th e above probes are

I " ' ..(su& .as the as e when the Jidtor wants the cxpcrt to clarify'=.However, yo u r h d d try m keep the inrcjccdonr nccaraiy md $mations to a minimum. T h e p in t of sp ec i fy ~n gsuch a fixed se t ofh g u i s d z p r o k i s ro constrairl th e expert to giving you al l , and only, theinformition you want.

The sample of diaIogue below is taken horn a real interview of this kind.It is t h e transcript of an interview by a knowledge engineer (KE) with anq p q ,(a)on V13U fault diagnms*.@:-4,%dy &&ed t h e port of th e computer.~ ~ ! f l c t i d . y ~ & & ' t h eF?

, .~ ; . ~ $ - k i f - ' sbLghming recently hen it's a good idea to check th e port t< .*.;-; . b u s t ltgbming tends to damage the ports.&k,&dj&' ioly ai&ativsr m thatq'Y q &at ought to be prefaced b y saying do that if i t was several keysA!,,:: -!*& effects + n o t necessarily all of t hem, b ut rnorc t h a n 2.BE Why does it have to be more than 2?EX Well if it was only o n c o r two keys doing funny things then the th ing

to do w d d be t o check the keys themselves t check th e contacts ofth e keys + check that they're closing properly + speed would affecta il keys, parity would affect a b o u t half th e keys.

This is quire a rich piece of dialogue. From this section o f the interviewalone we can extract the followinR rules:IF there has been recent l ~ g h t c n i n ~T H E N check port for damageIF there are two or fewer malfunctioning keysTHEN check the ke y contactsIF about half the keyboard is malfunctioningTHEN check the parity'In thc rransuriprs we use thc symbol + to reprcrrnt a pausc in the di1log.x

8/7/2019 12237_KE

5/25

IF the whole ktyboard is ma1funrtic)ningTHEN check the speedOf course these ru les miy need rcfining jn later cliritation srssion~,b ~ r rth c text of the d ia logue shows ho w th e usc of th c jpec i f ic probes h ~ srevealed

a welI-structured rcsponse from the expert t .I n r ll the in tervi tw yechniques (and in some of t h e vther generic t e chn iques

as well) there e x i s t a namher of dangers that have become fa~niliartoknowledge engineers. One problem is that expert5 will o n l y ~ : o d u c ewhatrhey can v e r b h e . If there are non-verbalitable aspects co thr domain , theInterv iew wiLl n o t recover them. Th i s ca n arise f rum w;o causes. It mAy b ethat tht knowledgr was never cxpl inc1~r cp r csen ted or articulated in termsof language (consider, f o r example , pattern recogninon expemse). T h e n therei the situation where th e knowledge was o n g n a l l * learnt expl in t ly in aprupositional form b u t th e experts may have c o m : ~ i i e d rh e knorv ledge to suchan ex ten t chat they regard the complex decisiorls they makc as based nnh u n ch es or intuitions; In fact, these decisions ire based upon large amountsof r emcmbercd d a t a an d experience, an d th e continual apphcation ofstrateg ies.In this situatiur: [hey tend t~ give black box replics "I don't know how I d ot h ~ t. . . ." , " I t IS obviously t h e right t h ~ n gto d o ." .

Arlodler problem horn th e observation t h ~ tp c ~ p l e(and experts uiparticular) often seek to jusdfy their densions in any way they can. It is acommon experience of the knowledge e n p e e r to get a pcrfcctly vahddeci.jian from an e xpe r t , an d thtn to be g i r t n a spurious justification Forthe>< and other reasons we have to supplement intervitws with addirimalmcrhods of elicitation. Elicitation should always consist of a programme oftechniques 311d ined~ods.This bnngs us on ro cms idcr another techniquem u c h favourrd by knowledge engineers.Protocol analysisProtocol A n a l v j i ~(PA) (considered in dctail bv Baisbridge in this book) isa generic t e r m ior a number of different ways of prrtbrming some form o fana!ysis of the expert(s) actually solving problems in th e domain. [n al l cnscst h e englrieer takes a ~ ~ ~ 3 r dof w h ~ tthe expert does-prcferably by vidcn oraudio t;lpt--or a t least by wnrtcn notcs. Protocols arc t h e n made from theserecords a nd t t c knowledge enylneer tries tu ext rac t meaningful ru lcj fromthc protocols.

Wlis can distinguish two general type.; of PA-t~rl-l in~ and of-line. In on-l ine PA t he expe r t is bejng recorded solving a problem, ar,d cilncurrcntly acommenrary is made. The nature of this r un l m en t a r y sper~ f i est h e twosubtypes of the on-!ine method. The expert perlorming the cask may bedrscr tbing what t h e y are doing as problem solv ing pr r~c reds .This IS iaUedt I n fact. 3 ~ S I I ~ I C~ e r o n d - p h o ~ rC I I L ~ ~ X ~ O ~techn~quewould bc tu prrsect thcrc r u l t r bark t o !hrcx per t a r d ark. abut thelr rrurhtulntss zcrlpc and so farth

8/7/2019 12237_KE

6/25

8/7/2019 12237_KE

7/25

8/7/2019 12237_KE

8/25

8/7/2019 12237_KE

9/25

8/7/2019 12237_KE

10/25

8/7/2019 12237_KE

11/25

8/7/2019 12237_KE

12/25

8/7/2019 12237_KE

13/25

8/7/2019 12237_KE

14/25

8/7/2019 12237_KE

15/25

8/7/2019 12237_KE

16/25

8/7/2019 12237_KE

17/25

8/7/2019 12237_KE

18/25

8/7/2019 12237_KE

19/25

8/7/2019 12237_KE

20/25

8/7/2019 12237_KE

21/25

8/7/2019 12237_KE

22/25

8/7/2019 12237_KE

23/25

8/7/2019 12237_KE

24/25

8/7/2019 12237_KE

25/25