31
Using Annotations in Online Group Chats + CHI 2014 - Na Li, Mary Beth Rosson / 윤정원 x 2014 fall

[1002 Lab meeting] Using Annotations in Online Group Chats

Embed Size (px)

DESCRIPTION

Using Annotations in Online Group Chats

Citation preview

Page 1: [1002 Lab meeting] Using Annotations  in Online Group Chats

Using Annotations in Online Group Chats + CHI 2014 - Na Li, Mary Beth Rosson / 윤정원 x 2014 fall

Page 2: [1002 Lab meeting] Using Annotations  in Online Group Chats

Using Annotations in Online Group ChatsCHI 2014

Page 3: [1002 Lab meeting] Using Annotations  in Online Group Chats

THE AUTHORSU

sing

Ann

otat

ions

in

Onl

ine

Gro

up C

hats

Na Li Mary Beth Rosson

College of Information Sciences and Technology Pennsylvania State University

1

Page 4: [1002 Lab meeting] Using Annotations  in Online Group Chats

2KEYWORDS

• Instant annotation

• Information interfaces

• conversation control

• design

• evaluation

Usi

ng A

nnot

atio

ns

in O

nlin

e G

roup

Cha

ts

Page 5: [1002 Lab meeting] Using Annotations  in Online Group Chats

3U

sing

Ann

otat

ions

in

Onl

ine

Gro

up C

hats

• 문서에 주석을 다는 것은

중요한 내용을 추출하고, 아이디어와 생각들을 표면화하는 널리 사용된 전략이다.

• online conversation는 글로벌한 회사들에게 중요한 커뮤니케이션 미디어이다.

• 온라인 채팅과 같이 ‘annotating dynamic texts’ 는 연구되지 않았다.

ABSTRACT

BUT

Page 6: [1002 Lab meeting] Using Annotations  in Online Group Chats

3U

sing

Ann

otat

ions

in

Onl

ine

Gro

up C

hats

• 문서에 주석을 다는 것은

중요한 내용을 추출하고, 아이디어와 생각들을 표면화하는 널리 사용된 전략이다.

• online conversation는 글로벌한 회사들에게 중요한 커뮤니케이션 미디어이다.

• 온라인 채팅과 같이 ‘annotating dynamic texts’ 는 연구되지 않았다.

Instant Annotation(IA)

BUT

Standard chat tool

a real-time annotation-enhanced chat tool

multilingual groups

ABSTRACT

Page 7: [1002 Lab meeting] Using Annotations  in Online Group Chats

• provide users ability to connect text-based comments

• offer an additional space for interaction among users

INTRODUCTIONU

sing

Ann

otat

ions

in

Onl

ine

Gro

up C

hats

4Collaborative annotation tool

• features must be …. real time communication + collaboration

Page 8: [1002 Lab meeting] Using Annotations  in Online Group Chats

- Some observations on the "live" collaborative tagging of audio conferences in the enterprise

• web-based application, Live Conference Dashboard (henceforth LCD)

• 댓글 지원하지 않음

5Related StudiesU

sing

Ann

otat

ions

in

Onl

ine

Gro

up C

hats

1. Collaborative tagging system

• 정교화되지 않음

• 실시간 오디오 미팅에서의 태깅

• 태깅하려면 LCD에 로그해서 태그하면 다른 사람들에게 태그들이 보인다.

• tag cloud, tagging dashboard, timeline

• 유저들이 태깅하는 것과 미팅에서 활발히 참여하는 멀티태스킹을 잘 못한다. - 인지적으로 부가되는 다른 요소들 때문에

Page 9: [1002 Lab meeting] Using Annotations  in Online Group Chats

• a collaborative annotation sidebar

• mark an important piece of content (due date), a meeting proposal,

an idea and reaction, and a to-do item.

5Related StudiesU

sing

Ann

otat

ions

in

Onl

ine

Gro

up C

hats

2. Instant Annotation(IA)

- Instant annotation: early design experiences in supporting cross-cultural group chat

• Instant Annotation (IA)

• 각각의 주석 섹션들은 옆에 대화창의 텍스트 라인에 맞춰있다.

-> 같은 컨텍스트 안에 또 다른 주석을 넣으면, 원래 남겨진 주석은 밑으로 내려가서 쌓이게 된다.

Page 10: [1002 Lab meeting] Using Annotations  in Online Group Chats

5Related StudiesU

sing

Ann

otat

ions

in

Onl

ine

Gro

up C

hats

2. Instant Annotation(IA)

- Instant annotation: early design experiences in supporting cross-cultural group chat

• 현재 annotation이 어딘지 살필 수 있고, 이전 annotation을 스크롤해서 볼 수 있다.

• 쌓여져 있는 annotation list에서, 유저들은 annotation section을

마우스 오버해서 확장하고 full listing 을 볼 수 있다.

Page 11: [1002 Lab meeting] Using Annotations  in Online Group Chats

How is an IA chat tool used?

How is user experience affected by the IA functionality?

What are the costs of IA in group chats?

1

2

3

we used the prototype to explore three research questions:

Page 12: [1002 Lab meeting] Using Annotations  in Online Group Chats

6MethodU

sing

Ann

otat

ions

in

Onl

ine

Gro

up C

hats

Using Instant Annotation In Group Chats

• a comparison study using mixed methods (behavior, surveys, interviews) • data collected from two conditions run at different points in time

Page 13: [1002 Lab meeting] Using Annotations  in Online Group Chats

Task & Procedure

• 4명의 참여자들이 그룹을 만들어서 토의

• 토의 주제는 “how to spend $5000 to support environmental sustainability”

• 15분동안, 최소 8개의 아이디어들을 내달라고 했다.

• 그리고 3개의 베스트 아이디어들을 골라달라고 했다.

• 브레인스토밍+decision-making 이 과정은 실제 사무실에서 회의하는 실제 상황과 비슷

• 업무 수행후, 참여자들은 사후 경험 조사 서베이 완성,

• 개인별 인터뷰

6MethodU

sing

Ann

otat

ions

in

Onl

ine

Gro

up C

hats

Using Instant Annotation In Group Chats

Page 14: [1002 Lab meeting] Using Annotations  in Online Group Chats

Survey

• 6 scale: communication quality를 평가 - e.g. “The conversation seemed highly interactive”

• 4 scale : satisfaction - e.g. “I enjoyed communicating with my group members”

!!• 두번째 그룹(IA)에서는 IA툴 반응에 대해 5개 더 조사

e.g. “I could easily review messages using this chat tool”

“I am distracted by the annotations in the side bar”

6MethodU

sing

Ann

otat

ions

in

Onl

ine

Gro

up C

hats

Using Instant Annotation In Group Chats

• 이전 online group discussion 연구에서 수행한 10 Likert-style rating scales 기준으로

• scale value range : 1= Strongly Disagree

5= Strongly Agree

Page 15: [1002 Lab meeting] Using Annotations  in Online Group Chats

Participants

6MethodU

sing

Ann

otat

ions

in

Onl

ine

Gro

up C

hats

Using Instant Annotation In Group Chats

• 각각의 조건들은 4 참여자들의 5그룹

: 각 그룹 —> 2 native + 2 non-native speaker

(성별, 교육수준, 언어능력) 유사 조건

( native와 non-native speaker들이 두 개의 다른 chat tool을 사용 할 때

개별 그룹의 chat behavior와 reaction을 정당하게 비교하게 한다. )

Page 16: [1002 Lab meeting] Using Annotations  in Online Group Chats

7DATA ANALYSIS AND RESULTSU

sing

Ann

otat

ions

in

Onl

ine

Gro

up C

hats

1. How Was the IA Tool Used?

• 4개의 IA group에 의해 생성된 annotaion의 수는 12~23개, total 90

• 참여자들은 실제로 IA 기능들을 사용했는데, 그룹들이 그 기능들을 사용하는 법은 다양했다.

five major use patterns

71% of the annotations

Page 17: [1002 Lab meeting] Using Annotations  in Online Group Chats

7DATA ANALYSIS AND RESULTSU

sing

Ann

otat

ions

in

Onl

ine

Gro

up C

hats

1. How Was the IA Tool Used?

• 왜 유저들이 IA 메인 채팅창이 아니라, side bar에 정보들을 추가하는가?

- Add Information 중에 몇 가지 케이스들을 집중 분석

! 1. 새로운 아이디어는 side bar에 제안

처음에 몇가지 안들을 쭉 확인하고, 다시 돌아가서 각각의 안들에 나온 아이디어들을 디벨롭

주석들이 아이디어 하나가 되는 것

2. 채팅에서 나온 아이디어를 확장

Page 18: [1002 Lab meeting] Using Annotations  in Online Group Chats

2. How Was User Experience Affected by the IA Tool?

7DATA ANALYSIS AND RESULTSU

sing

Ann

otat

ions

in

Onl

ine

Gro

up C

hats

• IA tool의 유저 경험을 조사하기 위해, 그들의 토의 경험들의 평가를 분석.

- 커뮤니케이션 과정과 토의에서 그들의 만족도를 초점으로.

!• 개인별 사후 인터뷰 진행

Page 19: [1002 Lab meeting] Using Annotations  in Online Group Chats

1) Quality of Communication (QC)

• Tool and Speaker-type

- a main effect of Speaker (F(1,36)=6.11, p<.02)

- no effect of Tool or interaction of Tool and Speaker. !

7DATA ANALYSIS AND RESULTSU

sing

Ann

otat

ions

in

Onl

ine

Gro

up C

hats

2. How Was User Experience Affected by the IA Tool?

- 커뮤니케이션의 품질을 평가하는 여섯 평가 척도로 집계.

Page 20: [1002 Lab meeting] Using Annotations  in Online Group Chats

IA 사용 집단에서, native 4.30/ non-native 3.30

IA tool and language fluency have influenced perceptions of conversation control.

7DATA ANALYSIS AND RESULTSU

sing

Ann

otat

ions

in

Onl

ine

Gro

up C

hats

2. How Was User Experience Affected by the IA Tool?

1) Quality of Communication (QC)

• non-IA 집단에서, native 4.11/ non-native 2.60

• Native speakers > non-native speakers

- QC ratings (overall mean 3.94) > (over- all mean 3.41)

!

- 커뮤니케이션의 품질을 평가하는 여섯 평가 척도로 집계.

Page 21: [1002 Lab meeting] Using Annotations  in Online Group Chats

• Interview suggest that IA may mitigate the “attenuation blocking”

problem common in group meetings

“I started typing something, and then I wiped it out, because somebody else already raised the issue. ... That's why I like the annotation,… “

-����������� ������������������  빠르게����������� ������������������  지나가는����������� ������������������  discussion����������� ������������������  point에����������� ������������������  따라갈����������� ������������������  만큼����������� ������������������  충분히����������� ������������������  빠르지����������� ������������������  않기����������� ������������������  때문에����������� ������������������  traditional����������� ������������������  chat에����������� ������������������  어려움을����������� ������������������  느낀다.����������� ������������������  ����������� ������������������  

����������� ������������������  ����������� ������������������  IA사용으로,����������� ������������������  언제����������� ������������������  어떤����������� ������������������  포인트에든지����������� ������������������  심지어����������� ������������������  다른����������� ������������������  토픽으로����������� ������������������  넘어갈때도����������� ������������������  아이디어를����������� ������������������  add할����������� ������������������  수����������� ������������������  있다.

7DATA ANALYSIS AND RESULTSU

sing

Ann

otat

ions

in

Onl

ine

Gro

up C

hats

2. How Was User Experience Affected by the IA Tool?

2) Mitigating Attenuation Blocking

(Interviewee 12, IA)

-����������� ������������������  일상적인����������� ������������������  채팅해서����������� ������������������  “Nice����������� ������������������  idea”라는����������� ������������������  코멘트는����������� ������������������  명확하지����������� ������������������  않지만,����������� ������������������  

����������� ������������������  ����������� ������������������  IA����������� ������������������  에����������� ������������������  이런����������� ������������������  코멘트를����������� ������������������  남기면,����������� ������������������  그룹원들에게����������� ������������������  그����������� ������������������  아이디어에����������� ������������������  대한����������� ������������������  더����������� ������������������  많은����������� ������������������  아이디어를����������� ������������������  논의하게����������� ������������������  한다.

Page 22: [1002 Lab meeting] Using Annotations  in Online Group Chats

A design implication from this observation

- 그룹으로 브레인스토밍하기 위한 툴들은 secondary channel을 제공해야한다.

- 빠르게 진행되는 토론에서 천천히 아이디어를 표현하고,

그 아이디어들을 발전시키는데 돕기 위해

7DATA ANALYSIS AND RESULTSU

sing

Ann

otat

ions

in

Onl

ine

Gro

up C

hats

2. How Was User Experience Affected by the IA Tool?

2) Mitigating Attenuation Blocking

Page 23: [1002 Lab meeting] Using Annotations  in Online Group Chats

• IA는 채팅 중에 수집된 아이디어들과 반응들을 통해, 팀원들 사이에서 분산된 인지들을 support해줄 수 있다.

7DATA ANALYSIS AND RESULTSU

sing

Ann

otat

ions

in

Onl

ine

Gro

up C

hats

2. How Was User Experience Affected by the IA Tool?

3) Supporting Distributed Cognition

“I want to comment on that one point, and I felt that was a better place in the annotation than in the chat sequence. Because it’s more of a supporting statement, I wanted to put in like ‘To do:’ what need to be done to get that idea going. And I just didn't feel that it wants itself to be in the discussion, it's more like a side note of ok heads up, we need to do this for this idea.” (Interviewee 10, study 2)

-����������� ������������������  With����������� ������������������  the����������� ������������������  IA����������� ������������������  space,����������� ������������������  it����������� ������������������  is����������� ������������������  easy����������� ������������������  to����������� ������������������  externalize����������� ������������������  one’s����������� ������������������  understanding����������� ������������������  of����������� ������������������  or����������� ������������������  reactions����������� ������������������  to����������� ������������������  the����������� ������������������  conversation,����������� ������������������  ����������� ������������������  

thereby����������� ������������������  sharing����������� ������������������  it����������� ������������������  with����������� ������������������  other����������� ������������������  team����������� ������������������  members.

Page 24: [1002 Lab meeting] Using Annotations  in Online Group Chats

7DATA ANALYSIS AND RESULTSU

sing

Ann

otat

ions

in

Onl

ine

Gro

up C

hats

2. How Was User Experience Affected by the IA Tool?

3) Supporting Distributed Cognition

A design implication from this observation

- 온라인 채팅 툴은 외적인 표현으로 그룹 멤버들간에 공유된 생각들을 조정하는데

도움을 줘야한다.

Page 25: [1002 Lab meeting] Using Annotations  in Online Group Chats

• Although IA did not change performance, it seems to promote perceptions of a better balance

in conversations, which helps to address issues stemming from differences in language fluency.

•Non-IA condition, all groups reported that native speakers assumed leader roles in their multilingual groups

7DATA ANALYSIS AND RESULTSU

sing

Ann

otat

ions

in

Onl

ine

Gro

up C

hats

2. How Was User Experience Affected by the IA Tool?

4) Balancing Conversation Participation

• IA condition, no groups reported having a leader.

reduced the tendency for groups to have dominant individuals controlling the conversation.

Page 26: [1002 Lab meeting] Using Annotations  in Online Group Chats

- 유창한 사람이 덜 유창한 사람이 인풋하는 걸 기다리는데 사로잡을만한 부수적인 활동을 제공해야 한다.

7DATA ANALYSIS AND RESULTSU

sing

Ann

otat

ions

in

Onl

ine

Gro

up C

hats

2. How Was User Experience Affected by the IA Tool?

4) Balancing Conversation Participation

“I was the one who was doing all the annotations. I had a couple of ideas, but I kind of slipped and let other people talk, while I was annotating. And once it slowed down, I added my own ideas.” (Interviewee 13, IA)

• 대화를 지배하는 수다쟁이는 자기가 인풋하는 것들을 균형맞출 수 있는 방법을 발견했다.

말하기전에 잠깐 기다렸고, 그룹위해서 주석을 다는데 시간을 썼다.

A design implication from this observation

Page 27: [1002 Lab meeting] Using Annotations  in Online Group Chats

• the post-task questionnaire included two subconstructs designed to evaluate the cost of using the IA tool.

7DATA ANALYSIS AND RESULTSU

sing

Ann

otat

ions

in

Onl

ine

Gro

up C

hats

3. What Are the Costs of the IA Tool?

• even for users with relatively high cognitive loads (e.g., non-native speakers),

annotations combine well with chatting.

• both constructs were moderately positive, with a mean of 4.0 for IA and 3.93 for standard tool.

Page 28: [1002 Lab meeting] Using Annotations  in Online Group Chats

8CONCLUSIONSU

sing

Ann

otat

ions

in

Onl

ine

Gro

up C

hats

1. adding instant annotations during real-time collaboration is possible

2. An analysis of the use patterns of the IA tool suggested diverse and subtle ways

in which it can assist group communication

3. participants seemed able to multitask well between the main chatting window and the IA side bar

4. as a memory aid in summarizing

5. to interact with others

Page 29: [1002 Lab meeting] Using Annotations  in Online Group Chats

In the future

Usi

ng A

nnot

atio

ns

in O

nlin

e G

roup

Cha

ts

• it will be interesting to extend the use of IA to real-time text chats for larger groups,

where richer and more flexible representations of the secondary channel may be needed.

• Other features that may help with larger groups

: notifications of new annotations as they appear

8CONCLUSIONS

Page 30: [1002 Lab meeting] Using Annotations  in Online Group Chats

1. 온라인 채팅으로 팀플/토론을 진행했던 경험 공유하고, 그 당시 그룹원들과 아이디어를 내거나, decision-making 했던 방법은?

!2. google docs처럼 협업 기능의 서비스 사용 경험의 장단점을 논의해보자.

10DiscussionU

sing

Ann

otat

ions

in

Onl

ine

Gro

up C

hats

Page 31: [1002 Lab meeting] Using Annotations  in Online Group Chats

감사합니다