National level data metrics framework development in Kouth Korea -Iljr Rha

September 17, 2015

Prof. Ilju Rha, Ph. D. & Prof. Cheolil Lim, Prof. Young Hoan Cho & Mina YooSeoul National University, Seoul, Korea

Seoul National University

National Level Data Metrics Framework Devel-opment for Learning Analytics

in South Korea

http://english.moe.go.kr/enMain.do

http://english.sen.go.kr/index.jsp

1

23

4

5

Introduction

Research Question

Methods

Results

Conclusion & Future Plan


*Introduction

http://news.naver.com/main/read.nhn?mode=LSD&mid=sec&sid1=105&oid=138&aid=0001988444 3

economy

business

politics

healthcare

computer sciences

art engineering


*Introduction

http://news.naver.com/main/read.nhn?mode=LSD&mid=sec&sid1=102&oid=020&aid=0002554478 4

BIG DATA in EDUCATION

Seoul National University 5

*Introduction

Digital Textbook

*Introduction of Research

6

7

8

9

10


To students’ learning.

helpengagepromotemotivateencourage …


Q. What should we do ?Learning analytics!


*Introduction of Research


Start-up Mega Planning

http://english.moe.go.kr/enMain.do

http://english.sen.go.kr/index.jsp


*Introduction of Research: 3-year plan

14

1st-yearModeling of Learning

Analytics

plan of collecting learn-ing data

Modeling of learning data analysis

Deduction of the appli-cation plan based on learning data analysis

2nd-yearApplication of Learn-

ing Analytics

Collecting 1st-year modeling based learn-

ing data, analyzing, application, and revi-

sion

Development of mod-els for LOD based

learning analytics ser-vice in the field of ed-

ucation

3rd-yearExpansion of the service based on

learning data

Application of models for LOD based learning analytics service in 2nd

-year

Application and revi-sion of adaptive and personalized learning prescription service


*Introduction of Research-1st year

Research Title : A basic research on learning ana-lytics model and plan of expansion

Research Period : Sep 11, 2014 ~ Jan 31, 2015 (5 months)

Institutions : Seoul National University Director : Prof. Ilju Rha, Ph.D. Dept. of Education, College of Education, Seoul National Uni-versity Co-researchers : Prof. Cheolil Lim, Ph.D.,

Prof. Younghoan Cho, Ph.D. Assistant researchers : 6 graduate students 15

16Verbert et al. (2012). p. 137. (Figure 3. Learner action model)


Q. What are plans forcollecting learning data?

*Research Question


*Research Methods

Literature Review Case studies of learning analytics Experts advisory councils and seminars


*Research Methods

Forums with specialists Visiting schools and focus group interviews with

teachers Interviews with teachers for review of the application

plans and learning analytics Research group meetings


Derivation from learners’ activities

Conformity to the International Standard

Easy communication among researchers

Meaningful units of activities in the field of education

*Research Question - The plan of collecting learning data

1

2

3

4

What are plans for collecting learning data?


* The plan of collecting learning data (1)

• The development of learning activities metrics Review of IMS learning activities metrics

• Utilization of applicable metadata

- Composition of learners’ activities and the data ob-ject• Focus on the elements of

learning design- tasks, resources, support



• The development of learning activities metrics– Review of the classification of teaching-learning activi-

ties• Ascher(1976)

• Classification of teaching-learning activities in F2F environment

• Horton(2006)• Linkage between activities

and media in E-learning environment

• Watkinson(2005)• Classification of E-learning

activities into 75 elements• Lim, Lim, and Kim(2008)

• Classification of digital text books into 49 teaching-learning activities

Supplement and modification • tools, basic activities, classifi-

cation of multiple activities• Tools : Management of learning

goals• Basic activities : Writing, mind

mapping• Multiple activities : Hands-on

learning



• The development of learning activities metrics– Review of models of instruction and lesson plans for uti-

lizing digital textbooks

• Kim et al. (2011). Class-room centered teaching-learning activities in 21st century.

• Rho et al. (2013). A re-search on models of in-struction for utilizing digi-tal textbooks.

Supplement and modification • Basic activities : writing,

mind mapping, speaking



• The development of learning activities metrics– Expert reviews and modification

Classification List of experts

Educational Tech-nology

Prof. Insung Jung, Ph.D., International Christian University, Japan

Prof. Il-Hyun Jo, Ph.D., Ehwa Womans Univer-sity, Korea

Prof. Yeonwook Im, Ph.D., Hanyang Cyber uni-versity, Korea

Educational Psy-chology

Prof. Jongho Shin, Ph.D., Seoul National Uni-versity, Korea

Computer Engineer-ing

Prof. Jang-Mook Kang, Ph.D., Korea University, Korea

Education Field Principal Manjong Yang, Jeongmok Elementary SchoolSupplement and modification

• Tools : Access management• Basic Activities : Download of writing resources, citation of re-

searches


* Process - The development of learning activities metrics

• The development of learning activities metrics– Classifying every learning activities for measurable units– Taxonomy for utilizing units of activities composing

learning behaviors as the analyzed data

The development of learning activities

metrics

IMS learning activ-ities metrics

Models of instructions and lesson plans for uti-lizing digital text books

Classifying teach-ing-learning types in domestic and in-

ternational level



Retrieved from http://www.imsglobal.org/IMSLearningAnalyticsW-P.pdf

• IMS Metrics profiles



The Differ-ence

The Ideal Vi-sion

Needs

Negotiation

Mega“National Level”

Macro“K-12 Education”

Micro“Institutional and

Individual Metrics”

Policy Mak-ers

(MOE, Province, School & Support

Institution)

Practitioners(Teachers, Re-

searchers, Technicians)

Process

Inputs

Integration“NDM framework”


* Results - The development of learning activities metrics

Proposed National Learning Metrics

Framework

TOOLS

Goal Manage-ment Scheduling Media Social Annotation Access

Management

Reading Lectures Writing Discussions Messaging Quiz

Speaking Projects Research Assessment Mind Map Gaming

COMBINED

Tutoring Collaboration Field StudyHomework

• Annotations• Page/block use• Media use• Lookups

• Frameset use• Scrub marks• View time• Weblink refs

• Assoc context• Goal setting• Subordinate

goal setting• Event pattern• Frequency

• Assoc context• Event patterns• Event profile• Time utilization

ACTIVITY

BASIC

ACTIVITY

• Media type• Frameset use• Scrub marks• View time• Usage context

• Connections• Assoc context• Message profile• frequency

• Highlights• Notes• Marks• Tags

• Assoc context• Input • Contents• Attachments

• Post mark• Frequency• Participation• Collaboration

• Assoc context• Outbound pool• Inbound pool• Attachments

• Scores• Attempts• Remediations• Assoc refs

• Questioning• Answering• Presentation• Communication


• Topics• Assoc context• Frequency• Feedback

• Connections• Assoc context• Message profile• Frequency

• Deliverables• Structure• Milestone perf• Group profile• Patterns

• Searches• Patterns• Citations• Topics

• Scores• Patterns(item)• Time utilization• Attempts• Completion

• Drawing• Frequency• Participation• Collaboration

• Progress• Cognition• Attempts• Hints• Collaboration

· Clicking (navigation, operation) · Swiping (navigation, close up, resizing) · Typing (note taking, memo, searching, communication) · Pen writing (highlights, memo, using color pens)

PERFORMANCECONTEXT

• Activity usage time on task

• Session time• Last access• Activity affinity• Content affinity• Task patterns• Correlations

• Institution• Course/Section• Learner profile• Course context• Path/Sequence• Usage context

• Grades• Progress• Rubrics

‐ Course goals‐ Topic objects‐ Qualitative

evaluation‐ Quantitative

scores• Patterns• Correlations

Learning Activity Metrics

INPUT PROCESS OUTPUT

• Time Stamp (Log in & Logout)• Duration• Frequency

LEVEL OF MET-RICS UTILIZATION · Government · Policy maker · School district · School · Teacher

ENGAGEMENT

• Assoc context• Frequency• Participation• Collaboration

29

TOOLS


Management



COMBINED







ACTIVITY

BASIC

ACTIVITY


















PERFORMANCECONTEXT












ENGAGEMENT


30

TOOLS


Management



COMBINED







ACTIVITY

BASIC

ACTIVITY


















PERFORMANCECONTEXT












ENGAGEMENT


31

TOOLS


Management



COMBINED







ACTIVITY

BASIC

ACTIVITY


















PERFORMANCECONTEXT












ENGAGEMENT


32

TOOLS


Management



COMBINED







ACTIVITY

BASIC

ACTIVITY


















PERFORMANCECONTEXT












ENGAGEMENT


Engagement Elements Sub-elements

Basic events

Clicking (C) C1. Navigation C2. Operation

Typing (W) W1. Note taking W2. MemoW3. Searching W4. Communication

Swiping (S) S1. Navigation S2. Close up S3. Resizing

Pen writing (P) P1. Highlights P2. Memo P3. Using Color pens

Time (T) T1. Timestamp T2. Duration T3. Interval

Data input &output Login-Logout, Installation, Download, Save, Print, Record, Capture, Bookmark

Elements of Engagement

33

TOOLS


Management



COMBINED







ACTIVITY

BASIC

ACTIVITY


















PERFORMANCECONTEXT












ENGAGEMENT


34


* Conclusion

The proposed national data metrics • Tentative attribute in theoretical level

• Systematic data collection for learning analytics in an as-pect of educational utilization of big data

• Learning analytics in national level

• Learning activities are the basis of learning analytics

• Used literature reviews and needs analysis methodology

• Practical implementation is needed to investigate its fea-sibility


* Future Plan

1st-yearModeling of Learning

Analytics

plan of collecting learn-ing data

Modeling of learning data analysis

Deduction of the appli-cation plan based on learning data analysis

2nd-yearApplication of Learn-

ing Analytics

Collecting 1st-year modeling based learn-

ing data, analyzing, application, and revi-

sion

Development of mod-els for LOD based

learning analytics ser-vice in the field of ed-

ucation

3rd-yearExpansion of the service based on

learning data

Application of models for LOD based learning analytics service in 2nd

-year

Application and revi-sion of adaptive and personalized learning prescription service


Q. What should we do?Learning analytics!

Q.. What kinds of data?Q… How to collect data?Q…. How to analyze data?Q….. Is it the most optimal?

* Future Plan


Questions?


Thank You!

Prof. Ilju Rha, Ph.D. [email protected]

Prof. Cheolil Lim, Prof. Young Hoan Cho & Mina Yoo

Dept. of Education, College of Edu-cation


mailto:[email protected]