Scaling up Adaptive Education Systems

Sergey Sosnovsky

Instructional

Content

Authoring for e-Learning M

et

ad

at

a

Inst

ructi

onal

Cont

ent

Authoring for Adaptive e-Learning

Instructional

Content

Authoring for Adaptive e-Learning

as It Should Be

Adaptive Education Systems

Strong methodological

background

Strong research community

Strong experimental

results

Strong funding supportThe

technology still stays

mainly in the lab

?

Adaptive Education Hypermedia

~700.000 USD funding:- US NSF grant #7525- US NSF grant #0447083- US NSF grant #DUE-0633494

Sosnovsky, S., & Brusilovsky, P. (2015). Evaluation of topic-based adaptation and user modeling in QuizGuide. User Modeling and User-Adapted Interaction, 25(4), (371–424).

Example 1

Adaptive Tutorial Feedback

~ 700.000 EUR funding:- ATuF: Adaptive Tutoring Feedback (Grants ME 1136/8-1 and NA 738/10-1)

Narciss, S., Sosnovsky S., Schnaubert, L., Andrès, E., Eichelmann, A., Goguadze, G., & Melis, E. (2014). Exploring feedback and student characteristics relevant for personalizing feedback strategies. Computers and Education, 71, (56-76).

Example 2

Adaptive Tutorial Feedback in a VR

~ 100.000 EUR funding:SafeChild: intelligent traffic safety training for children in virtual reality (Grant 01IS12050)

Gu, Y., Sosnovsky, S., & Ullrich C. (2015). SafeChild: an intelligent virtual reality environment for training pedestrian safety skills. In G. Conole, T. Klobucar, C. Rensing, J. Konert, É. Lavoué (Eds.) Proceedings of EC-TEL'2015: 10th European Conference on Technology Enhanced Learning (pp. 141-154). Berlin/Heidelberg, Germany: Springer.

Example 3

Adaptive Content Sequencing and Course Generation

~ 12.000.000 EUR

funding:

Sosnovsky, S., Dietrich, M., Andrès, E., Goguadze, G., Winterstein, S., Libbrecht, P., Siekmann, J., & Melis, E. (2014). Math-Bridge: Bridging the gaps in European remedial mathematics with technology-enhanced learning. In T. Wassong, D. Frischemeier, P. R. Fischer, R. Hochmuth, & P. Bender (Eds.), Mit Werkzeugen Mathematik und Stochastik lernen – Using Tools for Learning Mathematics and Statistics (pp. 437-451). Berlin/Heidelberg, Germany: Springer.

Example 4

Architecture of an AES

InstructionalContent

Interaction

UserModel

0..1..1..0..1..1

..AdaptationModel

Adaptation

M e t a d a t a

DomainModel

The Scale of the Problem

InteractionAdaptation

Two Principal Approaches

Content-based User-based

abc, def, ghi, jkl, mno, pqr, stu, vwx, yz

Interaction

abc, def,…

abc, def, ghi,…

Adaptation

abc, def,…

Interaction

Adaptation

Structured Instructional Content on the Web

• Plenty of high-quality, domain-oriented, well-formatted educational resources• Their content has been chosen, structured and formatted for the purpose of

explaining the domain knowledge by the experts in the domain to the novices• The structure of the domain knowledge has been used as a mean for organizing

these resources in a consistent and predictable manner• If extracted, this structure represents the model of the resource and the model of the

domain as the author understands it• This model may be incomplete, subjective, coarse-grained, but it might provide just

enough information to link to other resources and models

10

Tutorials

Manuals

Digital LibrariesDictionaries

Encyclopedias

Table of Contents

Links between Pages

Headers of Sections

Content Formatting Index

Textbooks

11

Example: @Connexions

12

Example: @Wikipedia

13

Example: @Wolfram MathWorld

14

Linking Textbooks to Ontologies• Topic-based model of an HTML-based Java

textbook automatically extracted and mapped to a central ontology already linked to a set of Java exercises

• Mapping serves as a bridge to jointly interpret learner’s reading and exercise attempts in terms of ontology and adapt access to textbook pages accordingly

Project 1

15

Evaluation: Three Versions of the System

Adaptive recommendation of

Open-corpus content

Adaptive recommendation of Closed-corpus content

--------------------------------------Based on manual indexing of the textbook topics with the

concepts of the ontology

The original textbook instead of

recommendation

Project 1

Evaluation Results

1. Learning difficultlearning material

2. Learning conceptual material

3. On all comparisons, no sig. difference between the experimental and closed-corpus conditions

Project 1

0.00

0.50

1.00

1.50

2.00

p = 0.043 Experimentalvs.

Textbook

0.000.501.001.502.002.503.00

Conceptual Easy Material Conceptual Difficult Material

Experimentalvs.

Textbook

p = 0.089 p = 0.023

0.00

0.20

0.40

0.60

0.80

17

Linking Textbooks to Textbooks

• Several LDA-based techniques are used to interlink sections from a set of HTML-based textbooks in a domain

• A manual mapping by experts is used as a golden standard

Linking

Linking

Project 2

Evaluation ResultsProject 2

Standardab-weichung

Écart type

development of an online intelligent educational solution to provide interlingual students with on-demand access to relevant learning

material in their mother tongue

19

• 3 PDF-based textbook on probability theory and statistics in EN/DE/FR:

Linking Textbooks across Languages Project 3

20

Multilingual Ontology of Statistics

• Source of concepts names:– ISI Glossary

(http://isi.cbs.nl/glossary/)– ~3500 terms in ~25 languages– Synonyms in all languages

• Source of relations and explanations:– dbPedia (http://dbpedia.org )– „related“ concepts– concept definitions– links to Wikipedia pages

standard scorez-score

note réduiterésultat typenote typique

ponctuation standardscore centré réduit

z-Scorestandardisierter Score

Standardpunktwertung

In statistics, the standard score is the (signed) number of standard deviations an observation or…

standard score

En probabilités et statistiques, une variable centrée réduite est une variable aléatoire…

Unter Standardisierung oder z-Transformation versteht man in der mathematischen Statistik eine …

Statistical Ratios

http://en.wikipedia.org/wiki/Standard_score

Type

FR

DE

EN

t-statistics Type

……

Project 3

Step 1: ToC-based structure

21

Step 1: PDF Textbook structure

extraction

Chapter1 Section1.1 Subsection1.1.1 Subsection1.1.2 … Section1.2 Subsection1.2.2…Layers of information:

1. Formatting layer (font attributes: size, family, boldness, indentations, etc.)

2. Structural layer (structural elements: ToC, index, breadcrumbs, important text, regular pages, links between pages etc.)

3. Semantic layer (topic-based hierarchy, keyword-based links to external models, semantic links between pages, types of learning objects within a book)

Project 3

Step 2: index

22

Chapter1 Section1.1 Subsection1.1.1 Subsection1.1.2 … Section1.2 Subsection1.2.2…

Step2:Textbook index extraction

term -> page#term -> page#term -> page#term -> page#term -> page#….

Project 3

Step 3: linking to ontology

23

Chapter1 Section1.1 Subsection1.1.1 Subsection1.1.2 … Section1.2 Subsection1.2.2…

term -> page#term -> page#term -> page#term -> page#term -> page#….

Statisticsontology

Step 3:Semantic Annotation

Project 3

Step 4: putting it all together

24

Statisticsontology

Other textbooks

Self-assessment generation

Semantic model of the textbook

Project 3

25

The resultsProject 3

26

Relevant Reading in One’s Mother ToungeProject 3

27

Linking across automatically extracted textbook models enables:

Integration and Cross-reasoning

Discovery and Retrieval

Navigation and Recommendation

Restructuring and Enrichment

Connection to External LinkedData Repositories

28

What’s next?

• Adding advanced NLP techniques• Integration of multiple textbooks• Combination with social/usage data

Improving the

modeling quality

• Detection of individual learning objects and their types• Cross-domain linking• Formalization of trustworthiness, complexity, quality of

information,…

Adding new features

• an intelligent library integrating textbooks across multiple relevant domains and providing a range of intelligent services

• a platform providing industrial workers with intelligent access to manuals

• a service on a publishers’ website recommending textbooks and their parts

Building new applications

Challenges of Modern Education

• Explosion of Knowledge– Technologies are evolving and becoming more complex– Students have to learn more than ever before– Students have to prepare for professions that do not yet exist– How to keep it effective and scalable?

• Explosion of Content– Internet is an unlimited source of educational content– Textbooks, tutorials, tests, simulations, exercises, examples, etc.– How can a student benefit from this, how can she find what’s right

• Explosion of Students– More people go to colleges every year– Many current workers will have to retrain during their careers– Many students need to receive education anytime, anyplace

How will our classrooms and teachers keep up?

31

Pre-test vs. Post-test (Easy Material) Pre-test vs. Post-test (Difficult Material)

Open-Corpus

Closed-Corpus

Text-book

General Learning Effect

0.00

2.00

4.00

6.00

8.00

0.00

2.00

4.00

6.00

8.00

p = 0.089

• H1: Scrorepre-test< Scorepost-test across all groups and conditions

Project 1

32

Knowledge Gain (Easy Material) Knowledge Gain (Difficult Material)

Open-Corpusvs.

Closed-Corpus

Open-Corpus vs.

Textbook

Effect on Learning Difficult Material

p = 0.043

0.000.501.001.502.002.50

0.00

0.50

1.00

1.50

2.00

0.000.501.001.502.002.50

0.00

0.50

1.00

1.50

2.00

• H2: KGopen-corpus = KGclosed-corpus for easy material

• H3: KGopen-corpus = KGclosed-corpus for difficult material

• H4: KGopen-corpus > KGtextbook for easy material

• H5: KGopen-corpus > Kgtextbook for difficult material

Project 1

33

What is the output of the following code segment? int a = 3 + 3;int b = 2 + 2;if (a != b) System.out.println(“ Not equal ”);if (a == b) System.out.println(“ Equal ”);

0.000.501.001.502.002.503.00

Conceptual CG (Easy Material) Conceptual CG (Difficult Material)

Open-Corpus vs.

Textbook

Effect on Learning Conceptual Material

p = 0.089 p = 0.023

There are two values of type boolean: ______________ and ______________

0.00

0.20

0.40

0.60

0.80

• H6,7: KGopen-corpus > KGtextbook for conceptual learning material ≈

Project 1

Collaborative filtering

Semantic annotation