Upload
baldwin-hutchinson
View
215
Download
0
Embed Size (px)
Citation preview
Aplication of CAS in automatic assessment of
math skills
Przemysław Kajetanowicz ([email protected])
Jędrzej Wierzejewski ([email protected])
Wrocław University of Technology
Agenda
• Knowledge assesment in math instruction• Automatic assessment in algebra E-
course: history, functionality, implementation, results and students’ feedback
• How automatic tests work• Recent developments:
Java+Mathematica-based tests• Future
Assessment of progress in math
• Measuring progress in math = measuring mastery in problem solving.
• Grading of math problems– Grading procedure usually reflects typical
solution structure– Common though not always applicable: partial
credit for partial solution• Assessment tools in computer-aided
learning systems– LMS-dependent – Stand-alone
E-course in algebra - overview
• Content structure– Lecture notes– Interactive exercises (Java-driven) – over 120
problem types supported– Practice tests (Java-driven)– Graded exams (Java-driven)
• Functionality of assessment tools– Random generation of data– Controlling the difficulty level at design time– Flexible grading (partial credit)– Step-by-step solution presentation– Completeness check– Initial correctness check
Lecture notes (1)
Studying…
… and exploring math
Lecture notes (2)
Studying…
…and self-testing
Exercises
Exercise… ...and solution on demand
Graphing problems and related tools
Functionality of tests (1)
Functionality of tests (2)
Dedicated tools for Gauss elimination
Functionality of tests (3)
Correct solution immediately available
Completeness check
A student can go back to test as many times as he/she wishes.
Initial correctness check
Chances to make corrections (enthusiastically greeted by students) – teachers decides on # of chances
E-course - implementation
• Spring 2005 - 55 students• Fall 2005 – 400 students• Fall 2006 – almost 1000 students• Hybrid instruction
– Classroom meetings (3 or 4 hours weekly)– (Fall 2006) grading based on
• 5 online e-exams (play the role of homework)• Final in-lab e-exam (proctored)• In-class activity (not much credit though)
Administrative grading system
• Available credit– 5 online e-exams: 5 problems, 1 hour, 25 points available;
best four exams count– In-class activity – 4 points available– In-lab final exam: 8 problems, 90 minutes, 40 points
available• Grading procedure based on student’s systematic
work– A minimum of 20 on the final necessary (but not
sufficient) to pass– 20% of online e-exams total score added to final
(available maximum = 20)– In-class credit added (maximum = 4)– Necessary total to pass = 31
Grading system in practice
Grading examples
19 2440
31
20 6
1520
0
10
20
30
40
50
60
To
tal s
core
(m
axim
um
64)
In-class credit
Online exams
Final exam
In-class credit 0 0 0 4
Online exams 20 6 15 20
Final exam 19 24 40 31
Fail Fail Very good Very goodRequired minimum
on final
Required total to pass
Fall 2005 – Final grades
3%
17%
10%
19%21%
18%
11%
0%
5%
10%
15%
20%
25%
NOTTAKEN
2 (= FAIL) 3 3+ 4 4+ 5 (VERYGOOD)
Grade
Rel
ativ
e fr
equ
ency
of
stu
den
ts
E-course results vs. traditional course (data after make-up exams)
GradePercentageof students
Fail 48.72%
Satisfactory 6.41%
Satisfactory + 17.95%
Good 3.85%
Good + 8.97%
Very good 14.10%
E-course
GradePercentageof students
Fail 84.11%
Pass 15.89%
Traditional course
(Fall 2005 results)
Question 1To what extent did you find the new form of the course (in particular, automatic tests and exams) helpful in your mastering the course material (as confronted with traditional way of learning)?
29%
30%
9%6%
26%
1
2
3
4
5
6
Question 2How willing would you be to sign up for other math courses, were they offered in a similar form?
6%
9%
11%3%
71%
1
2
3
4
5
6
Question 6How fairly, in your opinion, was your knowledge assessed by the system of automatic exams?
28%
25%
9%
9%
26%3%
1
2
3
4
5
6
Note. The exam questions were of similar type to those given on traditional exams. Presently, no statistical comparison is possible between the new and the traditional form in terms of how students judge the fairness (no survey was given to „traditionally assessed students”. The majority of students (74%) judged the system as fair (grades 4 to 6).
Students’ comments
The course is a new appealing form of study to me. It is stress-free. (…) Giving tests and exams that way implies that the grading system is fair. The course is a great thing, and studying algebra that way is truly easy. Thank you !
Student’s comments (2)
At the beginning I was terrified at the perspective of taking a course that was delivered in that way, but now I would like to wholeheartedly thank the instructors for the opportunity of being a part of this “experiment”. I can definitely say that thanks to this course I understood things that had been all Greek to me before (...) I truly believe that more courses should be taught that way. I wish to thank again for the opportunity to have been in this course.
Inside a test (Java-driven only)
• Hard-coded– Problem data generation– Solution presentation– Initial grading– Completeness check, initial correctness check– Database-related operations
• Parameter-driven– Difficulty level, problem sub-type– Scores for individual problems– Number of problems– Time assigned for solution– Specific test behaviour (completeness warning, # of tries
to make corrections etc.)– Training vs. administrative purpose of test
Java+webMathematica (1)
• Java:– Formulates problem
(generates problem data)– Accepts student’s
entry(ies)– Sends data and solution to
webMathematica• webMathematica:
– Compares solution with correct result
– Sends outcome back to Java
– Generates and sends back other elements (e.g. graphs or expressions needed for solution presentation)
Java+webMathematica (solution presentation)
Java+webMathematica (2)
Java+webMathematica (2 - solution)
Java+webMathematica (3)
Java+webMathematica (3 - solution)
• Xml:– Holds problem formulation (parametrized, so many
problems of a common type are supported)– Holds solution (parametrized)– Holds problem data sets
• Java:– Reads xml and behaves correspondingly
• Mathematica:– Encodes formulation and solution in xml, provides
necessary graphics etc.– Generates and encodes data sets in xml
Java+xml+Mathematica – version 1
Space-consuming (especially if graphics involved)
• Xml:– Holds problem formulation (parametrized, so many
problems of a common type are supported)– Holds solution (parametrized)
• Java:– Reads xml– Generates data sets
• Mathematica:– Encodes formulation and solution in xml, provides
necessary graphics etc.– NO data sets generated by Mathematica
Java+xml+Mathematica – version 2
Quick and space-saving
Java+xml+Mathematica (formulation)
<zestaw podtyp = "Y" >
<dK>8</dK>
<dX>2</dX>
<dO>OY</dO>
<dP>8</dP>
<dA>-1/8</dA>
<dB>2</dB>
<o><![CDATA[%M y %N = ]]></o>
<o><![CDATA[ %M x + %N ]]></o>
<p>-1/8</p>
<p>2</p>
</zestaw>
Data sets - xml (including correct result)
Formulation - xml
Java+xml+Mathematica (solution)
Solution (one of steps)
Java+xml+Mathematica (v. 2)
Data sets generation - Java
Formulation & solution - xml
Java+xml+Mathematica (v. 2) - solution
Solution (parametrized) - xml
Specific values - Java
Pros and cons of automatic assessment• Automatic assessment – main highlights
– Proves far more effective (see the following slides).– Gives a student more opportunity to demonstrate skills.
• E-course: 25 problems on online e-exams + 8 on the final• Traditional: 2 x 3 problems on mid-term „paper” exams + 5
on the final– Motivates a student to systematic work. – Every student gets different problems.– Saves time.
• Automatic assessment – downsides– Certain problem types (e.g. „prove that…”) cannot be
supported („as yet” - as dr. Wierzejewski says…).– Some students focus on solution algorithms rather than
on math concepts and methods.
Forthcoming future
• E-course in Calculus • Remedial course in secondary-school math• Continuation of teaching the e-course in algebra online• Completion of e-course in linear algebra
Thank you for your attention