ABA RoundtableMay 2014

IN THE BEGINNING, . . . .

• There was nothing.

OVERALL EXAM STRATEGY

• A MOVE TO CONSCIOUSNESS

SOME EXAMPLES

• Format

• Specifications

• Purpose

• Drafting guidelines

STRATEGY FOR EXAM DATABASE

• Make maximum use of available exam information each year

TACTICAL EXAMPLES

• We can now generate an exam in final format from the database

• This was humongous for us – we had previously used Word documents in preparing exam content for 21 specialty areas

• It reduced the work for our volunteers

MORE TACTICAL EXAMPLES

• Questions can be cataloged by subject and a unique ID number

• Questions can be tied to a specific provision in the exam specifications

• We can track who drafted, graded or reviewed• We have authority for the correct answer• We can determine if the question needs to be

revised• We can revise a model answer if needed

CONTINUED

• We can track performance on multiple choice and essay questions

• We can determine if a question is invalid and adjust scores accordingly in the database

• We generate a detailed report on multiple choice performance

• We can easily use different combinations of questions to keep an exam from being too predictable

CONTINUED

• We can better quantify the coverage of the exam so that topics were covered in the proper proportion to their importance in the specialty area

• We can determine how examinees performed by topic

• We can much more easily track changes to questions and retire items which are no longer valid