Upload
barbara-chandler
View
292
Download
0
Embed Size (px)
Citation preview
國外藥學教育成效評量
林香汶 Hsiang-Wen (Margaret) Lin, Ph.D.
中國醫藥大學藥學系暨碩博士研究所、中國醫藥大學附設醫院藥劑部
Behavior learning objectives After completing this lecture, you will be
able to: Appreciate Bloom taxonomy Compare and contrast common educational
methods used in US pharmacy education Be familiar with the concept of assessment Compare and contrast common assessment
methods used in US pharmacy education
Bloom Taxonomy Knowledge: arrange, define, duplicate, label, list,
memorize, name, order, recognize, relate, recall, repeat, reproduce state.
Comprehension: classify, describe, discuss, explain, express, identify, indicate, locate, recognize, report, restate, review, select, translate,
Application: apply, choose, demonstrate, dramatize, employ, illustrate, interpret, operate, practice, schedule, sketch, solve, use, write.
Analysis: analyze, appraise, calculate, categorize, compare, contrast, criticize, differentiate, discriminate, distinguish, examine, experiment, question, test.
Synthesis: arrange, assemble, collect, compose, construct, create, design, develop, formulate, manage, organize, plan, prepare, propose, set up, write.
Evaluation: appraise, argue, assess, attach, choose compare, defend estimate, judge, predict, rate, core, select, support, value, evaluate.
Benjamin Bloom, 1956
Common educational methods used in US pharmacy education
Lecture Lecture-based cases Case-based lectures Problem-based learning Journal Club/ Paper critique Learning by doing (i.e., presentations) Experiential training Lab/Recitation More…
What are better reasons for assessment?
To improve courses To improve academic programs To improve teaching To enhance student learning
What is assessment?
Assessment the process of documenting, usually in
measurable terms, i.e., knowledge, skills, attitudes and beliefs
a component of any instructional system
Goals of assessment Provide direction and motivation for future learning, including knowledge,skills, and professionalism Protect the public by upholding high professional standards and screeningout trainees who are incompetent Meet public expectations of self-regulation Choose among applicants for advanced training
What to assess Habits of mind and behavior Acquisition and application of knowledge and skills Communication Professionalism Clinical reasoning and judgment in uncertain situations Teamwork Practice-based learning and improvement Systems-based practice
Principles of assessment (I)
Epstein RM, 2007
Principles of assessment (II) How to assess
Use multiple methods and a variety of environments and contexts to capture different aspects of performance
Organize assessments into repeated, ongoing, contextual, and developmental programs
Balance the use of complex, ambiguous real-life situations requiring reasoning and judgment with structured, simplified, and focused assessments of knowledge, skills, and behaviors
Include directly observed behavior Use experts to test expert judgment Use pass–fail standards that reflect appropriate developmental levels Provide timely feedback and mentoring
Epstein RM, 2007
Principles of assessment (III) Cautions
Be aware of the unintended effects of testing Avoid punishing expert examinees who use shortcuts Do not assume that quantitative data are more reliable, valid, or useful t
han qualitative data
Epstein RM, 2007
Type of assessment methods
The most important distinctions are: (1) formative vs. summative(2) objective vs. subjective(3) referencing (criterion-referenced, norm-
referenced)(4) informal and formal (5) Whether it is practical, valid and/or reliable
Common assessment methods (I) Written exercises
Multiple-choice questions single-best-answer extended matching format
Key-feature and script-concordance questions Short-answer questions Structured essays
Assessments by supervising clinicians Global ratings with comments at end of rotation Structured direct observation with checklists for ratings
(e.g., mini–clinical-evaluation exercise or video review) Oral examinations
Epstein RM, 2007
Common assessment methods (II) Clinical simulations
Standardized patients and objective structured clinical examinations (OSCEs)
Incognito standardized patients High-technology simulations
Multisource (“360-degree”) assessments Peer assessments Patient assessments Instructor assessments Self-assessments Portfolios
Epstein RM, 2007
Teaching triangulation
Student performance
Peer assessment Self assessment
Instructor assessment
Not only one answer…No right or wrong answer…
Objective structured clinical examinations (OSCEs)
a performance-based tool comprised of a series of stations through which
all examinees rotate on a timed basis. In each station, the examinee is faced with a
simulated task or problem the examinee is required to perform specific functions to
complete the task or address the problem each examinee in an interactive station is observed and
assessed by a trained examiner using a standardized marking key Austin Z, 2003
Type of OSCE stations Interactive: with
“standardized patients": actors who have been specially trained to portray patients with specific medical conditions or drug-related problems
“standardized clients”: actors or other health professionals who have been specially trained to portray allied health professionals in an interdisciplinary health care context
Non-interactive: written responses to tasks or problems and involve no direct
observation and assessment Austin Z, 2003
Ideal subject content for OSCEs
Communication and interpersonal skills Ethical and professional judgment Complex ethical problem identification and
resolution skills
Austin Z, 2003
Features of OSCEs
Both the process and the product are tested The scoring using both analytical and global rating
is objective Questions being asked in response stations are
always objective Simulations can be used for acute cases and there
is scope for immediate feedback Patient variability and examiner variability are
eliminatedAustin Z, 2003
Advantages of OSCEs
Covering a broad range of clinical skills much wider than a conventional examination
Integrating the teaching and evaluation, and variety maintains student's interest
Increasing faculty-student interaction Testing a large number of students within a
relatively short time
Austin Z, 2003
Disadvantages of OSCEs High costs Time consuming to develop Have difficulties associated with developing and
administering Risk of observer fatigue if the observer has to record the
performance of several candidates on lengthy check lists All stations must invariably demand only equal time Breaking clinical skills into individual competencies is
artificial and not meaningful
Austin Z, 2003
Assessment Instruments used in OSCEs
Need both analytical and global rating scales Analytical checklist
a series of performance-based observations Including specific, observable, and measurable tasks e.g., include asking a specific question related to allergies,
or providing specific information to the patient regarding side effects of a medication.
Global assessment capture the candidate’s performance in important skills su
ch as communication and integrative problem solving
Austin Z, 2003
Example of global assessment
Austin Z, 2003
Performance Reports of OSCEs
Austin Z, 2003
OSCEs related lectures in 2008 AACP conference
Section: Performance-based teaching, learning and assessmentChallenges and Opportunities in Effectively imple
menting performance-based AssessmentOSCEology 101: overview of OSCEHow to develop and implement OSCEs in your sch
ool
Development of an OSCE Blueprinting Station development Case writing Case review and validation Standard setting Piloting Assembly of OSCE
Austin Z, 2008 AACP handout
Blueprinting
Process of defining content domain and process skills to be assessed
Based upon outcomes, expectations, or standards Level: defines complexity of performance expected Range: describes scope of performance expected
Austin Z, 2008 AACP handout
Levels and ranges
Simple Patient
Simple Problem
Complex Patient
Simple Problem
Simple Patient
Complex Problem
Complex Patient
Complex Problem
Austin Z, 2008 AACP handout
Defining levels and ranges
Attributes of “complex patient” include: Psycho-social issues AffectLanguage barrier Physical abilities Cultural Issues Financial Issues
Austin Z, 2008 AACP handout
Defining levels and ranges
Attributes of “complex problem” include: Multiple disease states Multiple DRPsUnusual/rare medical conditions Unusual/rare medications
Austin Z, 2008 AACP handout
Levels and ranges Patient has a cough requiring self-medication and has no allergies, medical conditions or complicating factors
Patient with a language barrier has a cough requiring self-medication and has no allergies, or medical conditions
Patient has a cough and is taking ACE-inhibitors and a statin but also has a low-grade fever and muscle aches/pains
Patient with multiple disease states unable to swallow oral medications is experiencing a cough
Austin Z, 2008 AACP handout
“The minimally competent student”
A factious concept, but one based on reality May be an amalgam of various students Able to meet ALL performance standards/
expectations but at the most basic level
Austin Z, 2008 AACP handout
Reflections
Is it a good idea to go for the OSCEs in Taiwan? As for high stakes vs. low stakes assessment ? As for summative vs. formative assessment ?
Based on the pharmacy education environment in Taiwan, are we ready to get into this kind of performance-based assessments?
What kinds of education reform are needed in Taiwan to facilitate pharmacy students to get ready to be a practitioners?
… More
Assignment
To work individually or work as a group to appraise the assessment methods used in the following two articles
Am. J. Pharm. Educ 2006; 70(5) Article 117
JAMA 2007;298(9):1010-1022
References Epstein RM. Assessment in Medical Education, N Engl J Med 2007;356:387-96. Cisnerosa RM, Salisbury-Glennonb JD, Anderson-Harpera HM. Status of Problem-Based Lea
rning Research in Pharmacy Education: A Call for Future Research. Am. J. Pharm. Educ., 2000. 66, 19–26
Austin Z, Gregory P, Tabak D. Simulated Patients vs. Standardized Patients in Objective Structured Clinical Examinations. Am. J. Pharm. Educ 2006; 70 (5) Article 119.
Austin Z,O'Byrne C, Pugsley J, Munoz LQ. Development and Validation Processes for an Objective Structured Clinical Examination (OSCE) for Entry-to-Practice Certification in Pharmacy: The Canadian Experience. Am. J. Pharm. Educ., 2003; 67 (3) Article 76
Lin HW, Pickard AS, Mahady GB, Popovich, NG. The development of a measure to assess pharmacists’ knowledge of herbal and dietary supplements. Am. J. Pharm. Educ., 2008; 72(3): Article 59.
Windish DM, Huot SJ. Green ML. Medicine Residents’ Understanding of the Biostatistics and Results in the Medical Literature. JAMA. 2007;298(9):1010-1022.
Austin Z, Marini A, MacLeod Glover N, Tabak D,Peer-Mentoring Workshop for Continuous Professional Development. Am. J. Pharm. Educ 2006; 70(5) Article 117
Garavalia LS, Marken PA, Sommi RW. Selecting Appropriate Assessment Methods: Asking the Right Questions. Am. J. Pharm. Educ., 2002. 66, 108-112
Mackellar A,Ashcroft DM, Bell et al. Criteria for the Assessment of Pharmacy Students’ Communication Skills With Patients. Am. J. Pharm. Educ. 2007; 71 (3) Article 50.
Ananthakrishnan N. Objective structured clinical/practical examination (OSCE/OSPE). J Postgrad Med 1993;39:82-4
Thank your for your attention
Questions???