Doing Assessment Like A Champ! (Plus, Tips for Raising Baby)
Michael Gress Interim Director of Institutional Effectiveness LRC 120, 4275
Assessing Like a Champ: No Human Sacrifices, Please!
A Champ Knows the Basic Steps
The Assessment Cycle: It’s Not That Complicated
Some Assessment Basics • Focus on improvement-- VU’s
assessment template stresses improvement—Boxes #6 & #7
• Poor results or student weaknesses are opportunities to improve learning
• Reporting poor results will not affect your position—assessment assumes student learning can be improved; the HLC wants us to report improvement plans
Some Assessment Basics • VU focuses on “authentic”
assessments—assess what students are normally assigned to show they have developed and mastered program skills
• You are not asked to develop “extra” tests or assignments in order to assess
• Choosing the right tools will enable you to do authentic assessment—another workshop
A Technical Note: improve.vinu.edu
• The place to find more assessment help
• The place to access past assessment reports—VU assessment is public!
• The cause of some report redundancy—outcomes, project titles, and success standards
What To “DO” in Box #1 Mission should include: •Program purpose, which should reflect or imply faculty goals and values for students •A broad description of curriculum learning opportunities •Identification of what graduates will be prepared to do
Program Outcomes DO’s and DON’T’s
DO: • Write 3-6 outcomes
reflecting what students will be able to know, do or value at the end of the program
• Begin with Bloom’s verbs—suggesting student action
• Construct a range of knowledge, skills, values
DON’T: • Write outcomes for a
single course • Write them all at a low
or high level of knowledge, skill, or value
Changing Mission or Outcomes • It’s OK to change
mission and outcomes • Collaborate with all
department faculty on the changes
• Note the changes in the left hand box of the form, the only left box that you will add to or work in—Amy says, “No changes, please!”
What to “DO” in Box 2
• Collaborate with program faculty to select 2 different program outcomes to assess
• If you’re starting new 3-year cycle, you should assess at least one new outcome
• If you assess one outcome for a second 3-year cycle, assess it at a different level or in a different class
UCC Critical Thinking Public Service Announcements:
• You can count UCC assessment as assessment of 1 outcome only; you should assess a program outcome for Outcome 2
• If you assess UCC courses, you need to have a critical thinking outcome in your list of outcomes; now is the time to add, if you must
• All faculty teaching UCC Liberal Education courses must put assignment on Bb and align with Critical Thinking outcome—Contact Debbie Stanzcak in the CTL
Box 2--More
• DO briefly describe the learning to be assessed
• DO visualize and describe what students should know, be able to do, or value at the time assessment occurs
• If your program is mapped, DO identify the level of learning
• DO explain the significance of the skill, based in research, experience, external recommendations, etc
Don’t identify significance as “this is a skill that graduates should know” or other obvious reasons true for any program
What to “DO” in Box 3—The Basics • Repeat the outcomes identified in
Box 2—there’s that redundancy! • Identify 2 (or more) projects per
outcome • Name the projects—be brief, but
identify learning to be assessed, assignment type, or how learning will be assessed (tool)
• Identify the year of the assessment in a 3-year cycle
• Briefly describe the project—what learning will be assessed and how
More Basics of Box 3—For Each Project • Identify the assessment tool—
paper and rubric, test, lab report or activity, presentation, etc.
• Identify the success standard • Identify courses in which
assessment will occur, faculty who will be responsible for the assessment, estimated sample size, semester when the assessment will occur, and faculty project leader
Box 3—Some Points to Remember
1. At least one project for each outcome must be a direct assessment—an assessment of a student’s work or performance that gives some measurable evidence of the student’s ability
2. One project might be an indirect assessment—a student’s, faculty member’s, employer’s, internship director’s perception of learning; tools such as surveys or reflective writing are used to collect indirect evidence
Box 3—Some Points to Remember
3. Make sure projects A and B are complementary, even if in different classes
4. Tools matter—select tools that will produce specific, measurable details about learning—2 tools to consider: analytic rubrics and reflective writing—another workshop
5. Set standards that will get into specific details • Do item analysis on individual test questions • Do rubric or checklist dimension analysis
Box 3—Some Points to Remember
6. Set realistic standards—consider what is manageable for an average class of students or acceptable to the profession
7. Set standards that are consistent with the sample size—it’s a math thing!
8. Vincennes and Jasper campus faculty should be working together on assessment if there is a companion Jasper program
“Step” 4—It Won’t Be a “Box”
• DON’T add any information! • DO send raw data and tools used
to Amy Coots—this is for an evidence room during HLC visit
• DO send test questions, rubrics, assignment descriptions, survey or reflection questions, etc.
• DON’T worry—data and test questions will not be accessible
The DO’s and DON’T’s in Box 5 • DO give data indicating strengths,
weaknesses, and trends • DON’T give analysis in the “Data”
sections—save for “Analysis” section • DO give specific data for specific skill
elements or items assessed: “76% of students achieved “acceptable” on supplying evidence” or “Only 51% correctly answered the question on alignment of the tool.”
• DO make sure data “adds up” and matches your success standard
Trend Data in Box 5 • A “trend” is a measure of the
same data over a period of years
• A “trend” is not a generalization about students in a single year
• If you’re in the first year of a project, you have no trend, so mark “N/A”
• In the second or third year of a project, compare data from previous year or years
Analysis in Box 5 • Analysis sets up improvements in
boxes 6 & 7 • DO consider what Projects A and
B combined tell you about student learning—how do results complement each other?
• DO consider and explain data indicating strengths, weaknesses, and trends
• DO give a causal explanation of both student success and failures
Analysis in Box 5 • DON’T blame nature, the
environment, students’ poor memory, or other non-curricular factors for student failures unless you can address them
• DO analyze results in terms of what you have control of or what you can control—the curriculum, your instruction, and you’re evaluation and tools
The DO’s and DON’T’s of Box 6 • DON’T add analysis in “Impact of
Previous Year’s Plan” section if you’re in the first year of a project; DO simply put “N/A”
• DO briefly describe the previous year’s curriculum improvement plan
• DO explain whether the plan improved success or not
• DO reference the data and analysis in Box 5 to support your claims about the impact of the plan
The DO’s and DON’T’s of Box 6 • DO develop a new improvement plan,
based on analysis of strengths, trends and weaknesses identified in Box 5
• DO include all program faculty in the planning, including those in Jasper
• DON’T describe what might be done or what program faculty discussed as possibilities—make a definite plan of action, one that can produce results
• DO consider in the plan professional development and coordinate it
Finally, Box 7—DO’s and DON’T’s • DO describe your assessment tools
and project improvement plans • DO make sure the plans align with
the data and analysis in Box 5 and any curriculum plans in Box 6
• DO make plans that will produce more specific data to be used to better judge strengths and weaknesses
• DON’T make unnecessary changes—use “N/A”
Some People’s View of Assessment
But, With the Right Focus
You, Too, Can Be An Assessment Champion
And Impress Your Friends and Colleagues!