Doing Assessment Like A Champ! (Plus, Tips for Raising Baby)

Michael Gress Interim Director of Institutional Effectiveness LRC 120, 4275

Assessing Like a Champ: No Human Sacrifices, Please!

A Champ Knows the Basic Steps

The Assessment Cycle: It’s Not That Complicated

Some Assessment Basics • Focus on improvement-- VU’s

assessment template stresses improvement—Boxes #6 & #7

• Poor results or student weaknesses are opportunities to improve learning

• Reporting poor results will not affect your position—assessment assumes student learning can be improved; the HLC wants us to report improvement plans

Some Assessment Basics • VU focuses on “authentic”

assessments—assess what students are normally assigned to show they have developed and mastered program skills

• You are not asked to develop “extra” tests or assignments in order to assess

• Choosing the right tools will enable you to do authentic assessment—another workshop

A Technical Note: improve.vinu.edu

• The place to find more assessment help

• The place to access past assessment reports—VU assessment is public!

• The cause of some report redundancy—outcomes, project titles, and success standards

What To “DO” in Box #1 Mission should include: •Program purpose, which should reflect or imply faculty goals and values for students •A broad description of curriculum learning opportunities •Identification of what graduates will be prepared to do

Program Outcomes DO’s and DON’T’s

DO: • Write 3-6 outcomes

reflecting what students will be able to know, do or value at the end of the program

• Begin with Bloom’s verbs—suggesting student action

• Construct a range of knowledge, skills, values

DON’T: • Write outcomes for a

single course • Write them all at a low

or high level of knowledge, skill, or value

Changing Mission or Outcomes • It’s OK to change

mission and outcomes • Collaborate with all

department faculty on the changes

• Note the changes in the left hand box of the form, the only left box that you will add to or work in—Amy says, “No changes, please!”

What to “DO” in Box 2

• Collaborate with program faculty to select 2 different program outcomes to assess

• If you’re starting new 3-year cycle, you should assess at least one new outcome

• If you assess one outcome for a second 3-year cycle, assess it at a different level or in a different class

UCC Critical Thinking Public Service Announcements:

• You can count UCC assessment as assessment of 1 outcome only; you should assess a program outcome for Outcome 2

• If you assess UCC courses, you need to have a critical thinking outcome in your list of outcomes; now is the time to add, if you must

• All faculty teaching UCC Liberal Education courses must put assignment on Bb and align with Critical Thinking outcome—Contact Debbie Stanzcak in the CTL

Box 2--More

• DO briefly describe the learning to be assessed

• DO visualize and describe what students should know, be able to do, or value at the time assessment occurs

• If your program is mapped, DO identify the level of learning

• DO explain the significance of the skill, based in research, experience, external recommendations, etc

Don’t identify significance as “this is a skill that graduates should know” or other obvious reasons true for any program

What to “DO” in Box 3—The Basics • Repeat the outcomes identified in

Box 2—there’s that redundancy! • Identify 2 (or more) projects per

outcome • Name the projects—be brief, but

identify learning to be assessed, assignment type, or how learning will be assessed (tool)

• Identify the year of the assessment in a 3-year cycle

• Briefly describe the project—what learning will be assessed and how

More Basics of Box 3—For Each Project • Identify the assessment tool—

paper and rubric, test, lab report or activity, presentation, etc.

• Identify the success standard • Identify courses in which

assessment will occur, faculty who will be responsible for the assessment, estimated sample size, semester when the assessment will occur, and faculty project leader

Box 3—Some Points to Remember

1. At least one project for each outcome must be a direct assessment—an assessment of a student’s work or performance that gives some measurable evidence of the student’s ability

2. One project might be an indirect assessment—a student’s, faculty member’s, employer’s, internship director’s perception of learning; tools such as surveys or reflective writing are used to collect indirect evidence

Box 3—Some Points to Remember

3. Make sure projects A and B are complementary, even if in different classes

4. Tools matter—select tools that will produce specific, measurable details about learning—2 tools to consider: analytic rubrics and reflective writing—another workshop

5. Set standards that will get into specific details • Do item analysis on individual test questions • Do rubric or checklist dimension analysis

Box 3—Some Points to Remember

6. Set realistic standards—consider what is manageable for an average class of students or acceptable to the profession

7. Set standards that are consistent with the sample size—it’s a math thing!

8. Vincennes and Jasper campus faculty should be working together on assessment if there is a companion Jasper program

“Step” 4—It Won’t Be a “Box”

• DON’T add any information! • DO send raw data and tools used

to Amy Coots—this is for an evidence room during HLC visit

• DO send test questions, rubrics, assignment descriptions, survey or reflection questions, etc.

• DON’T worry—data and test questions will not be accessible

The DO’s and DON’T’s in Box 5 • DO give data indicating strengths,

weaknesses, and trends • DON’T give analysis in the “Data”

sections—save for “Analysis” section • DO give specific data for specific skill

elements or items assessed: “76% of students achieved “acceptable” on supplying evidence” or “Only 51% correctly answered the question on alignment of the tool.”

• DO make sure data “adds up” and matches your success standard

Trend Data in Box 5 • A “trend” is a measure of the

same data over a period of years

• A “trend” is not a generalization about students in a single year

• If you’re in the first year of a project, you have no trend, so mark “N/A”

• In the second or third year of a project, compare data from previous year or years

Analysis in Box 5 • Analysis sets up improvements in

boxes 6 & 7 • DO consider what Projects A and

B combined tell you about student learning—how do results complement each other?

• DO consider and explain data indicating strengths, weaknesses, and trends

• DO give a causal explanation of both student success and failures

Analysis in Box 5 • DON’T blame nature, the

environment, students’ poor memory, or other non-curricular factors for student failures unless you can address them

• DO analyze results in terms of what you have control of or what you can control—the curriculum, your instruction, and you’re evaluation and tools

The DO’s and DON’T’s of Box 6 • DON’T add analysis in “Impact of

Previous Year’s Plan” section if you’re in the first year of a project; DO simply put “N/A”

• DO briefly describe the previous year’s curriculum improvement plan

• DO explain whether the plan improved success or not

• DO reference the data and analysis in Box 5 to support your claims about the impact of the plan

The DO’s and DON’T’s of Box 6 • DO develop a new improvement plan,

based on analysis of strengths, trends and weaknesses identified in Box 5

• DO include all program faculty in the planning, including those in Jasper

• DON’T describe what might be done or what program faculty discussed as possibilities—make a definite plan of action, one that can produce results

• DO consider in the plan professional development and coordinate it

Finally, Box 7—DO’s and DON’T’s • DO describe your assessment tools

and project improvement plans • DO make sure the plans align with

the data and analysis in Box 5 and any curriculum plans in Box 6

• DO make plans that will produce more specific data to be used to better judge strengths and weaknesses

• DON’T make unnecessary changes—use “N/A”

Some People’s View of Assessment

But, With the Right Focus

You, Too, Can Be An Assessment Champion

And Impress Your Friends and Colleagues!