23
1 SOAR School Counseling Program Department of Leadership, School Counseling, and Sport Management COMPREHENSIVE ASSESSMENT PROGRAM

COMPREHENSIVE ASSESSMENT PROGRAM - · PDF fileComprehensive Assessment Program Table of Contents ... PoM 1 – The Admissions Interview and Portfolio Review (Prior to Course Enrollment)

Embed Size (px)

Citation preview

Page 1: COMPREHENSIVE ASSESSMENT PROGRAM - · PDF fileComprehensive Assessment Program Table of Contents ... PoM 1 – The Admissions Interview and Portfolio Review (Prior to Course Enrollment)

1

SOAR School Counseling Program

Department of Leadership, School Counseling, and Sport

Management

COMPREHENSIVE

ASSESSMENT PROGRAM

Page 2: COMPREHENSIVE ASSESSMENT PROGRAM - · PDF fileComprehensive Assessment Program Table of Contents ... PoM 1 – The Admissions Interview and Portfolio Review (Prior to Course Enrollment)

2

Comprehensive Assessment Program Table of Contents

SOAR Program Mission Statement ……………………………………….. 3

“Plan” or “Program”? ……………………………………….. 3

Comprehensive Assessment Plan Overview ……………………………………….. 3

Comprehensive Assessment Philosophy and

Commitments ……………………………………….. 4

Student Assessment ……………………………………….. 5

Assessment of Student Competencies ……………………………………….. 6

The Points of Measurement (PoM) (Benchmarks) ……………………………………….. 6

Assessment of Student Competencies Matrix ……………………………………….. 9

Figure 1. Assessment of Student Competencies ……………………………………….. 12

SOAR Program Assessment ……………………………………….. 13

Assessment of Program Objectives Matrix ……………………………………….. 14

Figure 2. Assessment of Program Objectives ……………………………………….. 16

Key Assessment Findings and Program Actions/

Adjustments ……………………………………….. 17

Impacts of the SOAR Comprehensive Assessment

Program ……………………………………….. 22

Iterative Impact of the SOAR Comprehensive

Assessment Program ……………………………………….. 22

Future Direction of the SOAR Comprehensive

Assessment Program ……………………………………….. 23

Page 3: COMPREHENSIVE ASSESSMENT PROGRAM - · PDF fileComprehensive Assessment Program Table of Contents ... PoM 1 – The Admissions Interview and Portfolio Review (Prior to Course Enrollment)

3

Comprehensive Assessment Program

SOAR School Counseling Program

University of North Florida

SOAR Mission Statement:

The SOAR School Counseling Program at the University of North Florida prepares counselors

who have developed strong basic counseling, relational, and reflective skills; who demonstrate

sufficient multicultural content knowledge, skills, and practices; who practice from a sound

foundation of theoretical and research-based knowledge; who hold and maintain a strong

counselor identity; and who possess and exhibit the personal and professional dispositions

necessary to work effectively as school counselors.

“Plan” or “Program”?

Over the past five years, the SOAR has revised and extended our comprehensive assessment

program. Although, these revisions have been substantial in areas, they have been indeed

revisions to what we believe to have been a comprehensive assessment program. As such, in this

self-study we use the term “program” and not “plan.”

Comprehensive Assessment Program Overview

Our comprehensive assessment program has grown to include systematic processes for assessing

both the SOAR program and our students. More specifically, our comprehensive assessment

program appraises the SOAR program’s fidelity to programmatic objectives and the Learning

Environment and Professional Practice elements of the CACREP standards (Section I); and also

appraises the development of each SOAR student throughout the program and this assessment is

based on whether they meet the CACREP School Counseling program standards (as documented

and tracked within the UNF College of Education and Human Services (COEHS) Electronic

Candidate Assessment Tracking System (ECATS), their academic performance, professional

development, and personal/social development. In addition to assessing student development and

competency through the lens of the CACREP School Counseling program standards, academic

performance, professional development, and personal/social development; SOAR also evaluates

each student around five student competencies developed directly from the SOAR mission

statement.

Page 4: COMPREHENSIVE ASSESSMENT PROGRAM - · PDF fileComprehensive Assessment Program Table of Contents ... PoM 1 – The Admissions Interview and Portfolio Review (Prior to Course Enrollment)

4

The data from these multiple assessment processes and procedures are then compiled and

analyzed. The findings that emerge from these analyses are then regularly discussed and

examined by program faculty members in order to assess all aspects of the SOAR program and

subsequently develop recommendations and/or suggest changes/revisions/modifications to

SOAR curricula, course scope and sequence, programmatic policy and practices, faculty

activities, student recruitment and selection, student retention, monitoring of student progress,

and any other aspect of the SOAR program. These findings are then shared with SOAR students,

department and college administration, site supervisors, advisory council members, SOAR

graduates, and employers of SOAR graduates.

Finally, SOAR faculty believe that the same rigorous and purposeful assessment approach to

informing student, faculty, and program development must also be applied to our comprehensive

assessment program, itself. As such, our program faculty, current and past students, and advisory

council all review the assessment tools, processes, and procedures in order to improve upon it.

Comprehensive Assessment Philosophy and Commitments

The SOAR Comprehensive Assessment Program is framed around the notions that assessment

should be process- and outcome-driven, participatory, and include multiple stakeholder

perspectives when possible and appropriate. Moreover, our program is constructed around the

idea that assessment should be goal-oriented and provide both formative and summative

feedback loops. Furthermore, assessment tools should be useful to understand the fidelity and

outcomes of the SOAR program.

Fidelity- Does the SOAR program enact our stated procedures and activities in order to

achieve meet our mission and objectives? Does the SOAR program adequately and

meaningfully embed and include the CACREP standards within our program, processes,

and curricula?

Outcomes- Have students, faculty, and our program developed through our stated

procedures, activities, processes, and curricula? Have SOAR students met the CACREP

School Counseling program standards? Have SOAR students achieved competency for

the five SOAR student competencies developed from the SOAR program mission?

Page 5: COMPREHENSIVE ASSESSMENT PROGRAM - · PDF fileComprehensive Assessment Program Table of Contents ... PoM 1 – The Admissions Interview and Portfolio Review (Prior to Course Enrollment)

5

In order to track, document, assess, and make meaning from the development of our students,

faculty, and program; we constructed our SOAR Comprehensive Assessment Plan so that

CACREP Standards, Student Competencies, and Program Objectives are measured with the

following commitments:

Multiple points of measurement will occur for each student competency and program

objectives

Student competencies and program objectives will be assessed from multiple stakeholder

vantages and perspectives when appropriate

Multiple forms of assessment data are used, including:

o Outcome data for student competencies

Qualitative and quantitative

o Process data for program objectives

Qualitative and quantitative

In order to track program fidelity to program objectives

Assessments, both formative and summative, are informative.

o That is, resultant data are used to inform our understanding of current

performance (student, faculty, program), inform changes to the program and

training, and inform future development and/or revision and refinement of future

program structures, processes, objectives, and goals.

Student Assessment

In addition to tracking student development through academic performance and professional and

personal/social development, as well as in relation to the CACREP School Counseling program

standards, our plan assesses SOAR students around five competencies that SOAR program

faculty believe to be at the foundation of our mission. The assessment of student competencies is

accomplished throughout the academic year and is represented in chronological phases. The

Points of Measurement (PoM) correspond to how students ideally matriculate through our two-

year program. The assessment of student competencies begins when students are admitted to the

University and continues after students complete the program (graduate candidates and

graduates). The SOAR five student competencies are:

Page 6: COMPREHENSIVE ASSESSMENT PROGRAM - · PDF fileComprehensive Assessment Program Table of Contents ... PoM 1 – The Admissions Interview and Portfolio Review (Prior to Course Enrollment)

6

1. Strong basic counseling, relational, and reflective skills

2. Multicultural content knowledge, skills, and practices

3. Sound foundation of theoretical and research-based knowledge

4. Strong counselor identity

5. Personal and professional dispositions necessary to work as school counselors

Assessment of Student Competencies

The assessment of student competencies is accomplished throughout the academic year and is

represented in chronological phases. The Points of Measurement (PoM) correspond to how

students ideally matriculate through our two-year program and our assessment approaches for

them as they matriculate through – from their admissions process into their graduation. The

assessment of student competencies begins prior to when students are admitted to the University

and SOAR program and continues after students complete the program (graduate candidates and

graduates).

The Points of Measurement (PoM) (Benchmarks)

The seven phases and corresponding PoM for assessing each of the student competencies are:

Pre-Program (Pre-Candidacy)

PoM 1 – The Admissions Interview and Portfolio Review (Prior to Course Enrollment)

Upon admission to the University, students:

a. attend a mandatory Admission Interview that is facilitated by, at

least, two full time faculty members

b. submit a Portfolio, which includes

i. an Applicant Statement of Understanding

ii. Academic transcripts

iii. relevant entrance exam results (GK or GRE)

iv. an Individualized School Experience Contract

v. copy of teaching certificate, if applicable

Mid-Point Year 1

PoM 2 – Site Supervisor’s Evaluation of School Counseling Practicum Candidate (Mid-

point of First Year of Program)

Candidates are evaluated on relevant standards, at the end of their

Practicum field experience and by the Site Supervisor.

Page 7: COMPREHENSIVE ASSESSMENT PROGRAM - · PDF fileComprehensive Assessment Program Table of Contents ... PoM 1 – The Admissions Interview and Portfolio Review (Prior to Course Enrollment)

7

PoM 3 – Practicum Site Supervisor’s Evaluation

Year 1

PoM 4 – Student Self-Reflection on Basic Counseling Skills Development (During First

Year of Program)

Candidates complete written and video reflections about their

personal development of basic counseling skills during the Practicum and first

Internship experiences. Candidates are provided opportunities to discuss and

assess their development of basic skills with their field experience instructors and

Individual Supervisors.

Second Semester Evaluation

Year 1 & 2

PoM 5 – Observations of School Counseling Candidate using Florida Educator

Accomplished Practices (FEAPS) 2012 (During First and Second Internships)

During all internships, candidates are observed and evaluated on the state required

practices related to school counseling certification.

PoM 6 - – Site Supervisor’s Evaluation of School Counseling Internship Candidate

Candidates are evaluated on relevant standards, at the end of each

Internships experience and by the Site Supervisor.

PoM 7 – Internship Site Supervisor’s Evaluation

Third Semester Evaluation

Continuous

PoM 8 – Student Progress Reports (Continuous in Program Enrollment)

Student Progress Reports are conducted regularly and discussed by

SOAR faculty at monthly program meetings. Special attention is given to issues

that arise and are related to candidate development on case-by-case basis.

PoM 9 – Electronic Candidate Assessment Tracking System (ECATS) – Integrated

CACREP School Counseling Standard (Continuous in Program Enrollment)

Candidates are assessed by the standards required by the state

accrediting bodies through the college-provided ECATS indicators that are

embedded in relevant program courses.

PoM 10 – Proof of Membership in Professional Associations (Continuous in Program

Enrollment)

Candidates are encouraged to join and support local, state and

national professional associations.

PoM 11 – Conference Attendance and Trainings (Continuous in Program Enrollment)

Candidates are encouraged to join and support local, state and

Page 8: COMPREHENSIVE ASSESSMENT PROGRAM - · PDF fileComprehensive Assessment Program Table of Contents ... PoM 1 – The Admissions Interview and Portfolio Review (Prior to Course Enrollment)

8

national conferences and trainings to development relevant skills and experiences.

Graduate Candidate

PoM 12 – Florida Teaching Certification Exam (FTCE): Guidance and Counseling PK-

12 (Subject Area Exam) (By Completion of Program)

Candidates are assessed on professionalism and content area by the

state certification body, as evidenced by passing the state certification exam for

school counselors.

End Point/ Exit Evaluation

Graduate (Post Candidacy) PoM 13 - Employer Survey

Employers of our graduates are surveyed in order to understand their perceptions

regarding the effectiveness and level of preparedness of our graduates.

PoM 14 (Alumni Survey and/or SOAR Programmatic Formative Assessment)

Page 9: COMPREHENSIVE ASSESSMENT PROGRAM - · PDF fileComprehensive Assessment Program Table of Contents ... PoM 1 – The Admissions Interview and Portfolio Review (Prior to Course Enrollment)

9

Assessment of Student Competencies Matrix

Student

Competencies:

Point of Measurement

(Benchmark): Measurement Instrument/Tool:

1. The student has

strong basic

counseling, relational,

and reflective skills

Pre-Program (PoM 1) Admissions Interview/Portfolio Review

Mid-Point Year 1 (PoM 2) Site Supervisor’s Evaluation of School

Counseling Practicum Candidate

Mid-Point Year 1 (PoM 3) Practicum Site Supervisor’s Evaluation

Year 1 (PoM 4) Student Self-Reflection on Basic Counseling

Skill Development

Year 1 & 2 (PoM 5) Observations of School Counseling

Candidate using the Florida Educator

Accomplished Practices 2012

Year 1 & 2 (PoM 6) Site Supervisor’s Evaluation of School

Counseling Internship Candidate

Year 1 Second Semester Evaluation

Year 2 Third Semester Evaluation

Continuous (PoM 8) Student Progress Reports

Continuous (PoM 9) ECATS – Integrated CACREP School

Counseling Standards

Graduate Candidate End Point/Exit Evaluation

Graduate Employer Survey

Graduate Alumni Survey and/or SOAR Programmatic

Formative Assessment

2. The student has

strong multicultural

content knowledge,

skills, and practices

Pre-Program (PoM 1) Admissions Interview/Portfolio Review

Mid-Point Year 1 (PoM 2) Site Supervisor’s Evaluation of School

Counseling Practicum Candidate

Mid-Point Year 1 (PoM 3) Practicum Site Supervisor’s Evaluation

Year 1 (PoM 4) Student Self-Reflection on Basic Counseling

Skill Development

Year 1 & 2 (PoM 5) Observations of School Counseling

Candidate using the Florida Educator

Accomplished Practices 2012

Year 1 & 2 (PoM 6) Site Supervisor’s Evaluation of School

Counseling Internship Candidate

Year 1 Second Semester Evaluation

Year 2 Third Semester Evaluation

Continuous (PoM 8) Student Progress Reports

Continuous (PoM 9) ECATS – Integrated CACREP School

Page 10: COMPREHENSIVE ASSESSMENT PROGRAM - · PDF fileComprehensive Assessment Program Table of Contents ... PoM 1 – The Admissions Interview and Portfolio Review (Prior to Course Enrollment)

10

Counseling Standards

Graduate Candidate End Point/Exit Evaluation

Graduate Employer Survey

Graduate Alumni Survey and/or SOAR Programmatic

Formative Assessment

3. Sound foundation of

theoretical and

research-based

knowledge

Mid-Point Year 1 (PoM 3) Practicum Site Supervisor’s Evaluation

Year 1 & 2 (PoM 5) Observations of School Counseling

Candidate using the Florida Educator

Accomplished Practices 2012

Year 1 & 2 (PoM 7) Internship Site Supervisor’s Evaluation

Continuous (PoM 9) ECATS – Integrated CACREP School

Counseling Standards

Graduate Candidate (PoM

12)

Florida Teacher Certification Exam: Subject

Area

Graduate Alumni Survey and/or SOAR Programmatic

Formative Assessment

4. The student holds

and exhibits a strong

counselor identity.

Year 1 & 2 (PoM 6) Site Supervisor’s Evaluation of School

Counseling Internship Candidate

Continuous (PoM 10) Proof of Membership in Professional

Association(s)

Continuous (PoM 11) Conference Attendance/Trainings

5. The student

possesses and exhibit

the personal and

professional

dispositions necessary

to work effectively as

school counselors

Pre-Program (PoM 1) Admissions Interview/Portfolio Review

Mid-Point Year 1 (PoM 2) Site Supervisor’s Evaluation of School

Counseling Practicum Candidate

Mid-Point Year 1 (PoM 3) Practicum Site Supervisor’s Evaluation

Year 1 (PoM 4) Student Self-Reflection on Basic Counseling

Skill Development

Year 1 & 2 (PoM 5) Observations of School Counseling

Candidate using the Florida Educator

Accomplished Practices 2012

Page 11: COMPREHENSIVE ASSESSMENT PROGRAM - · PDF fileComprehensive Assessment Program Table of Contents ... PoM 1 – The Admissions Interview and Portfolio Review (Prior to Course Enrollment)

11

Year 1 & 2 (PoM 6) Site Supervisor’s Evaluation of School

Counseling Internship Candidate

Year 1 Second Semester Evaluation

Year 2 Third Semester Evaluation

Continuous (PoM 8) Student Progress Reports

Continuous (PoM 9) ECATS – Integrated CACREP School

Counseling Standards

Graduate Candidate End Point/Exit Evaluation

Graduate Employer Survey

Graduate Alumni Survey and/or SOAR Programmatic

Formative Assessment

Page 12: COMPREHENSIVE ASSESSMENT PROGRAM - · PDF fileComprehensive Assessment Program Table of Contents ... PoM 1 – The Admissions Interview and Portfolio Review (Prior to Course Enrollment)

12

Figure 1. Assessment of Student Competencies

Page 13: COMPREHENSIVE ASSESSMENT PROGRAM - · PDF fileComprehensive Assessment Program Table of Contents ... PoM 1 – The Admissions Interview and Portfolio Review (Prior to Course Enrollment)

13

SOAR Program Assessment

In addition to assessing student competencies, our comprehensive assessment plan evaluates our

overall program objectives. The program objectives are:

1. Recruits, selects, and retains qualified graduate students with attention given to

the inclusion of candidates from culturally diverse and underrepresented

populations.

2. Prepare school counselors who will serve as advocates, educational leaders, team

members, and consultants who will maximize opportunities for every student to

achieve academically and develop as active citizens;

3. Prepare our own faculty, students, our education and community to be highly

skilled and culturally competent practitioners who can best support and promote

equity and justice in our multicultural and pluralistic society.

4. Work collectively and collaboratively with diverse youth, other educators, and

families in order to facilitate individual and community development;

5. Develop and promote the highest quality teaching and learning experiences for

our students, our community and education partners, and ourselves by

continuously assessing our teaching and learning curricula, practices, and

processes;

6. Model community and professional identity through involvement and leadership

in community organizations and initiatives; local, state, and national professional

organizations; and through presentations at state, regional, and national levels.

Page 14: COMPREHENSIVE ASSESSMENT PROGRAM - · PDF fileComprehensive Assessment Program Table of Contents ... PoM 1 – The Admissions Interview and Portfolio Review (Prior to Course Enrollment)

14

Assessment of Program Objectives Matrix

Program Objective: Point of Measurement: Assessor

Vantage Point:

1. The SOAR Program recruits,

selects, and retains qualified

graduate students with attention

given to the inclusion of candidates

from culturally diverse and

underrepresented populations.

Candidate Data

Admissions Data

Information Session Information

Session Data

UNF SOAR Program Advisory

Council

Program Meeting

Minutes

2. The Soar Program prepares

school counselors who will serve as

advocates, educational leaders, team

members, and consultants who will

maximize opportunities for every

student to achieve academically and

develop as active citizens.

Alumni Survey Survey Data

Focus Group Focus Group

Response Data

SOAR School Counseling

Programmatic Formative

Assessment Response Form

Instructional Satisfaction

Questionnaire

ISQ Data

Site Supervisor’s Evaluation of

School Counseling Practicum

Candidate

Employer Survey

3. Prepare our own faculty, students,

our education and community to be

highly skilled and culturally

competent practitioners who can best

support and promote equity and

justice in our multicultural and

pluralistic society.

Presentations Faculty CV

Service to the Profession Faculty CV

Service to the Community Faculty CV

Conference attendance Faculty CV

Page 15: COMPREHENSIVE ASSESSMENT PROGRAM - · PDF fileComprehensive Assessment Program Table of Contents ... PoM 1 – The Admissions Interview and Portfolio Review (Prior to Course Enrollment)

15

Community Based Transformational

Learning

4. Work collectively and

collaboratively with diverse youth,

other educators, and families in

order to facilitate individual and

community development

Community learning initiatives

Partnership Schools

5. Develop and promote the highest

quality teaching and learning

experiences for our students, our

community and education partners,

and ourselves by continuously

assessing our teaching and learning

curricula, practices, and processes.

Student reflections Syllabi

Student evaluations

Faculty retreat

Faculty scholarship Faculty CV

Faculty presentations Faculty CV

Tracdat Benchmarks Tracdat Data

ECATS – Integrated CACREP

School Counseling Standards

ECATS Report

6. Model community and

professional identity through

involvement and leadership in

community organizations and

initiatives; local, state, and national

professional organizations; and

through presentations at state,

regional, and national levels.

Professional Organization

Membership

Faculty CV

Professional Organization

Leadership

Faculty CV

Community Organization

Membership

Faculty CV

Community Organization

Leadership

Faculty CV

Page 16: COMPREHENSIVE ASSESSMENT PROGRAM - · PDF fileComprehensive Assessment Program Table of Contents ... PoM 1 – The Admissions Interview and Portfolio Review (Prior to Course Enrollment)

16

Figure 2. Assessment of Program Objectives

Page 17: COMPREHENSIVE ASSESSMENT PROGRAM - · PDF fileComprehensive Assessment Program Table of Contents ... PoM 1 – The Admissions Interview and Portfolio Review (Prior to Course Enrollment)

17

Key Assessment Findings and Program Actions/Adjustments:

1. Emphasis on supervision early in the SOAR Program

Based upon a confluence of data from various stakeholder vantages and assessment

tools, the SOAR program continues to focus intensive supervision support during

students’ first year in the SOAR program. Over the past four years the data yielded

from our Alumni Focus Groups, Advisory Council meetings, Alumni Surveys, and

Student Course Evaluations have consistently expressed the value of lower faculty

supervisor-student ratios during students first field experience course (SDS 6940:

Practicum in School Counseling) that occurs during their first Fall semester.

Qualitative data from the Alumni Focus Groups and Advisory Council meetings have

placed this expressed value of lower faculty supervisors-student ratios within the

context of the earl immersion of our students in complex field environments. These

different groups of stakeholders both have firmly recommended the maintenance of

low faculty supervisor-student ratios in order to best support the development of

SOAR students as school counselors in resource-challenged, urban school

environments.

Program Action/Adjustment: As a result of findings emerging from our

Comprehensive Assessment Program, the SOAR program has maintained our

commitment to supporting SOAR school counseling student development in

challenging urban school environments by maintaining a 1-6 faculty-student

supervision ratio during students’ first semester of field experiences working

together as a cohort in our DCPS partner schools for their Practicum in School

Counseling course (SDS 6940). This 1-6 ratio applies to the group supervision

provided live by SOAR faculty. In addition, other SOAR faculty also provide

individual and/or triadic supervision that is structured through an innovative 1-credit

course called Supervision of Field Experience I.

2. Data Related to SOAR Program Curriculum

Data from a wide array of Points of Measurement (PoM) (Alumni Surveys, Student

Course Evaluations, SOAR Program Meeting Notes, End Point/Exit Evals, ECATS,

Page 18: COMPREHENSIVE ASSESSMENT PROGRAM - · PDF fileComprehensive Assessment Program Table of Contents ... PoM 1 – The Admissions Interview and Portfolio Review (Prior to Course Enrollment)

18

Alumni Focus Groups) have provided us with clear composite findings regarding

student experiences interacting with SOAR curriculum and in some cases making

recommendations regarding SOAR program curriculum. The more significant over

the past four years have been:

a. The Sequencing of Courses within the SOAR Program

Over the past four years that we have been constructing and using a more robust

Comprehensive Assessment Program, we have gathered data from a variety of

sources and PoMs related to how our courses are sequenced. Notably, findings

from numerous data sets (Alumni Surveys, Alumni Focus Groups, Advisory

Council, faculty meeting notes) highlighted collective perceptions that a program

sequence leading to a Summer Term graduation, limited employment options.

Also, these data sets produced findings involving student anxiety and concern

over the ability to earn sufficient internship hours when the final internship course

(SDS 6830) is provided during students’ final semester as well as the

recommendation that our group counseling course (MHS 6530: Group Leadership

Skills for School Counseling) be moved earlier in the program in order to better

prepare SOAR students for their work within our Duval County Public Schools

partnership schools.

Program Action/Adjustment: In response to the findings and recommendations

regarding SOAR Program course sequencing, we created more flexibility

regarding when students could begin in SOAR by providing them with the

possibility of beginning during the Summer terms. If students choose to start in

the Summer, then they would be eligible to graduate following the Spring

semester of their second year, thus potentially better positioning them to take

advantage of job openings earlier in the summer or late spring.

Additionally, we also moved the second internship course (SDS 6830) from the

Spring semester during the second year of the cohort to the Fall semester in order

to create opportunities for students to continue in internship experiences during

the following Spring semester if necessary, thus alleviating some of the anxiety

surrounding the accruing of internship hours.

Page 19: COMPREHENSIVE ASSESSMENT PROGRAM - · PDF fileComprehensive Assessment Program Table of Contents ... PoM 1 – The Admissions Interview and Portfolio Review (Prior to Course Enrollment)

19

Finally, we also repositioned our group counseling course (MHS 6530: Group

Leadership Skills for School Counseling) earlier in the program in order for our

students to be better prepared for small group work and classroom guidance

lessons that they engage in during their practicum and internship experiences

during their first year in the program.

b. The Value of Early Field/Clinical

Another consistent finding that has emerged throughout the our comprehensive

assessment of our program is the value current and former students and other

SOAR program stakeholders (via Alumni Surveys, Alumni Focus Groups,

Advisory Council, faculty meeting notes) place on the early field/clinical

experiences we provide and structure for our students. Data from each of these

instruments and processes have consistently highlighted the key attributes of the

unique context of the SOAR practicum (SDS 6940) and initial internship (SDS

6832). Responding students and graduates reported that because they began their

direct contact with students while they were developing basic counseling skills

they believed their development occurred in a coherent manner as the result of

being able to apply content knowledge and skills to actual practice in authentic

contexts. These students and former students also indicated that the provision of

live supervision by faculty members was a strength of the approach as it provided

support for their development and services provided to the students with whom

they worked.

Program Action/Adjustment: The data analyzed around the early field/clinical

experiences of students in the SOAR program supported our theoretical and

philosophical reasons for structuring our program with the emphasis on authentic

work with students/clients very early in the SOAR program. These findings in

hand, SOAR has continued this approach to our sequencing of curriculum while

continuing to provide opportunities in supervision and within the field courses for

SOAR students to reflect on the praxis involved when content knowledge and

skills are applied to authentic activities and experiences. As such, SOAR has

developed and implemented two 1-credit courses to ensure the supervision

Page 20: COMPREHENSIVE ASSESSMENT PROGRAM - · PDF fileComprehensive Assessment Program Table of Contents ... PoM 1 – The Admissions Interview and Portfolio Review (Prior to Course Enrollment)

20

component of our practicum and first internship course: Supervision for Field

Experiences I & II.

c. The Perception of Congruence in Course Curricula

As a result of our more focused attention through our Comprehensive Assessment

Program to student perception data from multiple sources (Alumni Surveys,

Alumni Focus Groups, Course Evaluations), SOAR faculty and the SOAR

Advisory Council began to see patterns in data gathered from students that

reported curricular content overlap between our research (EDF 6495: Research in

Counseling) and assessment (MHS 6201: Assessment in School Counseling)

courses. Student responses to open-ended prompts soliciting general feedback on

program experiences and discussions that occurred during Alumni Focus groups

seemed to coalesce around the perception that “inquiry” and how to engage in it

was a foundational element of both courses.

Program Action/Adjustment: As a result of the data that emerged from current

and former students regarding the perceived congruence between some curricular

content in the SOAR research and assessment courses, during the Spring 2010

semester, SOAR faculty facilitated follow-up Alumni Focus Groups and

discussions among SOAR faculty and within SOAR Advisory Council meetings

increasingly focused on the potential pedagogical value for creating one course

that could explore content from both courses and meet the important and

necessary CACREP content standards within the context of effective school

counselor practices. After these follow-up conversations and focus groups, SOAR

faculty decided to pilot the integration of assessment and research content into

one course. Subsequent student and graduate perception data (Alumni Focus

Groups and Alumni Surveys) and faculty perspectives have encouraged are

faculty to continue this innovation culminating with a formal course description

and title change (EDF 6495: Research and Assessment in School Counseling) that

will be fully implanted during the 2013-2014 academic year.

d. Recognition of Curricular Content Gap

When the SOAR Comprehensive Assessment Program developed to include the

CACREP Standard-Based SOAR Program Informative Assessment following the

Page 21: COMPREHENSIVE ASSESSMENT PROGRAM - · PDF fileComprehensive Assessment Program Table of Contents ... PoM 1 – The Admissions Interview and Portfolio Review (Prior to Course Enrollment)

21

Spring 2011 semester, our program faculty recognized a curricular content gap in

our coursework. Specifically, coverage and assessment of two CACREP School

Counseling Standards - C.6. (“Understands the potential impact of crises,

emergencies, and disasters on students, educators, and schools, and knows the

skills needed for crisis intervention”) and D.4. (4. “Demonstrates the ability to use

procedures for assessing and managing suicide risk”) – needed to be strengthened.

Program Action/Adjustment: As a result of these important data around these

essential standards to counselor practice, the SOAR program has added a 1-credit

course named, Crisis/Disaster Management, in order to help ensure that all of our

students are knowledgeable and skilled in such critical practices.

3. Need to Adjust Recruitment Approach and Retention Processes

The SOAR Comprehensive Assessment Program also generated data related to SOAR

processes, procedures, and policies related to the recruitment and retention of diverse

students. Specifically, data that emerged from advisory council meetings, program

meetings, alumni focus groups, and a new addition – applicant evaluation of admission

day processes, began to inform our understanding that we needed to build greater

flexibility into our admissions policies and procedures, that we needed to be more

innovative in our recruitment strategies and technologies, and that we needed to use data

more effectively in order to increase the efficiency and impact of our recruitment efforts.

Program Action/Adjustment: Based on the findings from our advisory council

meetings, program meetings, alumni focus groups, and applicant evaluation of admission

day processes, we made the following adjustments during the 2010-2011 academic year.

We moved to having two admissions dates and individual admissions consultations (for a

Summer term or Fall semester start), we have begun implementing a wider array of

recruitment strategies and technologies (e.g. video, listerservs, mass emails by academic

major at the UNF, etc.), and we have begun to track the turnout for

recruitment/information sessions based upon location, days, and times in order to

schedule more sessions that data show are related to higher applicant turnout.

4. Extended Commitment to Transformational Learning Opportunities of Place

Page 22: COMPREHENSIVE ASSESSMENT PROGRAM - · PDF fileComprehensive Assessment Program Table of Contents ... PoM 1 – The Admissions Interview and Portfolio Review (Prior to Course Enrollment)

22

Finally, alumni focus groups and alumni surveys have produced data converging around

the enthusiasm and pedagogical appreciation for transformational learning opportunities

for SOAR students that take place outside of our surrounding community. Respondents

through both of these assessment strategies have described the importance and value they

placed on experiential immersion programs and opportunities both outside (Turkey,

India, Belize) of and within the United States (New York City cultural immersion trip).

SOAR graduates who experienced one or more of these programs reported that they

believed the experiences increased their multicultural understanding and skills while also

giving them opportunities to reflect on their own cultural identities.

Program Action/Adjustment: The SOAR program will continue to seek out and/or

develop opportunities for students to participate in transformational learning

opportunities outside of our immediate geographical and cultural environment.

Furthermore, our program aspires to institutionalize some of these experiences by

building sustainable partnerships with more of the organizations and institutions involved

by using the partnership for the New York City cultural immersion experience with the

New York Institute of Technology as a model.

Impacts of the SOAR Comprehensive Assessment Program

SOAR program faculty believe that the continual development of our Comprehensive

Assessment Program has allowed us to make more meaningful, purposeful, and informed

decisions about our program, our policies, our practices, and our curriculum. Thus, in order to

help maintain programmatic momentum toward greater effectiveness, we have also begun using

the SOAR Comprehensive Assessment Program to assess the assessment program itself.

Iterative Impact of the SOAR Comprehensive Assessment Program

SOAR faculty who are committed to our professional identity as counselors, we collectively

place a great deal of importance on SOAR students meeting the CACREP School Counseling

Program Standards while also recognizing the challenges in tracking, assessing, and managing

how our students meet these standards. So over the 2011-2012 academic year, our program

leader for that year, Dr. Carolyn Stone, organized a series of focused reflections and discussions

within our program and advisory council meetings in which we examined the current capacity of

Page 23: COMPREHENSIVE ASSESSMENT PROGRAM - · PDF fileComprehensive Assessment Program Table of Contents ... PoM 1 – The Admissions Interview and Portfolio Review (Prior to Course Enrollment)

23

our Comprehensive Assessment Program to effectively track and manage student data related to

the CACREP School Counseling Program Standards so that we could properly assess those data

in a comprehensive way. Through that process, we determined that we needed to make some

changes and so in the spirit of distributed leadership, we re-purposed our College’s ECATS

system by incorporating each of the School Counseling Program standards that we could then

track both by student and by standard. This new and innovative use of an existing evaluation

technology for the essential programmatic practice of ensuring our students and program are

meeting CACREP Standards would likely only have occurred as a result of our full and

enthusiastic embrace of the power and impact of deep and purposeful inquiry by design as

promoted by our Comprehensive Assessment Program.

Future Direction of the SOAR Comprehensive Assessment Program

It is in the enthusiastic spirit of purposeful inquiry that we now would like to begin working with

other program stakeholders (students, graduates, other faculty, administrators, area employers,

area families and youth, etc.) in order advance our Comprehensive Assessment Program beyond

looking at fidelity and outcomes to impact. Specifically, we are looking forward to focusing on

the impact the SOAR program, SOAR faculty, and most importantly SOAR graduates have on

area students, their families, their schools, and our communities.