Data for accountability

www.pbisassessment.org ww.swis.org

www.pbiseval.org

Sustaining Communities of Practice Schoolwide positive behaviour support

Queensland Conference 2011

Presented by Anne W. Todd University of Oregon [email protected]

 Accountability  Progress Monitoring  Evaluation ◦ Measure fidelity of implementation ◦ Measure student outcomes

An Example Elementary Playground Problems

 Total enrollment= 550   3 classes per grade level   18 classrooms (30/class)

 Primary Problem Statement ◦  fighting and physical aggression on playground

  550 students full playground area, expectations, equipment use

 Precise Problem Statement ◦  High rates of physical aggression, disrespect and inappropriate

language on the playground during second and third grade recess. Many students are involved and it appears they are trying to get access to equipment/games   180 2ne/3rd graders, routine for accessing/sharing equipment/games

0

5

10

15

20

25

30

Planning time Implementation time:staff Implementation time: students

2 precison elements

4+ precision elements

hour

s

 Two things to measure ◦ Did we do what we planned and did we do it

well? ◦ Did it have a positive impact on student

Behaviour?

•  If using the same measurement tools, districts can compare the implementation status across schools

PBIS Assessment is a web-based application designed to assist in high-fidelity, sustained implementation of school-wide positive Behaviour support (SWPBS). The goal of the website is to improve the efficiency and accuracy with which tool/instrument can be used to complete three functions:

A. Initial Assessment of discipline practices to determine if and how SWPBS should be adopted.

B. Implementation Assessment of the fidelity with which SWPBS procedures are being used, and the design of “action plans” to improve implementation fidelity.

C. Sustained Assessment of the implementation of SWPBS at all three tiers to facilitator on-going use of core SWPBS features.

1. Improvement of the application structure: Research Only Assessment of Implementation

2. Inclusion of Other Surveys & Instruments : Evidence & research based tools Research, Annual Assessments & Progress Monitoring

3. Evaluation Template: PBIS Assessment will have the capacity to include an evaluation template for schools/districts in developing evaluation plans.

4. Action Planning: PBIS Assessment will incorporate an Action Plan that will combine information from all the surveys a school enters into PBIS Assessment. The data entered will be used to create one Action Plan that schools can use for Annual Action Planning.

The following will ALL transfer from PBIS Surveys to PBIS Assessment:

 Login Information  Survey Data  Open Survey Windows  All Previous Data

Research Tools Annual

Assessment Tool

Progress Monitoring Tool

Universal System (Tier 1)

SET *PreSET

BoQ 2.0 SAS

TIC 3.0 *PIC

Secondary & Tertiary Systems

(Tier 2 & 3) *ISSET SAS

*BAT *MATT *PIC

Outcome Tool/instrument: School Safety Survey

* Tool to be included in future version

Review Status and

Identify Problems

Develop and Refine

Hypotheses

Discuss and Select

Solutions

Develop and Implement Action Plan

Evaluate and Revise

Action Plan

Problem Solving Foundations

Team Initiated Problem Solving (TIPS) Model

  The School-wide Evaluation Tool (SET), an interview and observation protocol through which an external expert can evaluate the status of critical PBIS features both before and after implementation, data entry is available for coordinators only.

  The SET is a research instrument for determining the extent to which a school is implementing school-wide positive Behaviour support.

  The SET results provide a summary score that is used (a) to determine annual goals for school-wide effective Behaviour support, (b) to evaluate on-going efforts toward school-wide Behaviour support, (c) to design and revise procedures as needed, and (d) to compare annual accomplishments toward school-wide effective Behaviour support.

Feature Areas of SET Data Sources a)  expectations defined, b)  Behavioural expectations taught, c)  acknowledgement procedures, d)  correction procedures, e)  monitoring and evaluation, f)  management, and g)  district level support.

Permanent Products (e.g. discipline handbook, school improvement plan for safety related goal, instructional materials, meeting minutes)

Observations Staff Interviews Student Interviews

  The BoQ is an expedient, effective assessment tool that measures the degree to which a school is implementing the universal level of school-wide positive Behaviour support (SWPBS). Data entry is available for coordinators only.

  The process typically includes the following:   The coach (facilitator) first completes the Coach Scoring Form

using the Scoring Guide that provides operational definitions of the scores for each item.

  The team members then individually complete the Team Member Rating Form, a simplified version of the Coach’s Scoring Form that does not require the Scoring Guide.

  The BoQ has a total possible score of 100. This score is derived from the 3-8 items in each of the 10 subscales. Each item has a maximum value between one and three points, and points for the items are summed to obtain a Total Score.

 The Team Checklist is used to guide PBIS implementation team activities throughout the year.

 The checklist is used as a status report quarterly or monthly. Teams can use the results of the checklist to complete or revise an action plan for the year.

 The School Safety Survey is designed to assess risk factors and response plans for school safety and violence.

 The Safety Survey is completed annually by a minimum of five school staff, including the custodian.

 This information is useful in determining training and support needs related to school safety and violence prevention.

  The Self-Assessment Survey is used by school staff to evaluate the implementation of PBIS systems.

  The survey examines the current status and need for improvement of four Behaviour support systems: a)  school-wide discipline systems, b)  non-classroom management systems (e.g., cafeteria, hallway, playground), c)  classroom management systems, and d)  systems for individual students engaging in chronic problem Behaviours.

  The Self-Assessment Survey is typically completed when a school first begins adoption of School-wide Behaviour Support systems, and then annually (preferably in the Spring) thereafter as part of the team’s action planning process.

  School-wide  Non-classroom

•  Classroom •  Individual

  Expectations defined (question 1) Expectations taught (2) Reward system (3) Violations system (4-8) Monitoring (10-12) Management (9, 14-16) District support (17-18)

 Reliable & evidence-based  Consistent across states  Meets need(s) not duplicated by tool/

instrument currently in user.

PBIS Assessment Go To Webinars: Date Time

July 6 8:00 AM PST July 14 8:00 AM PST July 20 1:00 PM PST July 28 1:00 PM PST

August 3 8:00 AM PST August 11 8:00 AM PST August 17 1:00 PM PST August 25 1:00 PM PST

Register at www.pbisassessment.org

 www.swis.org ◦ Annual License   SWIS - $250 US/year   CICO-SWIS -$50 US/year   ISIS-SWIS - $150 US/year

Web-based application for entering problem Behaviour incidents and for generating reports for problem solving and decision-making

Tota

l Offi

ce D

isci

plin

e R

efer

rals

Total Office Discipline Referrals as of January 10

Newton, J. S., Todd, A. W., Algozzine, K., Horner, R. H., & Algozzine, B. (2009). The Team Initiated Problem Solving (TIPS) Training Manual. Educational and Community Supports, University of Oregon, unpublished training manual. 45


Grade Range Number of Schools

Mean Enrollment per school

Median ODRs per 100 per school day

K-6 2565 452 .22

6-9 713 648 .50

9-12 266 897 .68

K-(8-12) 474 423 .42


Elementary School with 150 Students

Compare with National Median 150 / 100 = 1.50 1.50 X .22 = .33


50

~80% of Students

~15%

~5%

0-1 office discipline referral

6+ office discipline referrals

2-5 office discipline referrals

Using office discipline referrals as a metric for universal screening of student social Behaviour


0

2

4

6

8

10

12

Aug Sept Oct Nov Dec Jan Feb Mar Apr May Jun

0-1

2-5

6+

Cum

ulat

ive

Mea

n O

DR

s

Cumulative Mean ODRs Per Month for 325+ Elementary Schools 08-09

Jennifer Frank, Kent McIntosh, Seth May


0

5

10

15

20

Sep Oct Nov Dec Jan

Avg.

OD

Rs

Per S

choo

l Day

School Months

School Avg. National Avg. = 3.9

Trevor Test Middle School 11/01/2007 through 01/31/2008 (last 3 mos.)

54

1.  Most Disruptions occur in Cafeteria 2.  Most Disruptions occur in Cafeteria between

11:30 AM and 12:00 PM 3.  Most instances Inappropriate Language occur

in Cafeteria between 11:30 AM and 12:00 AM

Now…use a Custom Graph to confirm (or disconfirm) your inferences, starting with Disruptions, by grade level


 www.pbiseval.org  Annual License ◦  State - $1000 US/year ◦ Region/District - $500 US/year

Aggregates PBIS Assessment Data and SWIS data by School, by Cohort (self defined), by District, by State

  Start with decision to be made   Provide support for getting accurate data   Integrate data sources   Share the data ----- regularly ◦  Include critical teams/ people for problem solving

  Make decisions based on the data ◦  Go for the small stuff

  Celebrate successes!

Review Status and

Identify Problems

Develop and Refine

Hypotheses

Discuss and Select

Solutions

Develop and Implement Action Plan

Evaluate and Revise

Action Plan

Problem Solving Foundations

Team Initiated Problem Solving (TIPS) Model

Technology

Data for accountability