Upload
hye
View
45
Download
0
Embed Size (px)
DESCRIPTION
Governor’s Prevention Initiative for Youth: Evaluation. Jane A. Ungemack, Dr.P.H. Evaluator University of Connecticut Health Center. Evaluation. Systematic efforts to collect and use information: Document program implementation Describe target populations/participants - PowerPoint PPT Presentation
Citation preview
Governor’s Prevention Initiative for Youth: Evaluation
Jane A. Ungemack, Dr.P.H.Evaluator
University of Connecticut Health Center
Evaluation
• Systematic efforts to collect and use information:– Document program implementation– Describe target populations/participants– Inform and improve program performance– Access program effectiveness– Increase accountability– Increase understanding
Governor’s Prevention Initiative for Youth
Long-term Goal: Reduce substance use among adolescents
Intermediate Goal: Reduce risk factors and increase protective factors for
substance use in the individual, peer, family, school and community domains
Target Group: 12-17 year old youth
Evaluation Framework OverviewII. Contextual Conditions
(Economic, Cultural, Risk Conditions, ATOD use)
2. State System 4. State-Level 6. State-Level 8. State-LevelCharacteristics/ Collaborative Intermediate Systems
Dynamics Strategies/ Outcomes ChangeActivities
Training/TA Training/TA
1. SIGMobilization
3. Sub-RecipientCharacteristics
Training/TA
5a. SubRecipient
Planning forScience-based
PreventionInterventions
7a. Sub-Recipient/CommunityImmediate
Outcomes & systemchanges
9. CommunityIntermediateOutcomes:Risk and
ProtectiveFactors
10. Long-termOutcomes:Behavioral
Impacts
5b. ProgramInterventions:
Activities/Actions
7b. ProgramOutcomes: Risk and
Protective Factor
Evaluation Approach
Process evaluation:• documents program implementation and
activitiesOutcome evaluation: • assesses program effects or impacts
Capacity-Building for Evaluation
Science-based approachEvaluation and assessment as integral parts of
the program design– Community-level – Program-level
Evaluation Team Relationship with Grantees
• Training and technical assistance• Instruments and administration protocols• Consultation and collaboration• Statewide coordination
Grantee Responsibilities
• Develop a program plan based on the logic model • Specify measurable program objectives• Cooperate and collaborate with UConn Evaluation
Team• Coordinate community survey• Collect and submit process data • Collect and submit outcome data • Commit time and effort to evaluation activities
Assessing Community-Level Outcomes: CSAP Requirements
School surveyUse of core substance use, risk and protective
factor measures
Community-Level Assessment: School Survey
• Mandated • 7th-10th grade students • Representative sample (minimum n=500;
125/grade level)• Year 1 and Year 3• First administration: February-April, 2000
School Survey
• Self-administered during a classroom period
• Anonymous and confidential• Parental consents • Sampling, instrument and administration
protocols provided by UConn Evaluation Team
School Survey
Measures– Demographic characteristics– Lifetime and current use of ATOD– Risk and protective factors– Limited community-specific items
School Survey
Grantee responsibilities:– Planning/coordination with UConn Evaluation
Team– Planning/coordination with school personnel– Instrument duplication– Data cleaning– Data entry– Analysis
Assessing Program-Level Outcomes: CSAP Requirements
• Select a minimum of three programs (for each of three domains)
• Measure program outcomes using core measures• Include a sufficient sample size for analysis• Collect pre- and post-test data
Program-Level Evaluation
Process evaluation:• Document program implementation and activities Outcome evaluation: • Assess program outcomes
Program-Level Evaluation: Process Evaluation
Each program will be responsible for reporting:– Prevention strategies– Types of activities– Dosage– Number served– Participant characteristics (age, gender,
race/ethnicity, etc.).
Process Evaluation
• Minimum Data Set (MDS) • Instruments, protocols, and training
provided by the UConn Evaluation Team
Program-Level Evaluation:Outcome Evaluation
• Based on the logic model, identify measurable objectives that you will address in your program
• Program objectives should be selected from one or more of the risk/protective factors included in the RFP list of Connecticut Intermediate Outcomes
Outcome Evaluation
• UConn Evaluation Team staff will work with each grantee to finalize program-specific objectives and measures
• All grantees will be asked to participate in a pre/post-test assessment of age-eligible participants as appropriate
Outcome Evaluation
Pre- and Post-Test Assessments • Youth participation will be voluntary• Confidential• Informed consents • Standardized instrument plus optional
program-specific items• Minimum sample size = 50
Considerations for Estimating Evaluation Costs
• Personnel (.25 FTE minimum recommended)
• Computer equipment • Photocopying • Office supplies • Data collection and cleaning• Data entry
Evaluation Themes for the Governor’s Prevention Initiative
• Evaluation at all levels• Science-based• Capacity-building • Collaboration• Coordination • Sustainability