July Newsletter
Strategic Prevention Framework
Step 5: Evaluation
Evaluation: Adding Razzle Dazzle to Your Prevention Programming Efforts

This number hasn’t been confirmed, but in my experience about 110% of preventionists are passionate about prevention and the work they’re doing. Their elevator speeches advocating for prevention are in their back pockets and ready to be shared at the first glimpse of eye contact from that unsuspecting county commissioner, they work into the wee hours of the morning to make sure everything is in place for that special community event, they meet nervously but energetically with the hesitant community leader that took them hours to contact. Surely, preventionists would give the energizer bunny a run for his money! Regardless of how much passion a preventionist has though, prevention outcomes cannot be achieved with a handful of motivated individuals.
 
We already know that in order for prevention efforts to be sustained over time, the community needs to be involved and sometimes others need to see the value of the prevention efforts and the changes that can, and have, occurred in the community before they invest. Regardless of the type of investment; time, space, energy, resources, they need to know the cause is worth their effort.
 
This is where evaluation comes in. Telling your story, sharing your data with the community and those who are making an investment, or that you would like to make an investment, will greatly increase your opportunities for collaboration. Not only does evaluation help you improve your impact on the community from year to year, but it helps you share that information in concise, measurable ways that will convince even the most reluctant of community leaders.
 
Right now it’s difficult to meet with the community face to face but it may be a good time to pull out your evaluation results and begin to formulate some ideas with your coalition on how to tell your story in various situations.
 
  • In response to public inquiries about the program
  • To educate decision makers.
  • To demonstrate that funds are well spent.
  • To make the target population aware of your program.
  • To show movement in program progress when planned outcomes will not be realized until the distant future.
  • To help mobilize resources.
  • To help your program get needed resources.
 
Evaluation is all about improving and enhancing prevention practice. Razzle dazzle that elevator speech with some interesting facts from your evaluation. If you don’t have results for your community yet, use some data from the evidence-based program you are implementing. There are many ways this can be accomplished. Check out the resources in this publication for ideas!
Additional Resources 
Impact and Value: Telling Your Program's Story
Centers for Disease Control
The purpose of this workbook is to help public health program administrators understand what a “success story” is, why it is important to tell success stories, and how to develop success stories
Conducting Focus Groups
Center for Community Health and Development, University of Kansas
Learn how to plan, prepare, conduct, and use focus group results to receive qualitative data for deeper understanding of community issues.
 
Online Training
Centers for Disease Control and Prevention
The following online training courses listed below were developed for a broad audience, including CDC staff, public health professionals, and those who analyze and inform policy. The goal of these trainings is to increase the learner’s understanding of major policy-related concepts and methods and to improve their capacity in developing and translating science to inform policy decisions.

  • Introduction to Policy Analysis in Public Health
  • Introduction to Economic Evaluation
  • Introduction to Policy Evaluation in Public Health
 
Centers for Disease Control and Prevention (CDC), Program Performance and Evaluation Office
This site features numerous evaluation resources including events and training opportunities, evaluation documents and tools, self-study guides and resources related to the CDC Framework for the Evaluation of Public Health Programs.  Highly recommended :   Introduction to Program Evaluation: A Self Study Guide.
 
The Evaluation Center
University of Western Michigan
This website features a variety of resources including a series of checklists that can be used as guides for evaluation planning, implementation, budgeting and metaevaluation (conducting evaluations of completed evaluations).
What's Happening Around the Region?
Training and Events
Webinar Series: PFS Academy 2020: Making the Steps of the Strategic Prevention Framework Work for You

Each webinar will begin 8:00 PT / 9:00 MT / 10:00 CT / 11:00 ET

The Mid America PTTC, in collaboration with the South Southwest PTTC, is offering a seven-part webinar series on the Strategic Prevention Framework.

SAMHSA’s Strategic Prevention Framework (SPF) provides practitioners with comprehensive guidance to more effectively address substance misuse and related behavioral health problems in their communities. This seven-part webinar series will explore this five-step, data-driven process to identify genuine prevention needs, build capacity and plans to address those needs, implement effective programs and interventions, and evaluate and continually improve prevention efforts.

At each step of the SPF, and in separate sessions, practitioners will learn to incorporate the guiding principles of cultural competence and sustainability to help support the implementation of SAMHSA’s Strategic Prevention Framework. 

Prevention contact hours available to those who register and complete this webinar.
 
  • July 21, Part 6: Sustainability, will provide participants with the elements of a sustainable prevention program and how to integrate sustainability into each step of the SPF.


Recordings of previous events in the series can be found here .
Check out our newest podcast!
Podcast Episode 32: Coalitions, Connection, and Self-Care

This podcast is a discussion with Crystal Fuller. Crystal provides training, technical assistance, funding administration, and helps prevention spread throughout her region in Nebraska. She has over 13 years working in prevention with shared insights on getting out into the community in this virtual world. We also get into the importance of self-care in days filled with Zoom and after Zoom meetings.
Drug Endangered Children: September Peer Sharing Call

Date: September 3

Please join us for our quarterly drug endangered children's peer sharing call. We will be joined by Eric Nation and Stacee Read from the   National Alliance for Drug Endangered Children.
silver-keyboard2.jpg
Self-guided Learning Courses
 
**New** Opioid Overdose Prevention and Infectious Disease Control: Opportunities for Collaboration

Informing Prevention: Adolescent 6-part Webinar series
  • Informing Prevention: Understanding Adolescent Development (1 of 6)
  • Informing Prevention: Effectively Engaging Adolescents in Interventions (Part 2 of 6)
  • Informing Prevention: Effective Use of Epidemiological Data (Part 3 of 6)
  • Informing Prevention: Effectively Using Technology for School-Based Prevention (Part 4 of 6)
  • Informing Prevention: The Effects of Drug Use on Adolescent Brain Development (Part 5 of 6)
  • Informing Prevention: Vaping Among Adolescents (Part 6 of 6)

Today’s Marijuana: Stronger, More Edibles, Confusing Information about Driving

Early Childhood Development: Toxic Stress and Adverse Childhood Experience s

Online Courses
All online courses can be accessed at: healtheknowledge.org/courses
 
If you are new to HealtheKnowledge, please log in or set up an account here: healtheknowledge.org/new-user
Check out the Mid America Prevention Technology Transfer Center website for additional resources and training!
Mid-America PTTC
The Mid-America Prevention Technology Transfer Center (Mid-America PTTC) is designed to serve as a prevention catalyst, empowering individuals and fostering partnerships to promote safe, healthy, and drug-free communities across Missouri, Iowa, Nebraska and Kansas. Our services are evidence-based, culturally competent, and locally focused. We provide intensive technical assistance to support organizations' and systems' efforts to implement evidence-based prevention strategies. The Mid-America PTTC also forms partnerships with local and regional stakeholders to ensure that the training needs of the region are identified and met.

The Mid-America PTTC goals are to:
  • Accelerate the adoption and implementation of evidence-based and promising substance misuse prevention strategies.
  • Heighten the awareness, knowledge, and skills of the workforce that addresses substance misuse prevention.
  • Foster regional and national alliances among culturally diverse practitioners, researchers, policymakers, funders, and the local communities.

To learn more about our services:  Mid-America PTTC
Epi Corner
Iris E. Smith, Ph.D., M.P.H.

The Fifth Step of the SPF: Evaluation


Although Evaluation is the fifth step in the SPF, it should never be considered as the last step. In actuality, evaluation has a role to play throughout the SPF Process. The SPF represents a cyclical and ongoing process. Evaluation results are used to inform future program planning and development. Evaluation has been defined as:
 
The systematic collection of information about the activities, characteristics and outcomes of programs to make judgements about the program, improve program effectiveness, and inform decisions about future programming. 1 ( M.Q. Patton, 1997; pg. 23)
 
A carefully thought out evaluation plan enables communities to document what they have accomplished; understand what worked, what did not work and why; and provide useful information to stakeholders, who may need to make decisions about the program or implemented strategies in order to make policy, funding, and other decisions. Evaluation findings can be used to recruit participants, seek funding, influence policy makers, and build credibility and community support for the program. Evaluation data should be collected at multiple time points during the life of a program and should be considered a dynamic process rather than an endpoint activity. 
 
Process or formative evaluation data is collected during the implementation process. Process evaluation answers the question: Was the program or activity implemented as intended (fidelity)? 
 
Outcome or impact evaluation data is usually collected after the program has been fully implemented or at the end of the program. Outcome evaluation data documents the changes that occurred as a result of the program/activity.
 
 In general, evaluation data should answer the following questions at each step of the SPF:
 
  • Capacity building: Based on the original needs assessment and capacity building plan, were the activities implemented as intended? Was there a change in the community and/or organizational capacity as a result? Which community sectors were engaged, and which were not? What challenges were encountered and how were they resolved?
  • Planning:  Were planning activities successful in engaging relevant stakeholders? Did the planning process go as planned? What worked, what did not? What information will be needed by program stakeholders to demonstrate to stakeholders that the program has been successful, and how will it be collected?
  • Implementation: Are program activities being carried out as planned or as intended based on the developer’s guidelines? Is the program reaching and engaging the segments of the community identified in the plan? What is working well and what is not? What challenges were encountered and what strategies are being used to address them?
  • Evaluation:  In addition to a synthesis of evaluation data collected in the preceding SPF steps, Step 5 focuses on answering the question: Did the program achieve its intended outcomes? Was there a change in the risk/protective factors that were the focus of the prevention plan? Were the changes consistent with what was anticipated? Were there any unanticipated changes that occurred as a result of the program/strategy?

Additional Resources
 
American Evaluation Association The American Evaluation Association is a professional association of evaluators devoted to the application and exploration of program evaluation, personnel evaluation, technology, and many other forms of evaluation. Evaluation involves assessing the strengths and weaknesses of programs, policies, personnel, products, and organizations to improve their effectiveness. AEA has approximately 7300 members representing all 50 states in the United States as well as over 80 foreign countries. Highly recommended:  List of online handbooks and texts .
 
Betterevaluation.org :   BetterEvaluation is an international collaboration to improve the practice and theory of evaluation. The BetterEvaluation team creates and curates information on choosing and using evaluation methods and processes, including managing evaluations and strengthening evaluation capacity.
 
Publications
 
Joint Committee on Standards for Educational Evaluation (2010). The Program Evaluation Standards: A Guide for Evaluators and Evaluation Users. Sage Publications; Thousand Oaks, CA.
 
Patton, MQ (2008). Utilization-Focused Evaluation, 4 th Edition. Sage Publications; Thousand Oaks, CA.
 
Mertens DM & Wilson AT (2019). Program Evaluation Theory and Practice, Second Edition. The Guilford Press; New York, NY.

_____________________________

1 MQ Patton. (1997) Utilization-Focused Evaluation The New Century Text, Sage Publications: Thousand Oaks, CA.
CONTACT US
573-635-6669