Problem Solving/Response to Intervention Evaluation Tool Technical Assistance Manual - Revised
Year: 2010
Type: article
Abstract: Problem Solving/Response to Intervention Evaluation Tool Technical Assistance Manual How to Use This ManualProgram evaluation of Problem Solving/Response to Intervention (PS/RtI) initiatives is a critical component of facilitating successful implementation.Complex educational systems require that key stakeholders take a systems view of facilitating change and develop plans to address variables likely to relate to successful implementation.Educators' knowledge and skills; school, district, and state policies and procedures; funding streams; and myriad other factors will likely impact whether educators will adopt PS/RtI practices.Although a comprehensive strategic plan designed to address these systemic factors is a necessary condition for successful implementation, it is not sufficient by itself.Formative data-based evaluation of needs within the educational system and the impact of any actions taken should be used to guide the development of (and modifications to) implementation plans.Key stakeholders who engage in this type of formative decision-making can focus more intensely on identified issues, thus responding to the specific needs of educators and the systems in which they operate.The development of a model to evaluate efforts to scale up PS/RtI implementation, however, poses several challenges.Questions about what issues to focus on, what tools to use, and how often to collect data, among others, can be difficult to address.It is with these difficulties in mind that the Florida Problem Solving/Response to Intervention Project created this technical assistance manual.Project staff have developed or adapted a number of tools designed to assist educational stakeholders in evaluating which systemic factors contribute to and/or hinder implementation of PS/RtI practices.Importantly, these tools align with the three stage systems change model outlined by the NASDSE RtI Implementation Blueprints (Elliott & Morrison, 2008;Kurns & Tilly, 2008).Progress can be evaluated toward (1) developing consensus among educational stakeholders regarding implementing PS/RtI, (2) developing the infrastructure necessary to support implementation, and (3) implementation of PS/RtI practices.The Project has been using data obtained from instruments administered in pilot schools implementing PS/RtI to inform scale-up across Florida.The purpose of this manual is to provide information about Project tools to educational stakeholders interested in using the instruments to inform PS/RtI implementation.Each chapter of the manual highlights a specific tool created to provide data on consensus development, infrastructure building, and/or implementation.A summary of the information available on each instrument follows.• Description & Purpose of the Instrument: Theoretical background, description of the instrument, and its intended use • Intended Audience: Suggestions for who should complete the instrument and who should use the results for decision-making • Directions for Administration: Strategies for administering or completing the instrument and examples of ways in which Project staff approached administration • Frequency of Use: Considerations when determining how often to use the instrument and general guidelines for frequency of use • Technical Adequacy: Available information on the reliability and validity of the instrument • Scoring: Strategies for summarizing data for decision-making • Training Required: Suggestions for training of individuals responsible for (1) administering or completing the instrument and (2) analyzing and interpreting the results • Interpretation and Use of Data: Suggestions for analyzing, displaying, and interpreting results • School-Level Example of Instrument Use: Examples of how data could be collected, displayed, and used to guide decisions made at the school-level ii How to Use This Manual Problem Solving/Response to Intervention Evaluation Tool Technical Assistance Manual Educational stakeholders involved in program evaluation of PS/RtI initiatives will have a number of factors influence decisions regarding what data collection tools and methods to use.Factors such as the specific evaluation questions asked; the time, personnel, and financial resources available to dedicate to program evaluation; and existing data collection requirements will undoubtedly play a role in the design and implementation of an evaluation plan.The information included in each section of this manual is intended to assist stakeholders in making decisions about how to evaluate scaling-up of the PS/RtI model and adapt the use of any relevant instruments to their specific circumstances.In other words, this manual is not intended to describe how stakeholders in schools, districts, or other educational agencies should pursue program evaluation efforts.Rather, the manual is intended to be a resource to stakeholders in the position of evaluating PS/RtI implementation.Potential users of this manual include all educational stakeholders facilitating the implementation and evaluation of PS/RtI practices.Specifically, the contents of this manual can assist school-, district-, and state-level personnel as well as stakeholders from other educational organizations (e.g., universities, Area Education Agencies) in their efforts to make informed decisions regarding PS/RtI implementation and its impact on important educational outcomes.To facilitate clear, concise communication of the information presented, each section describes use of the instrument at the school-and district-levels.Educational stakeholders from other units of analysis or entities can adapt the recommendations to meet their specific needs. iii Problem Solving/Response to Intervention Evaluation Tool Technical Assistance ManualWhat's New in This Version?Publication of the first iteration of this manual occurred during the Summer of 2010.Subsequently, the context in which educational stakeholders are implementing Problem Solving/Response to Intervention (PS/RtI) has changed (e.g., language used, federal and state policies, funding issues, advances in the field).Research further investigating the technical adequacy of several instruments contained within this manual has been conducted as well.Therefore, we have updated the manual to provide the most up-to-date information available regarding the technical adequacy of Project instrumentation and the context within which they should be used. Updates to the manual occurred primarily in four sections; the Introduction and chapters addressing the Beliefs on RtI Survey (formally called the Beliefs Survey), Perceptions of RtI Skills Survey, and the Coaching Evaluation Survey.We updated the Introduction to include more contemporary information on PS/RtI, systems change, and the variables measured by Project instrumentation.Changes to the chapters covering each of the three aforementioned instruments involved updating literature on the rationale for using the tool, additional data on its technical adequacy, and updating school-and district-level exemplars to reflect changes in the tools that resulted from psychometric analyses.Further analysis of their technical adequacy resulted in the elimination of several items from each instrument.Therefore, copies of the instruments that reflect the new item sets also are included. Problem Solving/Response to Intervention Evaluation Tool Technical Assistance Manual Introduction -Problem Solving/Response to Intervention and Data-Based Systems Change 1 Problem Solving/Response to Intervention Evaluation Tool Technical Assistance Manual68% of school districts surveyed indicated that they have fully implemented or are in the process of district-wide implementation of RtI.• Tier I instruction involves providing scientific, research-based instruction to all students (i.e., core instruction).Educators administer universal screening assessments three to four times per year and examine existing data to determine the overall impact of Tier I instruction, and screen for individual students not responding to the curriculum.• Tier II intervention (i.e., supplemental intervention) involves additional time and/or skill focus in the curriculum for students identified as at-risk through universal screening and other available information.Students receiving Tier II interventions are monitored more frequently (e.g., monthly) to facilitate decision-making regarding the effectiveness of the intervention plan developed through the problem-solving process.Although the majority of students should respond to Tier I and II instruction, estimates indicate that approximately 5% will require more intensive, targeted interventions available through Tier III services.• Tier III interventions typically involve highly idiosyncratic, intensive services that require the expertise of a diverse team of trained individuals.Educators monitor progress frequently (e.g., weekly) to make decisions regarding student RtI.Interventions developed for students receiving Tier III services may or may not involve resources outside of what can be realistical-Tiered Model of School Supports & the Problem-Solving Process ACADEMIC and BEHAVIOR SYSTEMS Tier 3: Intensive, Individualized, Interventions.Individual or small group intervention.Tier 2: Targeted, Strategic Interventions & Supports.More targeted interventions and supplemental support in addition to the core curriculum and school-wide positive behavior program.Tier 1: Core, Universal Instruction & Supports.General instruction and support provided to all students in all settings.Revised 10.07.09Figure 1.Problem Solving/Response to Intervention (PS/RtI) Model Diagram.ly expected in the general education setting.When the resources (e.g., time, materials, personnel) required exceed what is available through general education, then the student is considered for special education eligibility.Thus, in the PS/RtI model, special education becomes a mechanism for providing additional, intensive services to
Institution: University of South Florida
Language: English
Subfield: Developmental and Educational Psychology
Field: Psychology
Domain: Social Sciences
SDG: Quality education
Open Access status: green
OpenAlex ID: W1586904017