TASN Evaluation

As outlined in the TASN Vision and Mission, the purpose of the Kansas Technical Assistance System Network (TASN), as a system, is to increase the capacity of districts to implement and sustain the use of evidence-based practices addressing priority areas and practices identified by Kansas State Department of Education Special Education and Title Services (KSDE SETS). This professional learning and technical assistance system comes out of federal requirements of the Elementary and Secondary Education Act (ESEA) and the Individuals with Disability Education Act (IDEA), which both require the state education agency to provide direct support to districts addressing these special populations.

The TASN Theory of Change guides all evaluation activities. Following utilization-focused evaluation guidelines, each TASN project develops, in collaboration with the TASN Evaluation team, a logic model and evaluation plan. Each project-specific evaluation plan aligns with the Overarching TASN Logic Model. The evaluation plans are then implemented to determine the impact of the TASN network as a whole and of each project separately. Annual evaluation briefs highlight data related to outputs, outcomes, and impacts.

TASN initiatives adhere to implementation science research. Implementation science drivers include selection, training, coaching, facilitative administration, and performance assessment. Each project, together with KSDE and the Coordination and Evaluation teams, works to continually improve Evidence-Based Professional Development Implementation Practices.

TASN professional development is evaluated through a structured process. TASN aims to provide training and coaching that meets the needs of adult learners and builds the knowledge and skills of participants so that they can implement evidence-based practices which improve outcomes for students. The evaluation includes the following:

1. All TASN projects adhere to the Standard Operating Principles and Procedures. The principles and procedures provide guidelines that maintain the vision of TASN.
2. Each TASN project aligns with and promotes the vision, mission, and goals of the Kansas State Board of Education, including Kansas Education Systems Accreditation (KESA).
3. All TASN trainings are designed to address specific learning targets. These learning targets are provided in the session description so participants know what they will learn before registering for the training. Guidance is provided to TASN providers on Establishing Learning Targets.
4. The TASN Training Evaluation asks participants to provide feedback on the quality of the training and their next steps in implementation. TASN providers can also add customized items. Participants are encouraged to complete the training evaluation, as this information is used to improve training content and follow-up support.
5. TASN trainings include performance-based assessments or pre/post-assessment of knowledge. Aggregate data from these assessments are used to determine the effectiveness of each training in supporting participants to learn evidence-based practices. These assessments, with corrective feedback, also support the participants to reflect on learning.
6. To ensure that TASN trainings are skill based and address adult learning principles, the Observation Checklist for High-Quality Professional Development (HQPD Checklist) is referenced when developing trainings. The presenters also self-rate, and the evaluation team observes numerous trainings to monitor quality and promote continual improvement.
7. To support implementation, the majority of trainings are part of a series and include ongoing coaching. The Coaching & Facilitation Reflection promotes the continual improvement of coaching practices. TASN Evaluation observes numerous coaching sessions to support TASN providers in prioritizing and continually improving their coaching practices.
8. Initiative-specific fidelity measures and practice profiles provide participants with criteria for effective implementation of the practices. Examples include the Inclusive MTSS Implementation Scale, available on DistrictTools.org, and the Instructional Observation Tool, available on Observations.ksdetasn.org. Stakeholder input and feedback is necessary for informed decision making at the local level and statewide. Examples include the Family Engagement Survey and the Educate Kansas Recruitment and Retention Survey, both available on DistrictTools.org. For each of these fidelity and stakeholder measures, data summaries are automatically generated and available to district implementation teams to guide their continual improvement. Data across districts are aggregated to continually improve the implementation support provided through TASN.
9. The impact of implementation on students is evaluated following the project-specific evaluation plans. Short-term impacts are prioritized to support continual improvement of professional learning and implementation supports. Following utilization-focused evaluation principles, data are provided, when possible in real time, to educators and school leadership to inform their Plan–Do–Study–Act cycles. TASN providers and KSDE utilize aggregated data to inform professional learning and structures that promote implementation.