CAEP Annual Reporting Measures

Impact Measure #1: Completer effectiveness

As part of its CAEP accreditation, Relay engaged in a study on the impact of its completers on PK-12 learning and development and benchmarked its completers’ performance against other teachers working in the same school district. Relay found that its completers had statistically significant higher average value added scores in math in 2016, 2017 and 2018 in NYC. Among teachers working in the same district who did not attend Relay, their average value-added score was -.01 in all three years compared with Relay completers’ scores that were .41, .66, and .41 in each of the three years, respectively. In ELA, Relay completers’ growth was greater than those of non-Relay teachers in 2016 and 2017 but not 2018; however, these findings across all three years were not statistically significant. When examining average ELA scores across the three years, Relay completers were able to bring their students to average expected growth in ELA with a weighted average value-added score of .05 (value-added scores of 0 or above indicate average expected growth).

‍

Relay also engaged in a study of the teaching effectiveness of a sample of its completers using the Classroom Assessment Scoring System (CLASS) observation instrument. CLASS was designed by the University of Virginia’s Center for Advanced Studies in Teaching and Learning (CASTL) to assess the quality of teacher-student interactions that are associated with positive academic outcomes for PK-12 students. CLASS covers three domains: emotional support, classroom organization, and instructional support. These domains are further broken out into multiple dimensions. CASTL has validated the CLASS observation instrument in over 2,000 classrooms, and the observation can only be conducted by a certified CLASS observer who has completed CLASS inter-rater reliability training.

‍

Relay found that the completers included in its study are applying the knowledge, skills, and dispositions that its program is designed to impact. Specifically, Relay’s completers in K-3, upper elementary, and secondary classrooms outperformed teachers in comparison studies on nearly all CLASS dimensions.

‍

Tables 1 and 2 benchmark  Relay completer CLASS scores alongside CLASS scores collected from previous studies conducted by CASTL. Table 1 shows that on the K-3 CLASS rubric, Relay completer scores were comparable or higher than scores from the comparison studies for most CLASS dimensions. In particular, Relay completers outperformed teachers in the comparison studies on the Positive Climate, Teacher Sensitivity, Behavior Management, and Productivity dimensions. Table 2 shows that on the upper elementary and secondary CLASS rubric, Relay completer scores were higher or comparable than teacher scores in the comparison studies on every CLASS dimension.

‍

Table 1: CLASS Scores on K-3 Rubric Across Four Studies

‍

‍

Table 2: CLASS Scores on Upper Elementary and Secondary Rubrics Across Four Studies

Impact Measure #2: Satisfaction of employers and stakeholder involvement

Initial Level Programs

Findings from Relay’s 2022 Employer Survey demonstrate that the majority of principals are satisfied with the initial-level preparation their teachers received at Relay.

‍

1) 98% of surveyed principals responded that completers were moderately, mostly, or fully ready to meet the needs of students after completing their studies with Relay, with 85% stating that completers were fully or mostly ready.

‍

2) 89% of surveyed principals believe that completers contributed to an expected level of student learning growth in their classrooms.

‍

3) Principals provided the following feedback on specific aspects of completers’ performance in the classroom:

• 83% of principals responded that completers were “Very Well” or “Well” prepared to implement well-structured lessons.
• 87% of principals responded that completers were “Very Well” or “Well” prepared to maintain an academic learning environment where students are unafraid to academic risks.
• 81% of principals responded that completers were “Very Well” or “Well” prepared to make adjustments to practice based on assessment data.
• 77% of principals responded that completers were “Very Well” or “Well” prepared to meet the diverse needs of learners within the classroom including English Language Learners and students with special needs.
• 81% of principals responded that completers were “Very Well” or “Well” prepared to use self-reflection to improve practice.

Advanced Level Programs

Findings from Relay’s 2020 Employer Survey demonstrate that the majority of principals are satisfied with the advanced-level preparation their teachers received at Relay. The response rate from employers of advanced-level completers on the 2021 Employer Survey is too small to report any findings. Relay operates only one advanced level program with a small enrollment that leads to recommendation for special education certification. We are strategizing on how we can do more targeted outreach to improve the advanced-level response rate on future Employer Surveys.

‍

1) 82% of surveyed principals responded that completers were mostly or fully ready to meet the needs of students after completing their studies with Relay.

‍

2) 73% of surveyed principals believe that completers contributed to an expected level of student learning growth in their classrooms.

‍

3) Principals provided the following feedback on specific aspects of completers’ performance in the classroom:

• 73% of principals responded that completers were “Very Well” or “Well” prepared to consistently enforce high expectations for all students.
• 100% of principals responded that completers were “Very Well” or “Well” prepared to implement well-structured lessons.
• 73% of principals responded that completers were “Very Well” or “Well” prepared to maintain an academic learning environment where students are unafraid to academic risks.
• 82% of principals responded that completers were “Very Well” or “Well” prepared to make adjustments to practice based on assessment data.
• 80% of principals responded that completers were “Very Well” or “Well” prepared to meet the diverse needs of learners within the classroom including English Language Learners and students with special needs.
  

Impact Measure #3: Candidate competency at completion

Relay’s 2021-22 certification pass rates, by state, are provided in the tables below. These pass rates are based on candidates’ best attempt of program completers.

‍

‍