Blog 022: 2024 Summer Program Evaluation Reminder

August 22, 2024

Although there are many opportunities for nurse educators to conduct program evaluation activities throughout the year, summer is a time when many nurse educators engage in more program evaluation activities than usual. Many reflect on their accomplishments during the academic year and begin envisioning how they'll improve their program. For those that teach courses, summer may be a good time to address individual course improvements while program leadership may analyze the curriculum, assessment strategies, or program policies.

Mountain Measurement would like to invite educators to consider taking the time to review the performance of their graduates by heading to the NCSBN Performance Reports Website and opening up their program's NCLEX^® Program Reports. Although the performance of your program's graduates during the q1, spring, and annual reporting periods became available on May 1, 2024, May is often such a busy time that many educators don't have the opportunity to login and analyze the performance of their graduates until the summer. Mountain Measurement would like to remind educators that have not yet taken the time to review the performance of their graduates testing during these reporting periods to do so at this time.

Note that the most recent annual reporting period for which data are currently available covers graduates testing between April 1, 2023 and March 31, 2024 while the most recent semi-annual reporting period covers graduates testing between October 1, 2023 and March 31, 2024. This means that, for many programs, the most recent graduating cohort for which data are available is the winter (or December) 2023 cohort and the most recent spring cohort is the 2024 spring cohort.

Given that the quarter for which most RN nursing students graduate is the spring, the number of graduates completing the NCLEX-RN^® Examination is highest during the second and third quarter of the calendar year. In the past, many programs have not been able to access the performance of their spring cohort until November when the data pertaining to the fall reporting period become available. With the transition to the Next Generation NCLEX^® Program Reports, many programs now have access to the performance of their graduates on a quarterly basis. On August 2, the performance of many students that graduated with the spring 2024 cohort became available.

Figure 1.

Mountain Measurement would like to encourage educators to spend some time in August analyzing the q2 performance of their program's graduates. Keep in mind that the NCLEX^® Program Reports provide data on only first-time test takers, so the vast majority of candidates that test during q2 graduated in the spring. Although many spring cohort graduates test during q2, many others do not take the NCLEX^® Examination until q3. Therefore, Mountain Measurement recommends that programs interested in understanding the performance of their spring cohort should view the q2 estimates of performance as a preliminary snapshot of the performance of their spring cohort.

It is important to note that the q2 sample will be smaller than the fall sample for most programs. As a result, the q2 estimates of your graduates’ performance are less precise and less stable than the fall estimates which is also true of the estimates of the performance of the reference groups (jurisdiction, program type, and nation).

Figure 2.

Despite these limitations, Mountain Measurement encourages educators to conduct a complete analysis of the performance of their program's q2 performance by identifying the topics for which their program has the largest opportunity for improvement. Since, the results are preliminary, Mountain Measurement does not encourage programs to make any changes to their program using solely these results. Rather, the topics identified by the q2 estimates of performance should be monitored more closely as summer turns into fall, especially those topics that were also identified using the 2023-2024 reporting periods. Educators that teach courses should be aware of those topics which were identified as opportunities for improvement that most pertain to the courses that they teach. Educators in leadership positions should be prepared to triangulate these results with the results obtained from other assessment sources. Ultimately, the q2 analyses should be revisited in November when the fall estimates of performance are available.

Educators that take the time during the summer to engage in this type of program evaluation activity are exemplifying the spirit of data-driven education. As with most things in science, those that use empirical data to make change are likely to succeed.