Blog 007: Educational Program Code Basics and The Importance of Using Them Correctly to Maintain Data Integrity

Educational program codes are used by the NRBs, NCLEX^® candidates, educational programs, NCSBN and Mountain Measurement. The NRBs use them to identify and track programs as part of their daily efforts to regulate nursing education within their jurisdiction. NCLEX^® candidates use them when completing their application for licensure.  Educational programs use program codes to identify which NCLEX^® Program Reports to order. And, NCSBN and Mountain Measurement use educational program codes to identify and aggregate the data from the students graduating from each educational program when producing quarterly reports of program passing rates and the NCLEX^® Program Reports.

Educational Program Code Structure

An educational program code is structured as follows:

 US80507700

Figure 1. Description of the separate elements of an educational program code.

Figure 2. Example application for licensure with a field for educational program code.

Reasons for Educational Program Code Anomalies

Nursing graduates use the educational program codes to indicate the program from which they graduated when completing their application for licensure. Given that the data is entered manually by candidates completing paper applications, some candidates make mistakes when entering this information on their licensure application. As seen in Figure 2, online licensure applications often include the educational program code, the name of the educational institution and the nursing degree type. Even if students do not remember their educational program code, they should be able to recognize their institution name, as long as it is correct and up to date. By including the name of the institution, degree type and program code in the user interface, the online application prevents errors by making it is easier for candidates to enter the data correctly. Therefore, Mountain Measurement recommends that programs encourage their students to use online licensure and NCLEX^® application processes rather than paper applications.

Detecting & Rectifying Anomalies

In an effort to detect and rectify education program code mistakes, the individual NRBs validate the quality of these data as part of their routine data-processing efforts. Prior to approving a candidate to test, the individual NRBs review all licensure application data both manually and automatically to verify completeness and accuracy, and the dean for each program verifies that each graduate has fulfilled graduation requirements. Finally, Mountain Measurement runs quality control procedures on the educational program codes contained in the test results data.

Despite these extensive quality control efforts, program code anomalies still occur. These anomalies are usually detected by educational program directors when reviewing their program’s quarterly results. The results are provided to programs by their local NRB often as a PDF Report produced by Pearson VUE and NCSBN entitled “Jurisdiction Program Summary of all First-Time Candidates Licensed in All Jurisdictions”. When a program discovers an anomaly, it is important that they notify their local NRB as soon as possible. In January 2020, NCSBN implemented a new, streamlined process for making program code corrections. Once an NRB is notified of and validates a requested correction, they correct the program code on the Candidate Registration Page within the NCLEX Administration Website. If the candidate's education program is located in a different jurisdiction than the NRB that made the candidate eligible to test, the NRB where the candidate's education program is located would request a code change directly to the NRB that made the candidate eligible by completing a Candidate Level Change Request Form. For more information on the changes to the educational program code correction process, please visit the NCSBN program code pages. If a change is made prior to the completion of the data validation phase of NCLEX^® Program Reports production, then the change is also made in Mountain Measurement’s systems. However, once Mountain Measurement completes the data validation phase of NCLEX^® Program Reports production, which occurs approximately 1 week after the reporting period ends, it is impossible to make any further changes to the data being analyzed to produce the NCLEX^® Program Reports. This is because a change to one program’s data can impact the numeric values provided to all programs. For example, if the educational program code for one failing candidate is corrected, then the passing rate for two programs could change resulting in incorrect ranks for all of the programs reported in the Summary Overview Section (Pages 3.2 to 3.5) of the NCLEX^® Program Reports.

It is important to recognize the difference between the two types of mistakes that a candidate can make when entering their program code on their licensure application. The first mistake is a valid but incorrect program code. This mistake happens when a student enters a valid educational program code but enters one from a program different than the one from which they graduated. The licensure application processing systems often fail to detect these anomalies which results in the student’s NCLEX^® exam results being associated with a different program than the one from which they graduated. 

Figure 3. The Impact of a graduate using a valid but incorrect code

The other type of mistake that a candidate can make is the use of an invalid code. Invalid codes are generally detected by the NRBs or Mountain Measurement before the data is processed for the NCLEX Program Reports. With the increased use of online applications, where dropdown lists restrict data entry to solely valid codes, invalid code mistakes are thankfully becoming rarer. ⁽¹⁾

The Impact of Educational Program Code Anomalies on the values (numbers) published in the NCLEX^® Program Reports

Each quarter, Mountain Measurement runs analyses on the NCLEX^® Examination data, aggregating the test results of the candidates associated with each educational program code to produce one set of NCLEX^® Program Reports for each nursing program. Due to the relationship between the educational program codes and the NCLEX^® Program Reports, it is vital for graduates to enter their educational program code accurately on their application for licensure.

As stated earlier, a graduate may accidentally enter an incorrect educational program code on their application for licensure. When this happens, the graduate’s testing data are associated with a different program, which means that their program will receive a set of NCLEX^® Program Reports based on a dataset that is incomplete.

Having an incomplete dataset may or may not impact the validity of the values provided on the NCLEX^® Program Reports. Less than 50 anomalous program codes (0.025%) are detected for first-time test takers each year. Of these, rarely do more than one or two students from a single program provide incorrect program codes. As a result, the sample of data from each of the impacted programs still represents a very high percentage of the available data thereby giving the sample a very high probability of being representative. When the sample is representative, then sampling theory suggests that it does not matter whether or not the dataset is complete as the numeric values provided will be representative of the true performance of the program.

Despite the evidence from sampling theory to support the validity of anomalously coded NCLEX^® Program Reports data, even the best sample of data is not the same as a complete dataset. Although this is certainly true, the actual numeric values provided on the NCLEX Program Reports may or may not change as a result of one or two program code anomalies. For larger programs, a single missing graduate represents far less than 1% of the data, meaning that it is highly unlikely that the numeric values provided in the NCLEX^® Program Reports would change at all. However, for smaller programs, this may not be the case. For example, if 12 graduates from a program completed the NCLEX^® during a reporting period, each graduate represents over 8% of the data. If the missing graduate was the highest or lowest-performing student, then it is possible that some of the numeric values in the Reports would change. Though values in a large data set may change when a small number of program code anomalies are present, the sample is still representative of the complete data and any conclusions derived from that sample are valid.

Although it is unlikely that educational program code anomalies would compromise the validity of an edition of the NCLEX Program Reports, anomalies are certainly undesirable and prompt the uninformed to question the credibility of the data. Therefore, Mountain Measurement encourages nursing educators to take action to prevent educational program code anomalies from happening.

How do I Avoid Educational Program Code Anomalies?

Knowledge

To reduce these anomalies, all educators should know the official name(s) of their institution, the name(s) of their nursing program(s), and their educational program code(s). It is also important to share the name(s) and code(s) with their graduates. It is helpful to remind the graduates to put the correct name and educational program code on their application for licensure and their NCLEX^® registration.

Keep School-level Program Information Up-to-date

If a program(s) experiences a change, such as a change to the name of its host institution or programs or to the type of nursing programs offered by the school, it is important for the program to ensure that this information is current and accurate. This makes it easy for graduates to recognize the school and program name in the online licensure application or educational program list provided by NCSBN.

Unfortunately, Mountain Measurement cannot make changes to the school or program name. To make those changes, a program must contact their local NRB and follow their change process. Once the NRB has updated a program’s information in their system, it is the NRB’s responsibility to submit an “NCLEX Program Code Change Form” to [email protected] to ensure the updates are made in the NCSBN database. Occasionally, an NRB may fail to complete this step of the process resulting in the name remaining unchanged in NCSBN’s database, the published list, and the NCLEX^® Program Reports. After a program has completed the name change process with their local NRB, Mountain Measurement recommends that the program monitor NCSBN’s published list of educational program codes. If the name has not changed on this list within 90 days of the NRB reporting that the change has been made to the system, Mountain Measurement recommends that the program contact the NRB and remind them to contact NCSBN to update the program’s information in the NCSBN database.

The diagram below shows the basic process for updating an educational program code. As mentioned earlier, this process may vary slightly from NRB to NRB. Therefore, Mountain Measurement recommends that programs review the rules and regulations provided by their respective NRB whenever their program experiences change.

Figure 4. The Process for Changing Educational Program Information

Review Program Results Reports in Timely Manner

During the first two weeks of the quarter, program directors should be on the lookout for the quarterly Jurisdiction Program Summary of all First-Time Candidates, provided by your local NRB. This report, despite the vagueness of the name, contains a record of all students in the program who have taken the exam, for the first time, during the reporting period. Upon arrival, program directors should review and validate the accuracy of the information contained in the report. To validate the information, program directors should compare the total number of first-time graduates reported to the number of students that they expect to have completed the NCLEX during the quarter. If the totals do not match, then program directors should contact their local NRB, obtain a report providing the roster of candidates that completed the NCLEX during the reporting period, and identify any missing or unexpected students. Corrections are then submitted to the program’s local NRB. It is crucial that program directors submit any corrections within 7-10 days of receipt of the quarterly Jurisdiction Program Summary of all First-Time Candidates to ensure these changes are included in the data submitted to and used by Mountain Measurement for the NCLEX Program Reports.

Where do I go from here?

As outlined in this article, educational program codes serve as an identifier for a nursing program(s) and link a graduate’s NCLEX Exam results with their nursing program. The article provided several strategies to help educators prevent educational program code anomalies and maximize the quality of data being provided by the NCLEX Program Reports.

Educational program codes can additionally be used strategically to delineate a programs’ graduates in meaningful ways for conducting program evaluation. Mountain Measurement recommends that program directors serving at institutions with more than one nursing program review the upcoming article on educational program codes, due out in Spring 2020, detailing how programs can leverage educational program codes to maximize the usefulness of the NCLEX^® Program Reports.

Special thanks to Dr. Nancy Irland, Policy Analyst, Nursing Education and Assessment with The Oregon State Board of Nursing as well as Wei Xu, Psychometrician and Shu-chuan Kao, Manager of Measurement and Testing, Examinations at NCSBN for their assistance with this article.

¹ After a program closes, it’s program code will remain active for 3 years to allow candidates to test out. Although the vast majority of candidates complete the NCLEX within 3 years of a program's closing, it is possible for a candidate graduating from a closed to program to enter an invalid code, the one corresponding to the program from which they graduated.