If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
The COVID-19 pandemic has had a profound impact on prelicensure nursing education, leading to widespread disruptions that may have implications for nursing students’ learning and engagement outcomes. Understanding how the rapid shift to online and simulation-based teaching methods has affected new graduates’ clinical preparedness is critical to ensure patient safety moving forward.
Purpose
To assess the impact of institutional, academic, and demographic characteristics on prelicensure nursing students’ academic, initial postgraduation, and early career outcomes during the COVID-19 pandemic.
Methods
We conducted a mixed-methods longitudinal study focused on prelicensure registered nurse (RN) students entering the core of their didactic and clinical nursing coursework during the pandemic. This study uses a combination of real-time student and faculty self-report data, including externally validated instruments, within and end-of-program standardized test scores, and focus group findings. Various statistical methods, ranging from simpler descriptive and non-parametric methods to Generalized Estimating Equation (GEE) models and detailed textual analysis, are applied to assess student, faculty, and institution-level data.
Results
The final sample includes more than 1,100 student and faculty participants affiliated with 51 prelicensure RN programs located across 27 states. Leveraging more than 4,000 course observations collected from fall 2020 to spring 2022 and supplemented by the rich personal narratives of over 60 focus group participants, this study illuminates the breadth, scale, and ever-evolving nature of prelicensure RN programs’ efforts to maintain the continuity of nursing students’ education during the public health crisis. In doing so, it captures the many ways in which nursing administrators, faculty, and students sought to address the unparalleled challenges they confronted on a day-to-day basis. In particular, the findings provide critical insights into the efficacy of the changes nursing programs made to their course delivery formats to adjust to the confluence of rapidly evolving federal, state, and private restrictions to stem the spread of COVID-19.
Conclusion
This study stands as the most comprehensive assessment of prelicensure nursing education in the United States since the onset of COVID-19. It extends knowledge by linking potential deficiencies in students’ didactic and clinical education during the pandemic and their early career preparedness, clinical competence, and the patient safety implications therein.
The onset of COVID-19 in the United States affected nearly every aspect of the nursing profession over the past 3 years, from an increased reliance on simulation for undergraduate clinical education to expanded scope of practice for advanced practice registered nurses (
; Martin, Buck, & Zhong, 2023). As we now enter a post-crisis phase of the pandemic, it is incumbent on researchers to identify and differentiate between the possible short- and long-term consequences of the disruptions introduced by the confluence of rapidly evolving federal, state, and private restrictions that were enacted to combat the rising tide of infection across the country. Perhaps nowhere is this more important than in prelicensure registered nurse (RN) education, which has experienced unprecedented levels of change since March 2020, both in terms of the scale and speed of the adoption of more remote models of clinical education. To inform future policy decisions, it is essential that we learn from this public health crisis. The empirical evidence from this timeframe can provide important insights into the creation of more resilient educational models and health systems now and in the event of another emergency.
Since the early 2000s, the rapid expansion of prelicensure RN programs across the United States has made securing in-person clinical placements and qualified preceptors more difficult (
). To address these shortfalls, many programs have opted to substitute a proportion of their traditional in-person clinical placements for simulation-based experiences (SBE). In the past decade, nursing students using face-to-face SBEs under specific conditions have demonstrated learning outcomes comparable to those of students participating in traditional in-person clinical placements (Hayden et al., 2014). Furthermore, SBE students have even achieved better marks in specific knowledge areas than their traditional learner counterparts (
Using simulation with nursing students to promote affirmative practice toward the lesbian, gay, bisexual, and transgender population: A multisite study.
). Many of these studies, however, were based on the assumption that certain quality control measures would be standardized, such as the use of experienced faculty to conduct face-to-face simulation at certain preset thresholds and within the context of a minimum number of prescribed clinical hours. Despite the increased use of SBE and the research associated with it, the quality of prelicensure RN students’ educational experiences through integration of SBE has still been questioned.
The National Council of State Boards of Nursing (NCSBN) recognized the destabilizing potential of the pandemic and thus set out in April 2020 to design a study to longitudinally track and analyze the learning and engagement outcomes of prelicensure RN students under significantly less controlled conditions. Out of necessity, and often in response to clinical site restrictions prohibiting nursing students from entering facilities, education programs were forced to quickly pivot their course delivery methods, often with no additional training or resource support. Prelicensure programs that never relied on any form of SBE were now shifting 25% to 50% of their in-person clinical hours to simulation-based learning environments. Additionally, programs that had long employed face-to-face simulation were now exploring new modes of virtual simulation. Layered on top of this shifting landscape was the patchwork of public health guidance and restrictions that frequently varied significantly in terms of scope and duration by region, including at the state and local community levels. Through a combination of real-time student and faculty data collection using externally validated instruments and end-of-program standardized test scores, the NCSBN endeavored to identify the range of programmatic changes across the country and, the implications of these changes for new graduates’ early career practice and ultimately patient safety.
This important longitudinal study captures both the breadth and scale of prelicensure RN programs’ early and sustained changes to their course delivery formats to ensure some level of continuity in students’ education during the COVID-19 pandemic. As a natural experiment, this study benefits from these programs’ evolving responses to the pandemic within their local contexts and documents the range of strategies employed. Thus, the results serve as a marketplace of ideas in a manner that allowed NCSBN researchers an opportunity to identify organic trends that emerged from the empirical evidence itself and thereby derive insight based singularly on the outcomes achieved by the programs and students who participated. While not the first of its kind, the scope and rigorous design of this study illuminate the many innovative ways prelicensure RN programs sought to address the nearly unparalleled challenges they confronted on a day-to-day basis over the past 3 years. Furthermore, it provides the mechanisms for measuring the efficacy of these strategies. Most importantly, it extends knowledge to establish clear links with how potential deficiencies in students’ clinical education impacted their early career preparedness and clinical competence and the implications of any potential deficiencies for patient safety.
It has long been speculated and even anecdotally documented that the disruptions to traditional models of teaching and clinical training wrought by COVID-19 inevitably affected students’ learning and engagement outcomes. To better understand how they were affected and to quantify to what extent the pandemic impacted prelicensure nursing students’ career preparedness, NCSBN conducted a large sample mixed-methods longitudinal study. The present report provides critical insights into four important areas:
1.
It links student outcomes and instructional delivery format
2.
It quantifies the scale of programs’ early efforts to transition to SBE and online lecture content and correlates it to standardized examination results
3.
It captures new graduates’ experiences as early career professionals
4.
It relays aspects of prelicensure students’, faculty members’, and administrators’ lived experiences throughout the pandemic.
Taking the critical insights from these four areas together, this study not only serves as one of the most comprehensive assessments of these topics to date but also paints a detailed picture of the impact of the COVID-19 pandemic on prelicensure nursing education. In doing so, it helps identify the early and sustained successes of programs’ efforts to confront the public health crisis, as well as the areas in need of improvement to ensure more resilient frameworks are in place should another crisis emerge.
), coupled with shortages in personal protective equipment in early 2020, resulted in many prelicensure nursing students being restricted from accessing clinical sites (
). While well intentioned and borne out of necessity, the effects that these policies had on nursing education programs cannot be overstated. Prelicensure RN programs were forced to quickly pivot their teaching strategies to online course delivery formats for lecture content (
noted that this shift was particularly jarring for health profession educators, as most programs were traditionally administered in-person; thus, faculty and administrators were forced to rapidly develop online and simulated curricula, frequently in a manner entirely inconsistent with their own academic training (
Existing evidence suggests that employers were already generally uneasy with the quality of new nurse graduates’ clinical preparation and preferred to hire more experienced frontline staff (
). With this trend in mind and confronted by the reality of the public health crisis, NCSBN recommended a national practice–academic partnership model in 2020. The goals of the recommended partnership were twofold: (1) to provide nursing students with meaningful in-person clinical experiences during the pandemic and (2) to offer employers a means to alleviate staffing shortages at least temporarily (
described how faculty and students at the University of Alabama at Birmingham School of Nursing provided more than 10,000 hours of hospital staffing, more than 770 worked shifts—equivalent to 30% of the supplemental staffing during surge events—and approximately 46,000 vaccine encounters. Such partnerships, though likely underutilized (
), provided critical opportunities to support and improve the clinical education of prelicensure students at a time of need. They also likely mitigated, to some extent at least, the need for extra mentoring and continuing education for recent graduates to address perceived deficiencies, further bolstering an already depleted workforce (
During this turbulent period, U.S. boards of nursing (BONs) played an essential role in supporting prelicensure nursing programs, including issuing emergency guidance on permissible SBE usage (
). Specifically, many BONs adjusted thresholds on the use of SBE to replace traditional in-person clinical experiences—either by relaxing regulations regarding the proportion of clinical hours that could be substituted with SBE or by temporarily waiving regulations entirely—to allow programs to seek accommodations that best fit their needs and local contexts (
). While nearly all prelicensure RN education programs adapted their curricula in some manner in response to clinical site restrictions, shifts to increased SBEs were most pronounced in jurisdictions that adjusted their regulations (
). Over the past 2 decades, simulation has become a critical component of nursing education, largely spurred on by increased competition for clinical placements driven by the rapid proliferation of prelicensure RN programs that began in the early 2000s (
) established the first evidence-based criteria to assist regulatory bodies in evaluating institutions employing simulation-based clinical experiences and to support prelicensure nursing programs in establishing their own curricula. The use and thereby regulation of simulation in nursing education has only increased since the publication of this landmark report (
; Smiley & Martin, in press). In addition to Hayden et al.’s (2014) important work to set evidence-based thresholds for SBE substitution, other studies have documented the strengths of using SBE to introduce concepts of cultural competence (
Using simulation with nursing students to promote affirmative practice toward the lesbian, gay, bisexual, and transgender population: A multisite study.
). As a result, long before the pandemic, SBEs had become an attractive alternative to nursing administrators and faculty when in-person clinical placements proved too difficult to arrange, largely due to limited space and/or a limited number of qualified nurse preceptors (
The NCSBN National Simulation Study: A longitudinal, randomized, controlled study replacing clinical hours with simulation in prelicensure nursing education.
In parallel to the growth of SBE, virtual clinical simulation has experienced a less pronounced but similar trajectory. As early as 2018, Aebersold described virtual clinical simulations as a “small but growing part of [prelicensure undergraduate] simulation experiences.” The term virtual simulation is often used to describe a variety of interchangeable learning modalities, including three-dimensional learning environments (
). Virtual simulation also continues to evolve with extended reality, which includes multiple modalities (augmented, virtual, and mixed realities). During the early stages of the COVID-19 pandemic, when programs were confronted with significant and unprecedented restrictions to traditional in-person clinicals, the relatively low cost, general availability, and range of virtual options appealed to nursing educators who had up to that point relied on more traditional teaching methods (
). However, the adoption of virtual clinical simulation in nursing education has not been without growing pains. While preliminary research has found that virtual clinical simulations yield comparable results relative to manikin-based simulation in terms of students’ perceptions of learning (
Despite recent efforts to delineate between virtual simulation through a screen-based learning environment and virtual reality simulation, which allows for a 360-degree immersion (
), concerns emerged during the pandemic regarding programs’ misuse of established technologies as well as their lack of evidence-based educational approaches. For instance, some nursing programs exceeded NCSBN’s recommended guidelines, both in terms of maximum substitution thresholds and use of high-fidelity SBE (
), while others employed unproven virtual modalities for traditional clinical hours and strayed from even the few foundational elements underpinning virtual simulation (
). Nonetheless, as the pandemic recedes and enters a new post-crisis phase, virtual clinical simulation, which requires fewer resources, such as space, faculty, and time relative to more established in-person clinical placements and high-fidelity SBE, appears to be, at least to some degree, an established component of the nursing educational landscape (
), broader adoption of virtual simulation may be inevitable.
Given the systemic shock presented by the pandemic and the range of strategies employed by nursing education programs to counter it, it is unsurprising that emerging evidence on student outcomes has been mixed. Despite many challenges (
documented new nurse graduates’ frustration over the apparent mismatch between their clinical experiences and their role as nurses entering the profession during a global health crisis. Two other studies similarly noted that nurses’ perceived level of preparedness and their transition to the professional nursing role were adversely affected by the reduction in clinical time and transition to online learning (
). While preliminary, these disparate findings suggest a need for not only further insight into the impact of the pandemic on nursing education, but they also necessitate future studies assessing possible links to adverse patient outcomes (
The present cross-sectional study utilized a four-phase longitudinal design to assess academic and initial postgraduation outcomes for nursing students in the spring 2022 cohort. A brief summary of each phase is presented in this section, and additional details are reported in subsequent sections by phase.
Phase One started data collection with a brief initial outreach survey (Appendix A1) to administrators at active prelicensure RN (associate degree in nursing [ADN] or bachelor of science in nursing [BSN]) programs in the United States in July 2020. In total, NCSBN researchers identified a valid email contact for 1,604 unique program administrators. To ensure an accurate accounting, only one representative from each institution was permitted to respond and multiple survey submissions were restricted. As part of this outreach, NCSBN contacted program deans, directors, and chairs via email and asked them to report the proportion of simulation use, both high-fidelity and virtual, in the fall 2019 term and to project thresholds for the fall 2020 term via a Qualtrics (Provo, UT) survey. The instrument also included a question to gauge respondents’ interest in participating in NCSBN’s planned longitudinal survey, which at the time was set to launch in August 2020. Though all questions were voluntary, more than three-quarters (n = 410, 77.9%) of respondents said they were interested. All ADN and BSN programs that expressed interest in the study and otherwise met eligibility criteria were contacted for possible participation.
The initial outreach survey results from Phase One indicated that most programs adopted online didactic coursework to a much greater extent than simulated or virtually simulated clinical experiences in response to the COVID-19 pandemic. Thus, NCSBN stratified its study recruitment to reflect this reality (Figure 1). To this end, programs were initially categorized into one of three groups:
1.
Low Didactic Online: Programs that reported minimal shifts to online didactic coursework, defined as ≤25th percentile of observed change based on baseline survey results.
2.
Medium Didactic Online: Programs that reported low to moderate shifts to online didactic coursework, defined as >25th percentile but <75th percentile of observed change based on baseline survey results.
3.
High Didactic Online: Programs that reported significant shifts to online didactic coursework, defined as ≥75th percentile of observed change based on baseline survey results.
Within each of these groups, programs were then further stratified based on the proportion of their planned shifts to simulated or virtually simulated clinical experiences.
1.
Low Simulation/Virtual Simulation Change: Programs that reported minimal shifts to simulation/virtual simulation, defined as ≤25th percentile of observed change based on baseline survey results.
2.
Medium Simulation/Virtual Simulation Change: Programs that reported low to moderate shifts to simulation/virtual simulation use, defined as >25th percentile but <75th percentile of observed change based on baseline survey results.
3.
High Simulation/Virtual Simulation Change: Programs that reported significant shifts to simulation/virtual simulation, defined as ≥75th percentile of observed change based on baseline survey results.
Phase Two of the study began in fall 2020 and was separated into two parts: students and faculty. The student cohort selected for participation comprised undergraduates entering the core of their didactic and clinical nursing coursework during the COVID-19 pandemic. General inclusion criteria were students enrolled in prelicensure RN (ADN or BSN) programs for fall 2020 at a participating study site with graduation anticipated in spring 2022. Exclusion criteria included accelerated BSN students, degree completion students (RN to BSN students), any student who already held a nursing license (licensed practical nurse/licensed vocational nurse [LPN/LVN] or RN), students enrolled in exclusively online programs, and students in any programs without full approval from its state’s BON. All eligible students were invited to participate, and consent was obtained at the launch of the study.
Program and student characteristics determined the overall profile of the study sample. Thus, the faculty inclusion and exclusion criteria mirrored the undergraduate sample parameters by default. Any didactic or clinical faculty teaching eligible students at a participating study were invited to participate. Unlike students who were only consented once at the start of the study, faculty participants were recruited on a rolling basis at the start of each academic term. This approach allowed NCSBN researchers to capture the often dynamic (e.g., due to turnover) faculty workforce at participating sites. After the fall 2020 term, all returning faculty were able to skip the consent process and proceed directly to the pre-course survey questionnaire, while new faculty participants were provided background on the study and asked to consent.
To facilitate student and faculty recruitment, NCSBN asked each participating nursing program to designate an administrator or faculty who would serve as site research coordinator for the duration of the study. Informational and training sessions were then scheduled with these individuals prior to the study launch to provide an overview of eligibility criteria. Participation criteria were hierarchical, so site research coordinators were asked to limit student outreach to undergraduates with an anticipated graduation in spring 2022. All students in this cohort were invited to participate via email and in-person communications by the nursing program deans and then in follow-up correspondence by the site research coordinators at study launch. Faculty recruitment was equally targeted, but more fluid. Only faculty teaching eligible students were invited to participate via both email and in-person communications from their program deans and site research coordinators. As this cohort shifted term-to-term throughout the data collection period, faculty were recruited in coordination with the site research coordinator at the start of each academic term from fall 2020 to spring 2022. Once students and faculty consented to participate, NCSBN researchers had access to self-reported contact information, including names and emails.
Phase Three of the study commenced at the conclusion of each student’s academic program, at which point exit information including alternate contact information and core measures of each student’s academic performance, such as their course-level standardized examination scores (e.g., Assessment Technology Institute [ATI], Health Education Systems, Inc. [HESI], and Kaplan), were collected. Then, at intervals of 3 and 6 months, new graduates were asked to provide details on their postgraduation experiences. For new graduates who were employed at one or more of these intervals, NCSBN research staff provided an anonymous link to forward to their manager or a direct supervisor familiar with their work to provide an additional evaluation. Each new graduate’s manager was then provided with a formal letter regarding the purpose of the study, the expected time commitment, and safeguards to ensure data security and integrity. Participation incentives were offered at each interval to both new graduates ($75) and their managers ($50) per successful survey completion. Unfortunately, while new graduate participation was robust, managerial feedback was very limited and thus not included in the final analysis. NCLEX-RN test results were verified within 6 months of graduation.
For the fourth and final phase of the research, NCSBN engaged a qualitative research consultant to conduct targeted focus groups at the conclusion of the 2-year study window (June–September 2022). Participation in the qualitative component of this mixed-methods study was treated separately from the initial consent process. In other words, program administrators, faculty, and students enrolled in the quantitative portion of our study were once again recruited and consented to participate in the qualitative follow-up. The NCSBN research team was responsible for recruiting the program personnel, but the qualitative researcher facilitated all focus groups and completed the qualitative analysis of the de-identified results.
Human Subjects Protection
To protect the rights of study participants, a full research protocol was submitted to Western Institutional Review Board (WIRB) for review, which determined the study to be exempt under 45 CFR § 46.104(d)(1). Prior to beginning data collection, informed consent was documented for all participants.
Phase One: Descriptive Summary of Study Sample
Phase One of this national study provides a descriptive summary of the initial outreach survey institutional participants and the final institutional participant sample for the longitudinal study. This information was compiled from the results of the survey to administrators at active prelicensure RN programs in the United States in July 2020. All information was then supplemented through a secondary review of publicly available data reported by the institution, the program, and the U.S. Department of Education.
Methods
In April 2020, NCSBN designed an initial outreach survey (Appendix A1) to prelicensure RN programs that consisted of eight core items, including types of degrees offered, use of standardized examinations to measure student progression, and several items comparing program enrollment, face-to-face and virtual simulation use, and online lecture content for fall 2019 and fall 2020. Prior to final dissemination, the instrument was reviewed for face validity through coordination with experienced nurse regulators and educators. The survey was initially administered using Qualtrics (Provo, UT) in mid-July 2020 and remained available for 2 weeks with three regularly scheduled reminders per week. Responses collected via this survey were also used to determine eligibility for Phase Two of the study.
Site research coordinators at selected programs were than asked to complete a Study Induction Survey (Appendix A) prior to kick-off. This instrument primarily solicited more detailed information on the student and faculty compositions of the nursing program. These results augmented data collected via the baseline survey. Missing information and additional characteristics were then supplemented via secondary searches using the Integrated Postsecondary Education Data System (IPEDS), administered by the U.S. Department of Education, and each program’s public-facing website and marketing materials.
Data Analysis
Data are reported as frequencies and proportions for all categorical variables, while continuous variables are expressed as means and standard deviations or medians and interquartile ranges (IQRs), as appropriate. Generalized estimating equation (GEE) models were used to assess the significance of observed trends. All analyses were conducted using SAS version 9.4 (Cary, NC), and p ≤ .05 was established as a benchmark for statistical significance.
Results
Initial Outreach Survey
A total of 526 programs responded for a final response rate of 32.8%. According to the survey results from July 2020, median enrollment held steady year over year (Table 1). In fall 2019, respondents (n = 499) indicated a median enrollment of 120 (IQR: 60–225) students. In fall 2020, respondents (n = 499) anticipated similar numbers (Mdn: 124.5, IQR: 60–236). Not surprisingly, the use of face-to-face simulation to replace traditional clinical placements doubled during the pandemic. From fall 2019 to fall 2020, among 490 respondents, the median number of clinical hours completed in simulation grew from 15% (IQR: 9%–25%) to 30% (IQR: 20%–50%, p < .001). Nonetheless, few prelicensure RN programs (n = 74) expected to exceed the 50% simulation replacement threshold during the fall 2020 term.
Table 1Comparison of Baseline Prelicensure RN Program Information: Fall 2019 vs. Fall 2020
A majority of programs also indicated they planned to incorporate virtual simulation instruction into their curriculum during the fall 2020 term (n = 421, 85.9%). The number of programs that offered no virtual simulation hours decreased substantially from fall 2019 (n = 130) to fall 2020 (n = 11, p < .001). Year-to-year, the median number of clinical hours completed in virtual simulation grew at an even faster pace than face-to-face simulation use, from 5% (n = 419, IQR: 0%–10%) to 25% (n = 419, IQR: 15%–50%, p < .001).
The proportion of lecture hours completed online presented the sharpest shift. From fall 2019 to fall 2020, the median number of lecture hours completed online grew from just 2% (n = 478, IQR: 0%–10%) to 62% (n = 48, IQR: 30%–100%, p < .001). Strikingly, the number of programs that offered no online lecture hours decreased from 167 in fall 2019 to 21 in fall 2020 (p < .001). By contrast, the number of programs that offered all their lecture hours online increased from 10 in fall 2019 to 153 in fall 2020 (p < .001).
Inclusion for Longitudinal Study
More than three-quarters (n = 410, 77.9%) of programs indicated their interest in participating in the longitudinal survey, which at the time was slated to launch in August 2020. All ADN and BSN programs that expressed interest in the study and otherwise met eligibility criteria were contacted for possible participation (Figure 2). Specifically, programs that only offered an accelerated BSN or online option, or uniquely enrolled degree completion students (RN to BSN students) or students who already held a nursing license (licensed practical nurse/licensed vocational nurse [LPN/LVN] or RN) were excluded from further outreach. In addition, if a program did not have full approval from its state’s BON at the time of study launch, it was similarly excluded.
FIGURE 2Study Flow From the Baseline Survey to Longitudinal Study
In total, 51 prelicensure RN programs participated in the longitudinal study. These programs represented 27 U.S. states and comprised more than 700 student and more than 400 faculty participants. There was a near even split between BSN (n = 28, 54.9%) and ADN (n = 23, 45.1%) programs (Table 2). A plurality were located in urban areas (n = 23, 45.1%), but sizable proportions of the participating programs indicated suburban (n = 16, 31.4%) and rural (n = 10, 19.6%) as well. A majority of the sample programs were from public institutions (n = 35, 68.6%) on a semester-based academic schedule (n = 44, 86.3%). Most programs were well established, with a median of 54 (IQR: 34–68) years in operation. Combining out-of-state tuition rates for public institutions with private expenses, the median annual tuition reported was $20,169 (IQR: $10,446–$31,574).
Table 2Summary of Programs Participating in the Longitudinal Study (N = 51)
Participating prelicensure RN programs also reported a median of 13 full-time faculty (IQR: 9–26) and 17 adjunct clinical faculty (IQR: 5–33). The median student-to-faculty ratio was consistent across the programs (Mdn: 8, IQR: 8–9). Similarly, the number of required clinical hours was fairly bounded (Mdn: 681, IQR: 584–750). Median prelicensure nursing student enrollment was 52 (IQR: 30–98). The overall student demographics at participating programs suggested a fairly diverse student population with a median of 9% Hispanic (IQR: 2%–20%), 63% White (IQR: 50%–82%), and 88% female (IQR: 80%–92%). The mean age of students at participating sites was 25 (SD: 6.7) years. The first-time NCLEX-RN pass rate among all students with at least one test administration date between March and December 2022 (e.g., predominantly the spring 2022 cohort) at participating sites was 85.0%. The overall NCLEX-RN pass rate at participating sites during the same period was 91.0%.
Limitations
This was a voluntary opt-in research study. As such, efforts were made to secure as large and geographically and demographically diverse a sample as possible. However, due to the personal, professional, and institutional strains brought about by COVID-19, many programs opted to drop from participation during the first term of within-program data collection. Similarly, at the institution level, the students and faculty who opted to participate may not provide an entirely representative snapshot of the engagement and learning levels at the participating prelicensure RN programs. Existing literature and the results of this study confirm that the lived experience of the COVID-19 pandemic is not universal, but rather an individualized experience dependent on the personal, academic, and professional stressors it introduces. Further, a detailed breakdown by student race is not available due to incomplete institutional responses provided by site research coordinators. Finally, the aggregate NCLEX-RN results included in this analysis suggest the included programs outperformed national results over the same period. Thus, the estimates and outcomes reported in this document may underreport the true effect of the observed trends.
Conclusion
The 51 prelicensure RN programs that participated in the longitudinal study ranged from smaller private not-for-profit institutions with fewer than 20 nursing students to large flagship public institutions with nursing program enrollments in the hundreds. The summary results underscored the geographic, programmatic, and demographic diversity of our retained sample. Participating prelicensure RN programs hailed from 27 states. There was a near even distribution by region, with nearly half of the programs located in urban areas (n = 23) and the remaining half in suburban/rural (n = 26) locales. Most of the programs were in public institutions (n = 35), and most offered a BSN degree (n = 28). The racial (63% White) and ethnic (9% Hispanic) diversity of the nursing student populations in participating programs was strong. As is typical of the broader nursing profession, 88% of prelicensure RN students enrolled at these institutions self-identified as female. Overall, despite the initial challenges associated with recruitment, the breadth of the final program sample fit the contours of our initial study design (Figure 4) and allowed NCSBN to secure responses from more than 1,100 student and faculty participants, including more than 4,000 course observations.
Phase Two: Student and Faculty Self-Report Measures
In August 2020, NCSBN formally launched its longitudinal study with Phase Two which is divided into studentt and faculty self-report measures.
Student Self-Report Measures
Within-program information was collected from study participants when they consented to participate in the study and then throughout the study at the beginning and end of each course using a combination of general self-report and validated instruments, as described in the following Methods (see Appendix B).
Methods
Sample
The student cohort selected for participation comprised undergraduates entering the core of their didactic and clinical nursing coursework during the COVID-19 pandemic (e.g., fall 2020). Thus, all students in the spring 2022 cohort at a participating site were invited to enroll in the study via email and in-person communications by the nursing program deans and then in follow-up correspondence by the site research coordinators at study launch. The window to review background information and participation requirements, ask any necessary follow-up questions, and consent to participate remained open throughout the initial term during which their program opted to enter the study. Once students consented to participate, NCSBN researchers had access to self-reported contact information, including names and emails, so all subsequent correspondence and outreach was coordinated internally.
Self-Report Instruments
Prior to final dissemination among students, all general self-report instruments were reviewed for face validity by experienced nurse regulators and educators. Surveys were distributed via email and administered using Qualtrics. Precourse surveys remained open until the end of each term, and postcourse surveys were available up to 6 weeks following the conclusion of the course with regularly scheduled weekly reminders. To support continued participation in the study, incentives were provided at both the institutional and student levels.
Specifically, programs that documented evidence of at least one student and faculty consent following formal study launch were awarded a $1,000 stipend. A second $1,000 retention stipend was disbursed at the start of the fall 2021 term. At the conclusion of each academic term, four students at each participating site (204 total across the 51 sites) were drawn at random from the pool of participants who completed and submitted their end of course surveys. Each of these students were awarded a $25 electronic gift card.
The instruments used to facilitate within-program student data collection were as follows:
1.
Pregraduation didactic instruments:
•
Initial Student Consent and Demographic Questionnaire (Appendix B1)
•
Cognitive, Affective, and Psychomotor (CAP) Perceived Learning Scale (Appendix B2)
Cognitive, Affective, and Psychomotor Perceived Learning Scale
The Cognitive, Affective, and Psychomotor (CAP) Perceived Learning Scale was developed and tested with students enrolled in both online and campus courses. Thus, it has utility across the entire delivery spectrum from fully online and blended courses to fully face-to-face instruction (
Development of an instrument to measure perceived cognitive, affective, and psychomotor learning in traditional and virtual classroom higher education settings.
). The CAP Perceived Learning Scale demonstrated good internal consistency, with a Cronbach’s coefficient alpha of .79. Evidence of convergent and discriminant validity was also found. The CAP Perceived Learning Scale total scores can range from a low of 0 to a high of 54; scores for the three CAP subscales (i.e., cognitive, affective, and psychomotor) range from 0 to 18. Cognitive learning relates to the recall of knowledge and the development of intellectual abilities or skills. Affective learning pertains to the development of positive attitudes and behavior toward a particular topic. Psychomotor learning aligns more with skills development and task completion.
Student Course Engagement Questionnaire Modified
Research on college students’ learning has found that educational outcomes are strongly linked to the level and type of student engagement (
). The Student Course Engagement Questionnaire Modified (SCEQ-M) is designed to measure student engagement across learning modalities: in person, hybrid, and online. The SCEQ-M measures four dimensions of college student engagement with their courses: (a) applied engagement (9 items), (b) goal-oriented engagement (6 items), (c) self-disciplined engagement (5 items), and (d) interactive engagement (3 items). Instructions to complete the SCEQ-M are as follows: “To what extent do the following behaviors, thoughts, and feelings describe you, in this course? Please rate each of them on the following scale: 1 = not at all characteristic of me, 2 = not really characteristic of me, 3 = moderately characteristic of me, 4 = characteristic of me, and 5 = very characteristic of me.” Scores on the SCEQ-M vary from a minimum of 23 to a maximum of 115 for the entire scale. SCEQ-M demonstrated good internal consistency across the four engagement factors, with Cronbach’s coefficient alphas ranging from .71 to .86. Evidence of convergent and discriminant validity was also found.
The Clinical Learning Environment Comparison Survey (CLECS) was developed to assist nursing educators and regulatory bodies in better understanding how well the learning needs of students are met in traditional and simulated clinical environments (
). The CLECS is comprised of the following subscales: (a) communication (4 items), (b) nursing process (6 items), (c) holism (6 items), (d) critical thinking (2 items), (e) self-efficacy (4 items), and (f) teaching–learning dyad (5 items). CLECS demonstrated good internal consistency and reliability across the six subscales, with Cronbach’s coefficient alphas ranging from .73 to .90. Evidence of convergent and discriminant validity was also confirmed. In response to the rapid growth and utilization of virtual simulation during the pandemic, CLECS was extended and rebranded as CLECS 2.0 to capture students’ perceptions of learning using screen-based simulation as well. While reliability data for the CLECS 2.0 has not yet been established, preliminary evidence suggests its application in virtual environments is sound (
Meeting undergraduate nursing students’ clinical needs: A comparison of traditional clinical, face-to-face simulation, and screen-based simulation learning environments.
All model-based results are expressed as means and standard errors (SEs). Due to the longitudinal nature of the data tracking, the total number of study participants varied throughout the observation period; however, more than 4,000 course observations (including lectures and clinical experiences) from more than 700 students who consented to participate in this study were utilized. GEE models were used to assess the significance of observed trends. All analyses were conducted using SAS version 9.4, and p ≤ .05 was set as the threshold for evaluating statistical significance.
Results
Student Sample
While this was a voluntary study for which students opted-in, the profile of study participants aligned strongly with the overall institutional profile documented earlier (Table 3). The average age of students who participated in this longitudinal study was 25 (SD: 7.6) years. Approximately 12.5% of students self-identified as Hispanic, 75% as White, and 88% as female. Nearly one-third of participating students (n = 236, 32.9%) indicated they were Pell Grant recipients. A plurality was located in urban areas (n = 334, 46.5%), but sizable proportions of the participating students were located in suburban (n = 211, 29.4%) and rural (n = 157, 21.9%) locales. A majority of the student sample was enrolled at public institutions (n = 444, 61.8%) and were on semester-based academic schedules (n = 676, 94.2%). There were more BSN students (n = 432, 60.2%) than ADN (n = 286, 39.8%) students. The first-time NCLEX pass rate among participating students with at least one test administration date between March and December 2022 (e.g., predominantly the spring 2022 cohort) at participating sites was 89.6%. The overall NCLEX pass rate among participating students during the same period was 98.2%.
TABLE 3Demographics of Participating Students (N = 722)
For students with a NCLEX administration date between March and December 2022 (predominantly spring 2022 cohort).
98.2%
Note. ADN = associate degree in nursing; BSN = bachelor of science in nursing; SD = standard deviation. Observed n varies across reported or tracked student characteristics.
a Data presented as n (%) unless otherwise indicated.
b For students with a NCLEX administration date between March and December 2022 (predominantly spring 2022 cohort).
Overall, older students’ self-reported perceptions of learning were higher than those of younger students (p = .01), driven primarily by differences in cognitive and psychomotor results (Table 4). Similarly, Pell Grant recipients (M: 39.57, SE: 0.78) also reported higher CAP scores, again particularly in the areas of cognitive and psychomotor learning, compared to their peers who did not report a Pell Grant (M: 37.01, SE: 0.59, p < .01). Students who enrolled in in-person (M: 38.91, SE: 0.66) and hybrid courses (M: 38.82, SE: 0.63) also reported consistently higher levels of learning compared to those in online learning environments (M: 35.19, SE: 0.76, both p < .001). While in-person learning appeared stronger than online learning across both the affective and psychomotor subscales, hybrid delivery formats were reported as superior across all domains. Furthermore, self-reported CAP perceived learning scores also gradually increased over the reporting period, with students in the spring 2022 term consistently reporting the highest scores (M: 39.73, SE: 0.91), which represented a meaningful increase from fall 2020 (M: 36.92, SE: 0.69, p < .01), particularly in the areas of cognitive and affective learning.
Note. ADN = associate degree in nursing; BSN = bachelor of science in nursing; Ref = reference. All estimates are presented as mean (standard error). Linear associations by age and years in operation are presented, with values below 1 indicating an inverse relationship and estimates above 1 a positive relationship.
Compared to students in adult medical surgical rotations (M: 39.04, SE: 0.67), students’ self-reported perceptions of learning were lower in community rotations (M: 34.72, SE: 1.30, p < .01). While some variation was observed across the affective (maternal-newborn vs. adult medical surgical) and psychomotor (mental health vs. adult medical surgical) domains as well, none were sustained when reviewing overall perceptions of learning. Students enrolled at urban-based institutions (M: 35.79, SE: 0.71) consistently reported lower learning scores compared to both their suburban (M: 39.17, SE: 0.96) and rural (M: 39.21, SE: 0.96) counterparts (both p < .001). These overall results were driven primarily by divergences observed on the affective and psychomotor subscales. Finally, cognitive, affective, and psychomotor learning scores were higher for students enrolled in ADN programs (M: 40.43, SE: 0.63) compared to BSN programs (M: 36.00, SE: 0.64, all p < .001).
For the SCEQ-M, self-reported engagement was also higher among older students (p < .01), especially across the applied, self-disciplined, and interactive domains (Table 5). Students who enrolled in in-person (M: 94.73, SE: 1.31, p < .01) and hybrid courses (M: 94.06, SE: 1.16) frequently reported higher levels of engagement compared to those in online learning environments (M: 91.56, SE: 1.24). Notably, though, in-person learning scores were consistently stronger than online formats across all four subscales, which drove a significant overall difference. By contrast, hybrid results varied a bit more, resulting only in an overall trend despite superior results in the areas of self-disciplined and interactive engagement. Self-reported engagement was also lower for students enrolled in community rotations (M: 87.55, SE: 1.94) compared to students in adult medical surgical rotations (M: 93.98, SE: 1.09, p < .01). The inverse relationship was observed comparing maternal-newborn (M: 97.51, SE: 1.46) and adult medical-surgical rotations (p = .02). Urban-based students (M: 90.56, SE: 1.39) reported the lowest engagement (suburban M: 95.50, SE: 1.56; rural M: 96.31, SE: 1.59, both p < .001).
Note. ADN = associate degree in nursing; BSN = bachelor of science in nursing; Ref = reference. All estimates are presented as mean (standard error). Linear associations by age and years in operation are presented, with values below 1 indicating an inverse relationship and estimates above 1 a positive relationship.
For the CLECS 2.0 results, students who attended in-person clinical experiences (M: 91.14, SE: 1.13, p < .001) and face-to-face simulations (M: 89.31, SE: 1.37, p < .001) consistently reported better clinical learning compared to those in virtual simulated environments (M: 76.32, SE: 1.64, Table 6). This pattern held true across all six subdomains (all p < .001). Summed CLECS 2.0 scores also steadily increased over the data collection window, with students reporting meaningfully higher scores across the 2-year period and even with the passage of each subsequent term (all p < .05). This trend was also observed for the Holism domain, but for the remaining five subscales, broader differences only emerged between the 2 academic years. In other words, students enrolled in courses in the 2021–2022 academic year regularly reported that their clinical needs related to communication, nursing process, critical thinking, self-efficacy, and teaching–learning dyad were better met than they were in the 2020–2021 academic year.
Note. ADN = associate degree in nursing; BSN = bachelor of science in nursing; Ref = reference. All estimates are presented as mean (standard error). Linear associations by age and years in operation are presented, with values below 1 indicating an inverse relationship and estimates above 1 a positive relationship.
Compared to students in their advanced medical surgical rotation (M: 93.39, SE: 2.02), students in their adult medical surgical (M: 84.76, SE: 1.54, p < .001), community (M: 81.67, SE: 3.35, p < .01), fundamentals (M: 85.00, SE: 1.76, p < .001), and mental health (M: 87.05, SE: 2.26, p = .02) rotations all reported lower clinical learning scores. This pattern held across five (communication, nursing process, critical thinking, self-efficacy, and teaching–learning dyad) of the six sub-domains (all p < .05). Students in their maternal-newborn rotation (M: 89.31, SE: 2.40) were also more likely to report that their clinical learning needs were better met than those in their community rotation (p < .01). Students enrolled at urban-based institutions (M: 82.76, SE: 1.95) reported lower clinical learning scores compared to their suburban counterparts (M: 91.14, SE: 1.85, p < .001). This trend held across all six subdomains (all p < .05).
Discussion
The proportion of students who self-identified as Hispanic (+3.5%), White (+12%), and female (no difference) aligned strongly with the overall institutional profile, indicating good representation among students from the participating programs. Similarly, like the programs, a plurality of students were located in urban areas (+1.4%), but sizable proportions of the participating students indicated suburban (-2.0%) and rural (+2.3%) areas as well. There was a bit more representation among the student sample regarding institutional tax status, with around 60% enrolled at public institutions (-6.8%), but a little less representation on program type (BSN +5.3%). First-time and overall NCLEX pass rates among participating students were similarly strong to the overall program outcomes.
Pandemic disruptions to traditional academic teaching models led to significant shifts in students’ self-reported learning and engagement. For lecture-based courses, the delivery format and the stage of the pandemic drove meaningful differences among students’ self-reported outcomes. In-person and hybrid courses consistently documented higher levels of learning compared to online learning environments. Similarly, students who attended in-person clinical experiences or participated in face-to-face simulations also reported better clinical learning compared to those in virtual simulated environments. Lecture-based and clinical learning scores both gradually increased over the reporting period, with students in each subsequent term reporting consistently better results. Overall, this may suggest links to early disruptions driven by pandemic surge events, the loosening of restrictions over time, and students’ overall acclimation to new conditions.
Increases in students’ self-reported engagement were also documented in person and through hybrid learning environments. Notably, as the use of online learning platforms and virtual screen-based simulations ebbed over time, students’ reported engagement and learning increased. For instance, students participating in maternal-newborn and advanced medical surgical rotations, which appeared back loaded in the curriculum, often reported superior results in both rotations compared to earlier rotations in adult medical surgical and fundamentals. In addition, the community health rotation appeared to be a particularly poor fit for the new learning environments.
Interesting patterns also emerged based on select student demographics, as well as program location and type. Not surprisingly perhaps, stronger learning and engagement scores were documented among older students across both lecture-based and clinical settings. While Pell Grant recipients reported higher lecture-based learning scores, their engagement and clinical outcomes were comparable to non–Pell Grant recipients. Urban-based institutions consistently underperformed suburban and rural programs across the reported learning and engagement scales as well; however, these results were again driven primarily by course delivery format. While urban programs documented comparable rates of in-person clinical placements to their suburban and rural counterparts, they often substituted higher rates of virtual simulation (11.2% vs. 8.3% suburban and 4.0% rural) and comparatively lower rates of face-to-face simulation (68.1% vs. 72.5% suburban and 77.1% rural). In addition, urban institutions pivoted to online lecture learning (39.6%) at much higher rates than suburban (21.5%) and rural (6.5%) programs. Similar patterns emerged by program type, as BSN programs were significantly more likely to be in urban areas (81.0%) compared to suburban (32.1%) and rural (58.5%) locales.
Limitations
Since this was a voluntary opt-in research study, the students who opted to participate may not provide an entirely representative snapshot of the engagement and learning that took place at the participating prelicensure RN programs. Existing literature and the results of this study confirm that the lived experience of the COVID-19 pandemic is not universal, but rather it is often individual and dependent on the personal, academic, and professional stressors it introduces. Additionally, as with many longitudinal studies, observed attrition limited the number of observations available in advanced clinical experiences, relative to earlier clinical rotations, such as fundamentals and adult medical surgical. This phenomenon frequently presented in the form of slightly elevated SE estimates for certain courses.
Furthermore, the variable timing of students’ NCLEX-RN administration and the aforementioned issues of attrition over the 2 academic years resulted in standardized examination scores being available for only half the consented sample (n = 338). Thus, the descriptive summary of students’ NCLEX-RN results should be interpreted with caution as they provide only a partial snapshot of the cohort’s overall examination performance. Related, the aggregate NCLEX-RN results reported in this analysis suggest the included programs and participating students outperformed national results over the same period. Therefore, it is possible the estimates and outcomes reported in this document may in fact underreport the true effect of the observed trends. In addition, a detailed breakdown by student race is provided in the descriptive summary, but the low response n across non-White racial categories required that the variable be collapsed to a simpler White v. Non-White comparison for modeling. Finally, the findings of this analysis are correlational and do not support causal inference.
Conclusion
The one constant that emerged from the student self-report data was the multifaceted and wide-ranging impacts of programs’ shifts to remote and virtual learning. Students reported lower levels of learning and engagement in both online lecture-based and virtual clinical settings. These results manifested time and again in a variety of ways, including observed trends by program setting, type, and Pell Grant status. In addition, and not surprisingly, the effects of the pandemic dissipated somewhat over time. This was clear from term-to-term results as well as from differences by clinical rotation.
Faculty Self-Report Measures
This second part of Phase Two focuses specifically on faculty self-report measures. Within-program information was collected from study participants when they provided consent and then throughout the study at the beginning and end of each course using a combination of general self-report and validated instruments, which are described in the following section and provided in Appendices B and C.
Methods
Sample
Any didactic or clinical faculty teaching eligible students at a participating study were invited to participate. All faculty were invited to enroll in the study via email and in-person communications by the nursing program deans and then in follow-up correspondence by the site research coordinators at study launch. Faculty participants were recruited on a rolling basis at the start of each academic term. This approach allowed NCSBN researchers to capture the often dynamic (e.g., due to turnover) faculty workforce at participating sites. After the fall 2020 term, all returning faculty were able to skip the consent process and proceed directly to the pre-course survey questionnaire, while new faculty participants were provided background on the study and asked to consent. The window to review background information and participation requirements, ask any necessary follow-up questions, and consent to participate remained open throughout each academic term. Once faculty consented to participate, NCSBN researchers had access to self-reported contact information, including names and emails, so all subsequent correspondence and outreach was coordinated internally.
Self- Report Instruments
Prior to final dissemination of the surveys among faculty, all general self-report instruments were reviewed for face validity through coordination with experienced nurse regulators and educators. Surveys were distributed via email and administered using Qualtrics. Precourse surveys remained open until the end of each term, and postcourse surveys were available for up to 6 weeks following the conclusion of the course (faculty were emailed weekly reminders). Clinical instructors were also asked to provide up to two evaluations of each student who consented to participate using the Creighton Competency Evaluation Instrument (CCEI).
The instruments used to facilitate within-program faculty data collection were as follows:
The CCEI is a 23-item tool used by clinical instructors to rate students on behaviors that collectively demonstrate clinical competency (assessment, communication, clinical judgment, and patient safety). The tool is employed in this study to assess how well students progressed across three settings: in-person clinical, face-to-face simulation, and virtual simulation experiences. The CCEI has demonstrated good internal consistency, with a Cronbach’s coefficient alpha of .97 (Hayden, Keegan, Kardong-Edgren, & Smiley, 2014). Evidence also supported the convergent and discriminant validity of the tool.
Data Analysis
All model-based results are expressed as means and standard errors. Due to the longitudinal nature of the data tracking, the total number of study participants varied throughout the observation period; however, more than 4,000 course observations (including lectures and clinicals) from the more than 400 faculty who consented to participate in this study were utilized. GEE models were used to assess the significance of observed trends. All analyses were conducted using SAS version 9.4, and p ≤ .05 was considered statistically significant.
Results
Faculty Sample
The average age of faculty who participated in this longitudinal study was 48 (SD: 12.5) years (Table 7). More than 90% of faculty self-identified as female (n = 295, 94.0%), non-Hispanic (n = 309, 98.4%), and White (n = 285, 90.8%). Approximately half of participating faculty indicated they were either an instructor (n = 100, 31.4%) or adjunct professor (n = 83, 26.1%). Most participants reported a master’s degree (n = 174, 55.8%) as their highest degree and noted that they served in non-tenured positions (n = 223, 71.5%). Two-thirds of faculty taught clinical rotations (n = 294, 67.7%), worked in a BSN program (n = 269, 65.8%), and worked at a public institution (n = 274, 67.0%). Nearly all faculty taught on a semester-based academic schedule (n = 391, 95.6%). Just over half of the faculty taught at institutions in urban areas (n = 207, 50.6%), but sizable proportions indicated suburban (n = 129, 31.5%) and rural (n = 67, 16.4%) areas as well.
TABLE 7Demographics of Participating Faculty (N = 434)
Of the 294 faculty who reported teaching a clinical rotation, three-quarters (n = 225, 76.5%) reported a simulation component (n = 199, Mdn: 20%, IQR: 10%–30%). For most in this situation (n = 142, 68.3%), face-to-face simulation (n = 85, Mdn: 10%, IQR: 5%–25%) was substituted for in-person clinical training, but a notable proportion (n = 94, 45.2%) reported employing virtual learning environments as well (n = 85, Mdn: 10%, IQR: 5%–25%). Notably, most faculty reported very little experience with simulation-based instruction (n = 204, Mdn: 3 years, IQR: 2–7 years) and almost none of the faculty indicated they were Certified Healthcare Simulation Educators (CHSEs; n = 5, 2.4%). For those using screen-based simulation, only half (n = 42, 47.2%) reported prior experience teaching virtually. Most indicated their program leveraged online software packages, such as screen- or computer-based branching narratives (n = 49, 55.1%).
For faculty who taught through lectures, a much larger proportion (n = 111, 79.3%) reported prior, albeit minimal (Mdn: 2 years, IQR: 1–6 years) online teaching experience. Most indicated they taught similar material the prior term (n = 88, 63.3%). Not surprisingly, the proportion of faculty reporting remote lectures completely flipped year-to-year. In fall 2019, about 10% of faculty (n = 11, 12.5%) reported teaching the lecture component of students’ coursework entirely online, but that figure jumped to more than one-third (n = 51, 37.0%) in fall 2020. While approximately 70% of lecture-based instruction was delivered in person in fall 2019 (n = 62, 70.5%), a nearly identical proportion (n = 98, 71.0%) was delivered entirely online or in a hybrid setting in fall 2020. Unlike students’ clinical rotations, the proportion of online (Mdn: 50%, IQR: 20%–50%) and in-person (Mdn: 50%, IQR: 40%–75%) lecture content was evenly split.
Survey Findings
Overall, faculty consistently rated older students’ clinical competency higher than younger students (p < .01, Table 8). Similarly, White students (M: 20.04, SE: 0.22) also received higher CCEI scores compared to their non-White peers (M: 17.52, SE: 0.77, p < .01). Faculty tended to rate students who attended face-to-face simulation (M: 20.28, SE: 0.35) higher compared to those in virtual simulated learning environments (M: 17.62, SE: 1.17, p = .02). While not significant, a similar trend also emerged for students who attended in-person clinical placements (M: 19.84, SE: 0.75, p = .10). Unlike the student self-report measures, faculty observations and ratings of clinical competence gradually declined over the reporting period, with students in the spring 2022 term receiving significantly lower scores (M: 18.37, SE: 1.13) compared to the fall 2020 term (M: 20.85, SE: 0.62, p = .05).
TABLE 8Creighton Competency Evaluation Instrument (CCEI) Results by Student Characteristics
Faculty also consistently rated students in their advanced medical surgical (M: 21.78, SE: 0.24) and maternal-newborn (M: 21.58, SE: 0.24) rotations higher than all other rotations (p < .05). Faculty at urban-based institutions (M: 18.56, SE: 0.44) also scored students’ clinical competence lower compared to faculty at suburban (M: 19.93, SE: 0.30, p < .01) and rural (M: 20.65, SE: 0.26, p < .001) programs. Finally, faculty at more established institutions rated students’ clinical competency higher than did faculty at newer programs (p = .02).
Postcourse faculty responses indicated a good deal of continuity in instructional content from fall 2019 to fall 2020, with approximately three-quarters of respondents indicating they taught the same lecture (n = 189, 78.4%) or clinical rotation (n = 207, 77.8%) during the prior term (Table 9). However, between 40% to 50% of faculty indicated the format of their lecture-based (n = 124, 47.3%) and/or clinical instruction (n = 99, 41.3%) changed in fall 2020 due to significant shifts online or to face-to-face and virtual simulation. Faculty-reported engagement, work quality, and learning outcomes generally remained strong in fall 2020. Nonetheless, both didactic (n = 67, 35.9% much/less engaged) and clinical (n = 64, 32.0% much/less engaged) faculty reported notable proportions of reduced student engagement, as well as some dips in didactic (n = 46, 24.6% much/poorer quality) and clinical (n = 43, 21.6% much/poorer quality) work quality. Even against that backdrop, though, approximately four in five faculty noted learning outcomes were met or exceeded at about the same level as the prior academic year.
TABLE 9Postcourse Faculty Self-Reported Items
Faculty Survey Items
Lectures
Clinical Rotations
Taught Course Prior to Fall 2020
Yes
189 (78.4%)
207 (77.8%)
No
52 (21.6%)
59 (22.2%)
Updated Delivery Format (2019 vs. 2020)
Yes
124 (47.3%)
99 (41.3%)
No
138 (52.7%)
141 (58.8%)
Revised Delivery Format – Lecture
In person
117 (48.8%)
-
Hybrid
56 (23.3%)
-
Online
67 (27.9%)
-
Revised Delivery Format – Clinical
Mix of in person and face-to-face simulation
-
38 (31.4%)
Mix of in person and virtual simulation
-
23 (19.0%)
Mix of in person, face-to-face simulation, and virtual simulation
-
49 (40.5%)
Mix of face-to-face simulation and virtual simulation
-
11 (9.1%)
Current Engagement
Not engaged at all
3 (1.3%)
1 (0.4%)
Somewhat engaged
66 (27.6%)
34 (13.1%)
Generally engaged
127 (53.1%)
143 (55.2%)
Very engaged
43 (18.0%)
81 (31.3%)
Engagement – Term Comparisons
Much less engaged
8 (4.3%)
6 (3.0%)
Less engaged
59 (31.6%)
58 (29.0%)
No change
83 (44.4%)
89 (44.5%)
More engaged
30 (16.0%)
43 (21.5%)
Much more engaged
7 (3.7%)
4 (2.0%)
Current Work Quality/Performance
Very low quality (L)/performance (C)
2 (0.8%)
1 (0.4%)
Low quality (L)/performance (C)
19 (8.0%)
26 (10.1%)
Neither
41 (17.2%)
21 (8.1%)
Good quality (L)/performance (C)
147 (61.5%)
163 (63.2%)
Very good quality (L)/performance (C)
30 (12.6%)
47 (18.2%)
Work Quality – Term Comparisons
Much poorer quality (L)/performance (C)
5 (2.7%)
2 (1.0%)
Poorer quality (L)/performance (C)
41 (21.9%)
41 (20.6%)
About the same quality (L)/performance (C)
103 (55.1%)
126 (63.3%)
Better quality (L)/performance (C)
34 (18.2%)
27 (13.6%)
Much better quality (L)/performance (C)
4 (2.1%)
3 (1.5%)
Learning Outcomes
Did not meet learning outcomes
1 (0.4%)
-
Partially met learning outcomes
22 (9.2%)
28 (10.9%)
Met learning outcomes
202 (84.9%)
219 (85.2%)
Exceeded learning outcomes
13 (5.5%)
10 (3.9%)
Learning Outcomes – Term Comparisons
Many fewer students met learning outcomes
3 (1.6%)
2 (1.0%)
Fewer students met learning outcomes
30 (16.0%)
38 (19.2%)
About the same number met learning outcomes
119 (63.6%)
131 (66.2%)
More students met learning outcomes
32 (17.1%)
24 (12.1%)
Many more students met learning outcomes
3 (1.6%)
3 (1.5%)
Online Proportion
50.0% (25.0%–75.0%)
-
In-person Proportion
42.5% (10.0%–70.0%)
60.0% (30.0%–80.0%)
Face-to-Face Simulation Proportion
-
10.0% (0%–25.0%)
Virtual Simulation Proportion
-
10.0% (0%–25.0%)
Note. C = clinical; L = lecture. Observed n varies across reported or tracked faculty characteristics. Data presented as n (%) except for continuous variables, which are expressed as median (interquartile range).
The faculty demographic profile was less diverse than the student sample, with less than 10% of the participants self-identifying as non-White (9.2%), male (6.0%), and Hispanic (1.6%). Otherwise, the proportion of faculty from public institutions (67.0%) and programs located in urban areas (49.4%) suggested a comparable instructor composition. Most faculty were non-tenure track part-time instructors or adjuncts, and nearly two-thirds of participants taught clinical rotations. Notably, given the impact of course delivery format, most faculty reported very little experience with simulation-based instruction (Mdn: 3 years) and almost none indicated they were CHSEs (2.4%).
Like students, faculty tended to provide higher ratings for in-person clinicals and face-to-face simulation compared to online simulated learning environments. As before, this finding drove associations by program setting as well, with urban institutions appearing to underperform compared with suburban and rural programs. In addition, CCEI scores were highest for students in their advanced medical surgical and maternal-newborn clinical rotations. In contrast to student self-report outcomes, though, faculty observations and ratings of clinical competence gradually declined over the reporting period. Overall, faculty reported engagement, work quality, and learning outcomes generally remained strong in fall 2020. However, both didactic and clinical faculty reported notable reductions in student engagement and work quality.
Limitations
Considering that this was a voluntary, opt-in research study, it is important to note that the faculty who opted to participate may not provide an entirely representative snapshot of the outcomes at the participating prelicensure RN programs. As noted, existing literature aligns with the findings of this study and supports the view that the lived experiences of the COVID-19 pandemic is not universal, but rather individual and dependent on personal, academic, and professional stressors. Furthermore, there was an imbalance in the CCEI submissions, both at the institutional and faculty levels. Specifically, larger proportions of CCEI observations were submitted by certain faculty and institutions; thus, it could make insights gained from these scales less broadly applicable. In addition, a detailed breakdown by faculty race is provided in the descriptive summary, but the low response n across non-White racial categories required that the variable be collapsed to a simpler White v. Non-White comparison for modeling. Finally, the findings of this analysis are correlational and do not support causal inference.
Conclusion
Faculty responses resonated with student results, attesting to the broad disruptive effects of the pandemic on students’ learning and engagement. Given the importance of course delivery format, as confirmed across student and faculty self-report data, the mismatch in faculty experience and training with simulation is striking. The level of consistency across both cohorts made areas in which they diverged that much more pronounced. Nowhere was that more apparent than with faculty ratings of clinical competence over time. Unlike student reported outcomes, faculty tended to rate student performance lower as they progressed through the program and study. This may be an artifact of students’ aggregate or accumulated learning loss over the term of their academic program as faculty initially reported engagement, work quality, and learning outcomes generally remained strong in fall 2020 and did not diverge too greatly from prior terms.
Phase Three: Standardized Examination Measures and New Graduates’ Early Career Outcomes
The third phase of the study is divided into two sections: standardized examination scores and new graduates’ early career outcomes. The first section focuses specifically on the within-program and postgraduation standardized examination scores for participating students. The second section of Phase Three focuses specifically on new graduate self-reported career measures.
Standardized Examination Measures
Within-program information was collected directly from site coordinators at the conclusion of participants’ course of study. This date varied somewhat from program-to-program, but typically fell between March and June 2022. Postgraduation NCLEX-RN results were then tracked up to 6 months following the typical graduation timeframe for most student participants in the spring 2022 cohort (e.g., May 2022).
Methods
Following the formal closure of the within-program data collection phase of the longitudinal study, NCSBN collaborated with site coordinators directly to access within-program standardized test scores for student participants in this study. Relevant scores included ATI, HESI, and Kaplan examination results. NCLEX-RN results were captured in two ways. Student participants were contacted directly 3 and 6 months after graduation and asked to complete the New Graduate Nurse Performance Survey (NGNPS). To qualify to complete the NGNPS, these new graduates had to indicate they had taken and passed the NCLEX-RN and were presently employed in a nursing position. In addition, the Research Department coordinated with NCSBN’s Examination Division to review and report summary (e.g., program-level) NCLEX-RN examination scores from March 1 through December 15, 2022.
Additionally, NCSBN analyzed course (ATI, HESI, Kaplan) and NCLEX-RN examination scores at the program level. Thus, de-identified summary pass rates were aggregated to the program-level and compared against similarly aggregated program characteristics associated with both the institutional and student demographic profile, as well as changes to course delivery formats. Any changes to course delivery formats were captured both at the start of the COVID-19 pandemic (fall 2020) and through sustained data tracking throughout the reporting window (fall 2020–spring 2022) given the dynamic nature of the pandemic and programs’ responses to it. Programmatic and course delivery format details were reported primarily by administrative and faculty participants, but information from students’ self-report data was available for cross-reference and confirmatory purposes.
Summary NCLEX-RN results for all 51 participating programs were sourced directly from NCSBN’s Examination Division. Trends were analyzed to identify any potential correlations with select institutional characteristics as well as changes to course delivery formats, as reported by faculty participants. As for within-program scores, NCSBN leveraged participating faculty’s reporting throughout the data collection window (fall 2020–spring 2022) to accurately capture and characterize the dynamic nature of the pandemic and programs’ responses to it. This information was again aggregated to the program-level (e.g., to calculate mean increases to the proportion of course-level virtual simulation use) and appended to the summary NCLEX-RN results. Importantly, the summary NCLEX-RN results reported at the program-level for this portion of the analysis reflect the overall results of all test-takers at a participating site. Any test-taker who attempted the NCLEX-RN examination at least once between March 1 and December 15, 2022 was included.
Variable Coding
To standardize reported outcomes using within-program examination scores, ATI, HESI, and Kaplan results were combined and recoded as a composite binary outcome (pass/fail). Select independent variables were also recoded based on their underlying values to facilitate group analysis. Namely, programs’ changes to their course delivery formats, both at the start of the pandemic (fall 2020) and throughout the reporting window (fall 2020–spring 2022), were recast as binary predictors, using the median value from the underlying distribution as a cut-point. For instance, initial (fall 2020) changes to the proportions of face-to-face simulation programs used were binned based on values less than and greater than or equal to 15% (e.g., the observed median value in the sample). Similar recoding logic was applied to initial changes to virtual simulation usage (cut at 14%) and initial changes to online lectures (cut at 61%). From thereafter, changes at the course-level were averaged to determine an overall mean change to face-to-face simulation (cut at 12.5%), virtual simulation (cut at 20%), and online lectures (cut at 46.25%). Finally, the proportions of clinical (cut at 50%) and didactic (cut at 37.5%) courses with some manner of delivery format change were also tracked for each institution.
Data Analysis
Individual nursing programs served as the primary unit of measurement when examining students’ summary standardized test scores. Given the small institutional sample size (n = 51), continuous variables are expressed as medians and IQRs. Categorical results were reported as frequencies and proportions. Due to the small sample size and the often non-normally distributed nature of the aggregated results, all group comparisons employed nonparametric statistical measures to determine the significance of observed trends. For two-group comparisons on continuous outcomes, the nonparametric analog to the independent samples t test, the Wilcoxon-Mann-Whitney test, was employed; for comparisons of three or more groups, the Kruskal-Wallis test was used. For group comparisons on categorical outcomes, a Fisher’s exact test was used due to low expected cell counts. All analyses were conducted using SAS version 9.4, and p ≤ .05 was established as the threshold for statistical significance.
Results
ATI, HESI, and Kaplan results were shared for only 45.8% (n = 331) of student participants in this study. Across all course types, the overall pass rate was just 62.2%, with outcomes ranging from a 52.2% pass rate for students in their maternal-newborn rotations to 80.5% for advanced medical surgical. There was little variation in within-program standardized examination scores by aggregate institutional characteristics (Table 10). BSN programs (Mdn: 56.3%, IQR: 39.2%–67.6%) and ADN (Mdn: 52.2%, IQR: 42.9%–70.8%) programs reported comparable results, as did those institutions located in urban (Mdn: 66.7%, IQR: 31.2%–82.5%), suburban (Mdn: 50.3%, IQR: 44.8%–59.4%), and rural (Mdn: 52.0%, IQR: 39.8%–59.0%) areas. While the observed raw scores for programs that had pronounced difficulty arranging in-person clinical placements were lower, the comparison did not reach statistical significance (p = .18). However, variables that represented likely contingency measures did emerge. Namely, programs that reported pronounced increases in their utilization of virtual simulation (p = .05), and particularly those that indicated no additional institutional resources to support such a transition (p = .03), documented corresponding declines in their ATI, HESI, and Kaplan results. Conversely, those programs that increased their utilization of face-to-face simulation (p = .04) documented higher ATI, HESI, and Kaplan results.
TABLE 10Program Comparisons on Standardized Examination Scores
Within-program test scores include Assessment Technology Institute, Health Education Systems, Inc., and Kaplan examinations.
p
NCLEX Scores
p
Program type
BSN
56.3% (39.2%–67.6%)
.47
89.5% (80.4%–92.7%)
.50
ADN
52.2% (42.9%–70.8%)
88.5% (83.3%–91.3%)
Region
Urban
66.7% (31.2%–82.5%)
.85
88.0% (80.3%–92.5%)
.57
Suburban
50.3% (44.8%–59.4%)
88.3% (82.3%–93.1%)
Rural
52.0% (39.8%–59.0%)
91.1% (88.5%–92.7%)
Difficulty Arranging Clinical Rotations
Similar level of difficulty
52.2% (44.4%–77.9%)
.18
88.3% (82.3%–92.7%)
.97
Somewhat more difficult
62.5% (48.4%–66.7%)
88.5% (86.1%–92.7%)
Much more difficult
38.1% (0.0%–52.0%)
85.1% (78.1%–94.9%)
Resources for Virtual Simulation
Maintaining same levels prior
52.2% (39.2%–66.7%)
.03
90.9% (75.9%–93.1%)
.51
No
39.8% (38.1%–52.0%)
87.5% (80.4%–90.7%)
Yes
82.9% (63.0%–86.7%)
89.1% (83.1%–93.1%)
Initial Simulation (F) Increase
<15%
59.0% (39.2%–67.7%)
.49
90.9% (84.6%–92.7%)
.18
≥15%
50.3% (44.8%–70.5%)
88.5% (81.5%–91.3%)
Initial Virtual Simulation Increase
<14%
59.0% (38.1%–67.6%)
.31
89.9% (83.3%–92.7%)
.50
≥14%
52.1% (45.2%–77.9%)
88.7% (83.1%–92.5%)
Initial Online Lecture Increase
<61%
54.4% (39.8%–82.5%)
.21
90.8% (87.1%–92.9%)
.08
≥61%
53.8% (42.9%–66.7%)
87.7% (81.5%–91.3%)
Mean Simulation (F) Increase
<12.5%
39.8% (38.1%–59.0%)
.04
88.5% (80.0%–92.7%)
.34
≥12.5%
63.0% (45.2%–86.7%)
87.5% (83.3%–93.1%)
Mean Virtual Simulation Increase
<20%
66.7% (39.8%–86.7%)
.05
87.5% (75.9%–92.5%)
.25
≥20%
45.2% (39.2%–59.0%)
88.5% (80.3%–93.1%)
Mean Online Lecture Increase
<46.25%
54.1% (39.2%–62.5%)
.23
88.7% (83.3%–92.7%)
.47
≥46.25%
63.0% (45.2%–67.6%)
90.2% (84.6%–92.5%)
Total Clinical Courses Changed
<50%
56.3% (39.2%–77.9%)
.43
90.8% (80.3%–92.7%)
.28
≥50%
58.7% (44.8%–67.2%)
87.9% (81.6%–93.1%)
Total Didactic Courses Changed
<37.5%
48.6% (39.5%–74.4%)
.27
92.6% (86.1%–94.2%)
.003
≥37.5%
62.8% (45.6%–67.2%)
86.7% (78.7%–89.9%)
Note. ADN = associate degree in nursing; BSN = bachelor of science in nursing; F = face-to-face simulation. Observed n varies across reported or tracked faculty characteristics. Data are expressed as median (interquartile range).
a Within-program test scores include Assessment Technology Institute, Health Education Systems, Inc., and Kaplan examinations.
Comprehensive NCLEX-RN results were available for more than 4,000 new graduates at participating programs. Across the 51 sites, the overall NCLEX-RN pass rate was 91.0% with outcomes ranging from 76.9% to 100%. Students’ first-time NCLEX-RN pass rate was 85.0%, with outcomes ranging from 69.2% to 100%. Like the within-program standardized examination scores, there was little variation in NCLEX-RN results by aggregate institutional characteristics. BSN (Mdn: 89.5%, IQR: 80.4%–92.7%) and ADN (Mdn: 88.5%, IQR: 83.3%–91.3%) programs reported comparable results, as did those institutions located in urban (Mdn: 88.0%, IQR: 80.3%–92.5%), suburban (Mdn: 88.3%, IQR: 82.3%–93.1%), and rural (Mdn: 91.1%, IQR: 88.5%–92.7%) areas. While programs’ summary NCLEX-RN results did not vary much based on their increased utilization of face-to-face and virtual simulation, they did correlate somewhat with the delivery format for lecture-based content. Specifically programs that reported the most pronounced increases in online didactic instruction documenting slightly lower overall NCLEX-RN results (p = .08). This difference was most evident for programs that reported the delivery formats for a larger proportion of their lecture-based courses had been updated and moved online due to the pandemic (p = .003). Such shifts were consistent throughout the 2-year window, but they were most pronounced during the 2020–2021 academic cycle (51% vs. 41.9%, Figure 5).
FIGURE 5Proportion of Lectures Completed Online by Term
Participating programs were also categorized into one of two groups based on whether or not they met or surpassed the 80% first-time NCLEX-RN passing threshold and were compared on an array of institutional characteristics (Table 11). Most programs in the study (n = 43, 84.3%) met this standard, while eight (15.7%) fell short. For programs that reported overall NCLEX-RN pass rates equal to or greater than 80%, outcomes ranged from 80% to 100%. Programs that documented a first-time NCLEX-RN pass rate below 80% had first-time pass rates that ranged from 63.6% to 78.7%. As with earlier comparisons on standardized examination results, there was little variation in NCLEX-RN results by aggregate institutional characteristics. Similarly high proportions of BSN (n = 22, 78.6%) and ADN (n = 19, 90.5%) programs reported first-time NCLEX-RN pass rates equal to or greater than 80%. In addition, no discernable patterns emerged based on region (p = .31) or reported difficulty of arranging in-person clinical placements for students (p = .42).
TABLE 11Program Comparisons on Aggregate First-Time NCLEX Pass Rates
In-program test scores include Assessment Technology Institute, Health Education Systems, Inc., and Kaplan examinations.
62.5% (38.1%–63.0%)
52.2% (42.9%–70.8%)
.38
Years in Operation
65 (43–76)
54 (37–58)
.21
Full-time Faculty
24 (13–40)
11 (8–22)
.11
Required Clinical Hours
630 (150–675)
697 (550–750)
.07
Fall 2020 Student Enrollment
37 (20–109)
54 (30–97)
.40
Proportion of White Students
75.0% (63.0%–84.0%)
60.0% (36.5%–82.0%)
.06
Proportion of Female Students
80.0% (78.0%–90.0%)
88.0% (83.5%–92.5%)
.20
Initial Simulation (F) Increase
25.0% (15.5%–25.0%)
34.0% (20.0%–50.0%)
.07
Initial Virtual Simulation Increase
20.0% (15.0%–35.0%)
9.0% (0%–19.5%)
.04
Initial Online Lecture Increase
65.5% (15.0%–92.0%)
80.0% (45.0%–100%)
.27
Mean Simulation (F) Increase
9.7% (1.0%–26.9%)
12.2% (2.5%–20.0%)
.41
Mean Virtual Simulation Increase
15.9% (3.2%–36.7%)
21.7% (9.5%–33.3%)
.23
Mean Online Lecture Increase
65.0% (30.0%–75.0%)
42.5% (36.6%–75.0%)
.48
Total Clinical Courses Changed
41.7% (20.0%–60.8%)
50.0% (28.6%–66.7%)
.26
Total Didactic Courses Changed
53.9% (37.5%–78.0%)
33.3% (25.0%–80.0%)
.20
Note. ADN = associate degree in nursing; BSN = bachelor of science in nursing; F = face-to-face simulation. Observed n varies across reported or tracked faculty characteristics. Data are reported as n (%) except for continuous variables, which are expressed as medians (interquartile ranges).
a In-program test scores include Assessment Technology Institute, Health Education Systems, Inc., and Kaplan examinations.
Consistent across other reported outcomes, programs that fell short of the 80% first-time NCLEX-RN passing threshold often relied on higher levels of virtual simulation (p = .04). By contrast, there was a trend toward programs that met or surpassed the 80% first-time NCLEX-RN passing threshold, often relying on higher levels of more established face-to-face simulation methods (p = .07). Shifts to increased use of both face-to-face and virtual simulation were most pronounced during the 2020–2021 academic year (40.4% vs. 27.2%, Figure 6). Often, the number of clinical hours a program required provided critical context for interpreting the potential impact of these types of trends, as it did in the National Simulation Study (
The NCSBN National Simulation Study: A longitudinal, randomized, controlled study replacing clinical hours with simulation in prelicensure nursing education.
). Here again, despite the limitations of a smaller institutional sample, a trend emerged regarding the number of required clinical hours as well. Participating programs that required fewer clinical hours (Mdn: 630, IQR: 150–675) tended to document lower first-time NCLEX-RN pass rates compared to programs that required more clinical hours (Mdn: 697, IQR: 550–750, p = .07).
FIGURE 6Proportion of Clinical Experiences Completed Through Simulation by Term
There was little variation in within-program standardized examination scores by aggregate institutional characteristics. Programs that reported pronounced increases in their utilization of virtual simulation, particularly those that indicated no additional institutional resources to support such a transition, documented significant declines in their ATI, HESI, and Kaplan results. NCLEX-RN results were consistent, with programs that fell short of the 80% first-time NCLEX-RN passing threshold often relying on higher levels of virtual simulation. By contrast, programs that increased their utilization of more established face-to-face simulation methods documented higher ATI, HESI, and Kaplan results and, more often than not, met or surpassed the 80% first-time NCLEX-RN passing threshold. Similarly, programs that reported the delivery formats for a larger proportion of their lecture-based courses had been updated and moved online due to the pandemic were more likely to report lower NCLEX-RN results, albeit still well above the national average.
Limitations
Despite WIRB approval and documented student consent, many programs in our sample opted not to share nursing students’ within-program standardized test information due to concerns regarding participant privacy and ambiguity over institutional policy. This, along with issues of attrition over the 2-year study period, resulted in standardized examination scores only being available for approximately half the consented student sample (within-program scores n = 331). Thus, this section provides only a partial snapshot of students’ examination performance and results should be interpreted with caution. Furthermore, given student study participants’ superior NCLEX-RN results (compared with overall NCLEX-RN results reported by NCSBN), it’s likely within-program scores are artificially deflated due to missing data. All findings of this analysis are correlational and do not support causal inference.
Conclusion
Despite the limitations and availability of standardized examination measures, several interesting patterns emerged that confirmed student and faculty self-report data. Namely, programs’ increasing reliance on virtual simulation and online lecture delivery after the onset of the pandemic correlated with lower student performance. The inverse was true of increased use of more established face-to-face simulation methods. Importantly, this is likely due to context. The simulation thresholds put forth by NCSBN (
) were rarely met and almost never exceeded in our institutional sample. Furthermore, the median number of required clinical hours in the set was 681 (IQR: 584–750). Both elements provide critical context for interpreting the potential impact of the utilization trends observed in this study, as they did in the National Simulation Study (
The NCSBN National Simulation Study: A longitudinal, randomized, controlled study replacing clinical hours with simulation in prelicensure nursing education.
). Adherence to established evidence-based guidelines on face-to-face simulation use appears to have resulted in consistently strong student outcomes.
New Graduates’ Early Career Outcomes
May 2022 marked the formal completion of the within-program data collection phase of the longitudinal study. This part of Phase Three focused on early career outcomes among the student participants in this study.
Methods
In May 2022, prior to graduation, exit information, including alternate contact information for all student study participants, was solicited to supplement the program email listserv. Then, at intervals of 3 and 6 months after graduation, new graduates were asked to provide details on their postgraduation experiences. For new graduates who were employed at one or more of these intervals, NCSBN research staff requested they submit responses to the NGNPS. In addition, new graduates were given an anonymous link to forward to a manager or direct supervisor familiar with their work to provide an additional evaluation. To encourage participation, a monetary incentive was offered for each valid submission, as well as a dual monetary incentive for both the manager and new graduate for each complete managerial submission. All surveys were administered using Qualtrics. The 3- and 6-month surveys launched on August 15, 2022, and November 15, 2022, respectively. Both remained open for 6 weeks, with three regularly scheduled weekly reminders. The managerial surveys remained open throughout the final 3 months of the early career data-tracking window, accessible with an anonymous link provided to participating new graduates.
Survey Tools
The NGNPS (Appendix D) was developed by the Nursing Executive Center of the Advisory Board Company. It consists of 36 items that assess clinical knowledge, technical skills, critical thinking, communication, professionalism, and management of responsibilities on a six-point Likert scale, where 1 = lowest rating and 6 = highest rating (
The NCSBN National Simulation Study: A longitudinal, randomized, controlled study replacing clinical hours with simulation in prelicensure nursing education.
). Evidence of convergent and discriminant validity was found.
As supplement, the Critical Thinking Diagnostic component of the survey instrument further assesses nurses’ critical-thinking ability using the same 6-point Likert scale across five items in each of the following areas: (a) problem recognition, (b) clinical decision making, and (c) prioritization. As before, the Critical Thinking Diagnostic also demonstrated good internal consistency, with a Cronbach’s coefficient alpha of 0.98 (
The NCSBN National Simulation Study: A longitudinal, randomized, controlled study replacing clinical hours with simulation in prelicensure nursing education.
). Evidence of convergent and discriminant validity was also found.
Variable Coding
Select independent variables were recoded based on their underlying values to facilitate group analysis. Most variable coding outlined in previous sections remained consistent. Coding related to programs’ changes to their course delivery formats from the prior section was again referenced. For this portion of the analysis, the only new variable coded was the number of years a student’s academic program had operated, which was recast as a binary predictor using the median value from the underlying distribution as a cut-point. Specifically, a program’s years in operation were defined based on values less than and greater than or equal to 54 years (e.g., the observed median value in the sample).
For the outcomes, scores were summed for each of the six domains of the NGNPS and the three domains of the Critical Thinking Diagnostic scale. For the NGNPS, each of the 6 domains (clinical knowledge, technical skills, critical thinking, communication, professionalism, and management of responsibilities) have 6 individual items. Thus, scores were summed by domain and then divided by six to aid interpretation using the original Likert scale of 1 to 6 (1 = lowest rating, 6 = highest rating). For the Critical Thinking Diagnostic scale, the three domains (problem recognition, decision making, and prioritization) each had five individual items. Thus, for these domains, summed scores were divided by five to again aid interpretation using the original Likert scale.
Data Analysis
New graduate and institutional descriptive data are reported at the practitioner level. Summary results include frequencies and proportions for all categorical variables, while continuous variables are expressed as means and standard deviations or medians and IQRs, as appropriate. All model-based results are expressed as means and standard errors. Due to the longitudinal nature of the data tracking, GEE models were used to assess the significance of observed trends to account for repeated measures. All analyses were conducted using SAS version 9.4, and p ≤ .05 was considered statistically significant.
Results
New Graduate Sample
The new graduate respondent profile strongly aligned with the overall institutional and student participant profiles (Table 12), confirming relative continuity throughout the study on broad sample characteristics. Overall, 187 unique new graduates participated in the postgraduation surveys for a total of 301 responses. Of the 187 graduates, 130 (69.5%) indicated they passed their NCLEX-RN at some point during the early career tracking, and 120 (64.2%) reported being employed. For the remainder of the results, the denominator is the 120 employed nurses. The mean age of nurses who submitted early career responses was approximately 26 (SD: 7.3) years. Most nurses self-identified as female (n = 107, 92.2%), non-Hispanic (n = 98, 84.5%), and White (n = 92, 79.3%). Nearly one-third of participating nurses (n = 39, 33.6%) indicated they were Pell Grant recipients. More participants were graduates from BSN programs (n = 70, 59.8%) than from ADN programs (n = 47, 40.2%). After graduation, there was a pronounced geographic shift among study participants, with a plurality of nurses indicating they attended nursing school in an urban area (n = 50, 41.7%) but nearly three-quarters (n = 81, 71.1%) identifying their primary workplace setting as urban. Nearly all respondents indicated they worked as an RN at a hospital/medical center (n = 107, 93.9%).
TABLE 12Summary of Employed New Graduates (N = 120)
Note. ADN = associated degree in nursing; BSN = bachelor of science in nursing; IQR = interquartile range. Observed n varies across reported or tracked student characteristics.
More than three-quarters of new nurses (n = 91, 82.0%) reported they had or were participating in a transition to practice residency. Similarly, most nurses worked 12-hour shifts (n = 79, 71.2%), with a plurality indicating employment on an 8-hour night shift (n = 45, 40.5%). Correspondingly, nurses reported working a mean of 37 (SD: 6.1) hours per week and approximately 12 (M: 11.7, SD: 1.5) hours per shift. While the median patient load for nurses in critical care units was two (n = 46, IQR: 2–4), the median inpatient load for most other inpatient nurses (e.g., medical-surgical, labor and delivery, etc.) was four (n = 101, IQR: 2–4). However, these numbers may be influenced by the ongoing nature of most new graduates’ transition to practice residency (see details in the following paragraphs). The median patient load for the three RNs who reported working in long-term care facilities was 22 (range: 16–24). Most nurses reported the difficulty level of their patients was “just right” (n = 87, 78.4%), but a sizable proportion indicated “too challenging or difficult” (n = 22, 19.8%).
Survey Findings
An assessment of the NGNPS results by nurse characteristics revealed very consistent outcomes across tracked demographic criteria (Table 13). Whether or not respondents participated in a transition-to-practice (TTP) residency program had little bearing on their self-reported ratings across clinical knowledge, technical skills, critical thinking, communication, professionalism, and management of responsibilities (all p > .05). However, by the 6-month survey administration, only five of the 91 nurses (5.6%) had completed their residency. Similarly, respondents’ sex, ethnicity, and Pell Grant status did not inform on any of their performance ratings (all p > .05). From the first survey wave (e.g., 3 months) to the second survey wave (e.g., 6 months), there were observed gains for critical thinking, communication, and management of responsibilities, but none reached the level of statistical significance (all p > .05). By contrast, there were significant gains in technical skills reported by participants over the 6 months (3-month M: 4.81, SE: 0.08 vs. 6-month M: 4.99, SE: 0.07, p = .02). Performance scores were comparable between the two time points for the remaining two domains of clinical knowledge and professionalism.
TABLE 13New Nurse Graduate Performance Survey Results by Nurse Characteristics
The most pronounced and consistent differences that emerged were by participant race. For five of the six NGNPS domains, non-White respondents self-reported significantly lower performance scores. For clinical knowledge, respondents who self-identified as non-White reported a mean performance score of 3.67 (SE: 0.14), compared to 3.95 (SE: 0.05) for their White counterparts (p = .01). Meaning, non-White respondents were less likely to rate their proficiency with the underlying skills that comprise this domain as highly as White nurses. Similar patterns emerged for technical skills (M difference: -0.48, p = .01), critical thinking (M difference: -0.52, p < .01), communication (M difference: -0.47, p < .01), and management of responsibilities (M difference: -0.69, p = .03). Even for the sixth and final domain, professionalism, there was an observed mean difference of -0.27, with non-White respondents reporting a mean of 4.93 (SE: 0.16) compared to 5.20 (SE: 0.05) for White participants.
A closer examination of career performance metrics revealed that the widespread disruptions to higher education wrought by the pandemic impacted nurses’ self-reported sense of proficiency across several domains (Table 14). Changes to program course delivery formats drove most observed differences. For instance, a pronounced reliance on face-to-face simulation in fall 2020 led to lower mean ratings on technical skills (M difference: -0.28), communication (M difference: -0.33), and management of responsibilities (M difference: -0.30) across early career nurses (all p < .05). This effect dissipated over the 2 years, though, as no significant effects were observed by mean face-to-face simulation use from fall 2020 to spring 2022 (all p > .05).
TABLE 14New Nurse Graduate Performance Survey Results by Program Characteristics
Note. ADN = associate degree in nursing; BSN = bachelor of science in nursing; F = face-to-face simulation; D = Delta/Change. Observed n varies across reported or tracked faculty characteristics. All estimates are presented as mean (standard error).
By contrast, early (e.g., fall 2020) adoption of virtual simulation did not drive differences across the six performance domains, but mean virtual simulation use over the 2-year reporting window did have an effect. Sustained and high uptake of virtual simulation over the 2-year period led to lower mean ratings within the critical thinking (M difference: -0.25) and management of responsibilities (M difference: -0.34, both p < .05) domains. Similar patterns emerged at higher thresholds of online delivery of lecture material from fall 2020 to spring 2022. Specifically, prolonged use of higher proportions of online lectures led to lower mean ratings on technical skills (M difference: -0.34), communication (M difference: -0.33), and management of responsibilities (M difference: -0.34, all p < .05).
An issue of scale, as measured by the overall proportion of clinical and didactic courses affected, also emerged. For instance, graduates of programs that reported increased face-to-face simulation or virtual simulation use for more than 50% of the clinical rotations were more likely to report lower proficiency within the technical skills (M difference: -0.27) and communication (M difference: -0.24, both p < .05) domains. In addition, graduates of programs that reported increased use of online lecturing, specifically for more than 37.5% of their didactic courses, were more likely to report lower perceived proficiency for their clinical knowledge (M difference: -0.21), technical skills (M difference: -0.35), critical thinking (M difference: -0.27), communication (M difference: -0.28), and professionalism (M difference: -0.24, all p < .05).
The results of the New Nurse Graduate Critical Thinking Diagnostic Survey were also crosstabulated by select nurse and institutional characteristics. As with the nurse performance metrics, the diagnostic results were largely consistent across tracked demographic criteria (Table 15). Neither survey timing nor TTP residency participation informed on nurses’ perceived diagnostic proficiency (both p > .05). Respondents’ sex, ethnicity, and Pell Grant status similarly did not correlate with their self-reported diagnostic ratings. As with performance outcomes, non-White respondents were less likely to rate their proficiency with decision-making as highly as White graduates (M difference: -0.29, p = .01).
Note. TTP = transition to practice. Observed n varies across reported or tracked faculty characteristics. All estimates are presented as mean (standard error).
Unlike career performance metrics, clinical transitions to simulation and remote learning did not appear to impact nurses’ self-reported sense of proficiency across the three critical thinking diagnostic domains of recognition, decision making, and prioritization (Table 16). There were no significant correlations between students’ diagnostic skills and programs’ pronounced reliance on simulation and online learning in fall 2020 (both p > .05). Similarly, prelicensure RN programs’ sustained use of distance learning strategies throughout the 2-year data collection period did not drive meaningful differences in nurses’ self-reported diagnostic proficiency (all p > .05). Interestingly, though, graduates of BSN programs consistently reported lower mean ratings than graduates of ADN programs on problem recognition (M difference: -0.-27), decision making (M difference: -0.24), and prioritization (M difference: -0.26, all p < .05).
TABLE 16New Nurse Graduate Critical Thinking Diagnostic Survey Results by Program Characteristics
Note. ADN = associate degree in nursing; BSN = bachelor of science in nursing; F = face-to-face simulation; D = Delta/Change. Observed n varies across reported or tracked faculty characteristics. All estimates are presented as mean (standard error).
Early career performance and diagnostic critical thinking metrics revealed that the widespread disruptions to higher education wrought by the COVID-19 pandemic impacted nurses’ self-reported sense of proficiency across several domains. Changes to programs’ course delivery formats drove most observed differences, but differences based on the type, magnitude, and timing of the changes emerged. For instance, a pronounced reliance on face-to-face simulation in fall 2020 led to lower mean ratings across select domains, but these effects dissipated over the 2-year period. Conversely, early (e.g., fall 2020) adoption of virtual simulation did not drive meaningful differences across nurses’ self-reported proficiencies, but programs’ sustained reliance on virtual simulation and online lecturing over the 2-year reporting window did. In both clinical and didactic settings, an issue of scale emerged. Nurses who graduated from programs that reported higher proportions of courses with updated and revised delivery formats frequently reported lower perceived performance and diagnostic critical thinking proficiencies.
In only one area did nurses’ demographic characteristics inform on any of their performance or diagnostic ratings consistently: participant race. Nurses who identified as non-White reported lower proficiencies across six of the nine tracked performance and diagnostic critical thinking domains. Minority nurses were less likely to rate themselves as proficient with the underlying skills that comprise the domains of clinical knowledge, technical skills, critical thinking, communication, management of responsibilities, and decision making. For select domains, graduates of BSN programs reported lower mean ratings as well. Only for the technical skills domain did recent graduates perceive a significant increase in their proficiency over the 6-month period.
Limitations
Since this was a voluntary opt-in research study, the new graduates who opted to participate may not provide an entirely representative snapshot of the outcomes at the participating prelicensure RN programs. Existing literature and the results of this study confirm that the lived experience of the COVID-19 pandemic is not universal but rather dependent on the personal, academic, and professional stressors it introduces. In addition, a detailed breakdown by new nurse graduate race is provided in the descriptive summary, but the low response n across non-White racial categories required that the variable be collapsed to a simpler White v. Non-White comparison for modeling. Finally, while new graduate participation in this study was robust, managerial feedback was unfortunately very limited and thus not included in the final analysis. It is unclear why tested outreach strategies that had an established track record of success previously failed to gain traction in the present study. Additional research on managers’ perceptions of new graduates’ performance and diagnostic skills is necessary. Finally, the findings of this analysis are correlational and do not support causal inference.
Conclusion
Early career survey results confirm a disproportionate impact of course delivery format changes on domains related to new graduate performance. These include clinical knowledge, technical skills, critical thinking, communication, professionalism, and management of responsibilities. In nearly all instances, the early career outcomes align with and illustrate the real-world impact of the deficiencies identified by students and faculty throughout their academic training and brought about by the pandemic. Unlike career performance metrics, clinical transitions to simulation and remote learning did not appear to have as pronounced an impact on nurses’ self-reported sense of proficiency across the three critical thinking diagnostic domains of problem recognition, clinical decision making, and prioritization. While some of the additional differences observed in the early career results (e.g., program type) tie back to course delivery format, one area of additional concern did emerge in the findings: race.
In part, the differences that emerged by participant race can be attributed to other documented patterns. For instance, non-White nurses were significantly more likely to attend an urban-based program (27.8%) compared to a suburban (16.3%) or rural (11.6%) institution. As documented prior, these institutions were significantly more likely to rely on remote and simulation-based learning environments. Perhaps this was due to both the severity of the COVID-19 infection rates in those areas as well as the restrictions put in place to address them. Nonetheless, a few other criteria that displayed similar degrees of overlap with changes to course delivery format earlier, such as region, program type, etc., did not emerge in this section. This suggests that the perceived deficiencies self-reported by non-White nurses are likely more nuanced and warrant further investigation.
Phase Four: Focus Group Outcomes
The fourth and final phase of this study focused on participants’ lived experiences during the pandemic. This qualitative research revealed the untold experiences of faculty, students, and administrators and the lessons learned during this crisis. This portion of this longitudinal, mixed methods study illuminates the transient and everlasting impacts the pandemic has made on educating students for years to come to include those humanistic factors vital to teaching and learning in nursing education.
Methods
The qualitative section of this multi-site national study incorporated a hermeneutic phenomenological methodological approach. The purpose of this qualitative phase was to address the following research question: What was the experience of faculty, students, and administrators in nursing education during the COVID-19 pandemic? Phenomenology was used in this study to emphasize the phenomenon (ie, the pandemic) as it presents through the lived experiences of the participants (
). Through hermeneutic phenomenology, multiple interpretations may be derived from the experience, and the researcher thus remains open to what may be revealed through interpretation of the text.
Sample and Sessions
Focus groups were created based on a purposive sample selected from the faculty, students, and administrators who agreed to participate in the previous phases of this longitudinal study beginning in August 2020. Approval was obtained from the WIRB and participants were recruited via email with an incentive of $100 for participation. The qualitative focus group sessions were conducted in June through August 2022. The participants were placed in discussion groups distinctly associated with their role as faculty (4 groups; n = 26), students (3 groups; n = 22), or administrators (2 groups; n = 16). The group sizes ranged from 5–9 participants per discussion. The focus group approach allowed for the creation of dialogue and collaboration with others in phenomenological research (
). The individual voice of the participant was preserved by the researcher through an open approach by inviting every participant to share their experiences.
The focus groups were facilitated using a web-based, virtual modality (e.g., Zoom). Participants were placed in a virtual waiting room until the research group confirmed each participant was permitted to join the discussion. Participants could choose whether to use the camera and/or chat feature, if desired, and the audio from each session was recorded. The semi-structured interview included the overarching question about the nursing education experience during the COVID-19 pandemic with prompts regarding virtual and online learning, simulation, personal and professional challenges, and open time for anything participants thought was important to share. Each group discussion lasted 60–90 minutes. After the interview, the audio recording was transcribed, and pseudonyms were provided for the participant names to ensure anonymity.
Data Analysis
This hermeneutic phenomenological approach focused on the experiences of faculty, students, and administrators during the COVID-19 pandemic. The researcher acknowledges pre-understandings of this study were not separated but rather brought forth to reduce interpretive bias. Participant text analysis was underpinned by the philosophical works of Heidegger and Gadamer. The hermeneutic interpretation of a phenomenon is shaped with multiple possibilities and made explicit through thematic analysis while engaging within the hermeneutic circle. Gadamer postulated the hermeneutic circle involves an interplay with the participant text and prior knowledge to uncover meanings of the lived experience (
). This interplay with participant text was further delineated in coordination with the NCSBN research team using a 5-step approach to data analysis with central tenets of Gadamer’s work as described by
The analysis of the text was coded using NVivo. The data were shared with the NCSBN research team through a secure cloud-based account where de-identified interpretations of the text were uploaded and discussed at regular intervals during each step of the data analysis for this phase of the study. Each member of the NCSBN research team participated in the reading of texts and discussion of possible coding and interpretations as led and provided by the lead qualitative researcher. The steps to interpretation are described in Table 17 following a Gadamerian framework (
TABLE 175-Step Process to Focus Group Data Analysis
Step 1: Research Question
Determined prior to data collection
Step 2: Pre-understandings
Researcher team’s pre-understandings were identified through current knowledge and preconceived notions
Step 3: Gain Understanding
Dialogue conducted with the participants to gain understanding; researchers remained open to possibilities
Step 4: Transcribing and Analyzing
Immersion – Recorded interviews were transcribed and reviewed twice for accuracy; participants were de-identified (pseudonyms were used and all identifying information was removed from the transcription); dwelling with the data and initial notes of interpretations were discussed with the research team while remaining open to multiple perspectives Understanding – Codes were developed in NVivo based on participant text; research team provided input into the codes Abstraction – Codes were merged for all 3 groups (faculty, students, and administrators) and common categories and subthemes were developed through the codes Themes – Synthesis of codes and themes was developed through interaction with the text, coding, and the underpinning philosophy to determine the relationship of the “parts to the whole” meaning of the interpretation Illustration – Phenomena were linked to the literature Critique – Final interpretations were developed and confirmed with the research team
Step 5: Trustworthiness
Trustworthiness was established for credibility, transferability, dependability, and confirmability (
Source: Adapted from Alsaigh, R., & Coyne, I. (2021). Doing a hermeneutic phenomenology research underpinned by Gadamer’s philosophy: A framework to facilitate data analysis. International Journal of Qualitative Methods, 20. https://doi.org/10.1177/16094069211047820
postulated that hermeneutics “is in the play between the traditionary text’s strangeness and familiarity to us, between being a historically intended, distanced object and belonging to a tradition. The true locus of hermeneutics is this in-between” (p. 295). The interpretation of the data for this study focused on finding the in-between through the interactions between the transcriptions of the focus groups and communication with the research team (Step 4). Edmund Husserl proposed that to see a phenomenon for what it was, the interpreter needs to bracket, or remove, any biases or presuppositions to understand the true essence of a phenomenon (
), which relates to Step 2 of the data analysis. Heidegger expressed an ontological approach to inquiry and was concerned with what it meant to be human (Dasein) as an inseparable part of being-in-the-world (
further expanded on the notion of being-in-the-world in that language creates a medium for understanding this experience through interpretive efforts. The interpretation can be described as a fusion of horizons that allows the researcher to see things upon the horizon rather than what is close at hand (
). It opens the possibilities for new understandings by furthering the horizon.
Results
The focus group analysis resulted in three themes and six subthemes that were indicative across the faculty, student, and administrator groups. The themes (Table 18) lend to a somewhat, albeit not exclusively, sequential nature of understanding, beginning with the first theme of Humanness of Nursing Education to the development of New Horizons for Healthcare in the end. The interpretive efforts provided insights for the future and lessons learned via theme development.
At the time when the world changed in March 2020 due to the COVID-19 pandemic, nursing programs had to quickly pivot to continue educating students. It was a period of many unknowns, and everything traditionally bestowed for educating students was challenged. Faculty were struggling to find new ways of delivering content, students had to adjust to a new learning environment, and administrators were frantically trying to find ways to provide structure and resources for this new way of being. It came to the question, “What was available to be re-purposed for education at a time of extreme uncertainty?” Technology became a central component in the delivery of education.
Adaptation
Participants discussed the various modes of educational delivery such as simulation, web-based lectures, videos, and creative remote teaching strategies (i.e., homemade laboratory materials). They emphasized how adaptation was needed, for example:Shawn (administrator):[S]tudents depended so much on the library for their Wi-Fi or at school or [a coffee shop]. When everything shut down, they didn’t have a place to study or to get online…. I know students were in the bathroom, literally doing the online lecture portion because they had shared an apartment with the family… they had kids that were school age, so everybody was online, and they didn’t have enough Wi-Fi…. Our school had to number one, provide [digital tablets] to our students. Number two, we had to figure out how to give them all hotspots.
While there were access modalities to be used via the internet, the experience of how to implement this way of teaching and learning was much different than traditionally experienced. Administrators needed to provide costly resources for students and faculty that were not budgeted. Many in-person experiences were eliminated for a time during the COVID-19 pandemic, and faculty needed to find a way to continue to support students’ education. Further expansion of technology was relayed from an administrator, who shared their experience of program adjustment in a rural setting:London (administrator):We had one student…who had to drive 150 miles to the campus and sit in their car and do all the work using the internet from the college.... It was very hard to catch cheating in these situations. The numbers actually make it look like it’s higher before the pandemic than during, and we really have no way of verifying what’s true, so it’s been a little frustrating that way.
It was not only about the cost financially, but the sacrifices students were making to stay in school. In addition, concerns were raised about the lack of security with the new systems, as indicated with the comments shared about potential cheating from many of the participants. This faculty member shared similar experiences of living in a rural area:Ali (faculty):I teach in a rural community college and pivoting to online was a huge challenge because I live in an area where many people don’t have high speed internet access in their homes…my program was all face to face, so no one on the nursing faculty was accustomed to teaching anything online…. We tried to accommodate all of those accessibility issues by providing laptops and hotspots for them to take home....
Participants shared how resources were needed to deliver content online and how obstacles to education delivery via the internet were not always overcome. Participants also shared their perspectives regarding why the system was not sufficiently prepared to support rural communities. Life for the participants also became more complicated with adapting to homelife, education, and the uncertainty of the pandemic.
Overwhelmed
The participants’ experiences with technological challenges extended to an overwhelming number of factors, as this student shared:Andy (student):They had us doing so many different computer programs… and it was just so difficult. I mean, what helped me was those specific teachers that were just really understanding and they [would say], “Oh, well you missed these assignments online.” Well, I didn’t even know. It’s on this program or this program. “Oh well, that’s okay. I’ll help you get it through. I’ll extend it for you.” Just that flexibility, I mean, that’s…what got me through.
The support and flexibility students received was critical to overcoming these seemingly insurmountable challenges. The participants were deeply rooted in the expected ways content was traditionally delivered in nursing education. The change in content delivery during the pandemic brought to fruition how these views of delivering content were transformed.
used the term enframing, which can be defined as a way to create order and is related to an aspect of teaching and learning experienced by the participants. Enframing can be illuminated through how participants strived to create order out of chaos during the pandemic, similar to a checklist of items. Through enframing, a danger can be explicated, as this student shared:Val (student):I don’t know if everyone was always well taken care of during nursing school as they would be if we were in person. We couldn’t go to office hours to talk to a professor. Sometimes [in the past] I’d sit on a couch with a professor who would just kind of talk things through and I have a little mini meltdown about nursing and it kind of turned to [during the pandemic], “Okay, if you want questions, it has to be Teams” [internet communication], which just isn’t the same, and so junior year was very hard for me personally and I almost considered dropping out of nursing school because of the emotional toll.
The overwhelming loss experienced by not being in person highlights the human factors in teaching and learning. Students shared their experiences of becoming a nurse, which was not necessarily only situated around content delivery.
The participants were facing incomprehensible stressors related to the lack of interpersonal interactions, death, and personal illnesses. Students feared the repercussions of not being able to attend clinical experiences and getting out of progression in the program. Participants also shared a tension during this time that resulted in many faculty and administrators quitting and retiring early. One faculty shared what it was like to leave at the time of the pandemic:Rowan (faculty):It was just lonely. It was the loneliest retirement. I just slipped away…even though we had Zoom and everybody said goodbye, everything was virtual, even the awards ceremony. I received an award, and it was just so weird.
The lack of connection and personal interaction was shared by many participants, and technology often did not successfully bridge this divide. When faculty quit, a tremendous shortage was met by the administrators, and they were faced with even more challenges to re-build a nursing program. Administrators were overwhelmed by the changes and how much the changes impacted the operations of the program.Kelsey (administrator):I’ve been in a faculty for years and have been in this position for now about 13 years. If I would’ve been an administrator for the first couple of years…I would not still be here because it was that challenging.
Administrators were tasked with enframing through resource allocation while faculty and students struggled to create order out of the content delivery. This enframing and search for order was desperately needed in a time of change and chaos. Participants were overwhelmed and lacked the human connections to make the multiple adaptations needed surrounding the experience.
Theme 2: Fostering Salience Through Turmoil
A nursing education program is structured to meet program, accreditation, and regulatory standards when preparing a student for their future role as a nurse. Faculty and administrators develop curricula and an environment conducive to didactic, clinical, and laboratory-based learning while maintaining safety for patients and standardized processes as students’ progress through a program. When students transition to nurses, a sense of salience has been described as going from both building a knowledge base and learning how to attend to clinical judgments about patient care (
). The transition that occurred during the pandemic while in nursing school was a time of great turmoil and the experience was reflected by the participants.
Engagement in Learning
Participants frequently focused on the experience of learning and adapting to the change from in-person to virtual delivery of course content across discussion groups. The change to this new modality of teaching and learning occurred at the start of the pandemic in the United States in March 2020. Students shared how they experienced engagement with this new way of delivering content:Adrian (student):[didactic] One thing our professors did was ask us, “What can we do to help more,” which is really nice. Everyone was like, “It’s boring.” That’s what we said, so their idea of engagement was… “Okay, now we’re going to go into breakout rooms as our engagement activity,” and then we were in our breakout rooms and no one really said anything, or we didn’t really understand what was going on.[virtual simulation][Y]ou always had an idea of what you were supposed to say, and the patients…were very predictable…so you never really had to think on your feet.Dorian (student):I did not retain a single thing. I would do the same thing like, “Oh, I can listen,” and maybe do something else, so I feel like I’m not bored because even just listening over online…there was nothing to engage with, and I feel like half of the professors almost read off of their PowerPoints…so then that was extra not engaging…. I never feel like I learned as much as I did my first year in person.
The learning modalities described above did not create a sense of salience in using clinical judgment. In contrast, the following students described engaging experiences in virtual simulation:Val (student):[Faculty] would assign the [mental health] modules and we would complete them and then we came in for a small group discussion where we discussed how we did on those assessments and what things we felt like were helpful or not helpful…I think that was a big part of the success of the virtual simulation. But also, I think for me, and even some of my other colleagues mentioned, that sometimes with the mental health patients or geriatric in person, you get kind of like, “Oh, what am I going to say?” so for me, it was a great option of practicing those phrases or what we would say because in a real life situation, if you have a manic patient and you haven’t had that before, sometimes you can kind of be like, wide eyed, deer in headlights…. Then, once I was in the ER with clinicals, I actually dealt with actively manic patients and used some of the same phrases that I practiced.Kai (student):We did the high-fidelity manikins…simulations online too…. You would do some of the simulations before you went into the high fidelity [simulation], it gave you like a practice run, and then you felt more confident going into the actual simulation.
These students described an experience of learning through simulated delivery methods. When students were engaged in debriefing with the faculty or preparation ahead of the experience, the connection with transformative thought was experienced. The student makes a transition in a nursing education program through comportment into thinking like a nurse, which can be met through the engagement with faculty as the guide to learning.
Faculty and students both experienced a transition in moving away from how we have always done it to a new way of thinking about nursing education. This faculty member shared their thoughts about the impact going through the pandemic has made:Dakota (faculty):I don’t know that we’ll ever go back completely to how it was because we’re finding that the shared experience and... debriefing is key… what we’ve learned out of the debriefing sessions from our simulation experiences has shifted how we deal with post-conference and debriefings in clinical as well. But that supportive processing is so important in ways that we knew, but we know at a different level now.
Through this transition, faculty participants recognized the true value of debriefing and allowing students to hear how faculty think through a patient case. Teaching and learning modalities during the pandemic provided insight into the experience of changing teaching strategies. One faculty member described some creative approaches to addressing the challenges introduced by clinical site restrictions and some efforts to provide grace to students during this difficult period.Micah (faculty):People were using the neck of a 20-ounce pop bottle to be a catheter… We brought them in 2 weeks early and ran a boot camp of sorts to kind of get them through the skills, lots of grace…. We kind of caught them up over the year because... I couldn’t fail them out of a program for not being able to pass the skill that they really didn’t have any opportunity to practice…. When we did come back in the fall, we were back, full time, but a lot of facilities weren’t letting our students in, so we’d have a group of 10 students, they’d only allow five students.
Faculty used creative teaching and learning strategies to engage students in the experience. Over the 2 years, full capacity in clinicals was a continued challenge and faculty needed to overcome many obstacles to increase engagement in learning.
Power
The focus of nursing education has typically centered around student learning. The COVID-19 pandemic caused educators to shift attention toward task-oriented procedures and addressing operational aspects that were not part of the focus in the past. Many prelicensure RN programs shifted to virtual clinical simulation or made other adjustments to their delivery formats in response to shifting state regulatory standards. Some states moved to allow a waiver for 100% virtual learning, while others did not place a percentage on the shift. In addition to program adjustments, there was the task of monitoring COVID-19, as this administrator described:Ainsley (administrator):I loved my role as [an administrator]. It used to be very focused on building community partnerships and on our practice partners and working with the workforce development people in the state and the Department of Health and really sort of expanding interprofessional educational opportunities…. I had to abandon that except for things I was already grant funded to complete. I abandoned about 80% of that work because my role ended up being the keeper of documents. Who’s vaccinated, who’s not? Who signed an attestation form, who didn’t? Who’s been fitted for their N95?
The shift in focus from program development to COVID-19 management created a disproportionate and continued demand on addressing current roles, while creating an entire new system of tasks. Administrators and faculty were required to address these competing demands, which expanded to state and national perspectives.
Differences in how local and state officials addressed COVID-19 varied widely across the nation, and that greatly informed on what program changes were implemented and how. The administrators adjusted to rapidly changing policies from the organization, state, accreditation, and national aspects, and this included times when there were discrepancies between state policies and accreditation. States wanted programs to increase student enrollment, while the policies being proposed did not match regulatory, accreditation, or program standards, as this administrator described:Eden (administrator):[State legislatures] said that any program could expand as much as they wanted to, without any hindrance from the board of nursing…previously the board would tell us you’re accredited for [baseline numbers] and if you go more than the percentage of this, you have to ask for permission and make sure that you have enough clinical sites and faculty in order to support your request…. [The state legislature] [also] didn’t want them to stop working as nurses just because they couldn’t pass the NCLEX…a lot of my time has been taken up with political advocacy, working with both the [state] Nurses Association and the [state] board of nursing to try to correct some of these bills.
This administrator also described state legislation that was different than the state BON regulatory standard. Administrators were faced with being political advocates on top of handling administration changes to policies for their programs. In addition to these challenges, administrators were navigating the burden of losing faculty, as one administrator described the twofold impact of early retirements offered at the university layered on top of a hiring freeze:Shawn (administrator):We were imposed on a hiring freeze. We were not able to replace any of our full-time faculty who took the early retirement, but at the same time, the college suffered a tremendous amount of loss of students. They were pressuring us to increase our enrollment and we also got pressure from the community because they needed more nurses, and so, they were asking us to increase enrollment and create new programs.
Participants were navigating an experience of power shifts in nursing education that extended well beyond the walls of the program. The impact on these changes were experienced at all levels and continued to pervade the personal lives of faculty, students, and administrators.
Participants were feeling the pressures of school, politics, and policies while dealing with the devastating impacts of lives being lost and patients dying alone during the pandemic. Participants discussed this perspective while trying to continue maintaining high standards in nursing education. Confusion existed, and participants shared multiple perspectives regarding personal aspects during the pandemic:Carson (student):I felt forced into getting the [COVID] vaccination, even though I wanted to wait just a little bit longer to see how my life plans worked out. I think that the vaccine mandate really is why I completed the program because if it were up to me, I probably would’ve waited…. I’ve wanted to be a nurse for as long as I can remember. Am I really going to give up all that I had worked for, for my curiosities about a vaccine?
It was an experience of the unknown at the time for both students and faculty. The struggle with making decisions about life circumstances at the time of the pandemic is one many participants shared. Students were faced with changing to adapt to the pandemic in addition to maintaining academic progress in the program based on those decisions. These types of challenges were also shared by administrators who further discussed the COVID-19 vaccine:Chris (administrator):We had… students who chose not to get vaccinated, and so, this was a significant issue with finding them clinical sites, getting COVID-19 vaccine exemptions…. [We] even had an attorney [who] threatened to sue the state board of nursing, threatened to sue us, that has caused a lot of professional concerns, and that’s part of the reason why we’ve lost some of our faculty members.
Administrators were pulled away from the routine aspects of their positions to attend to legal and student clinical site compliance issues. Regardless of beliefs or misinformation campaigns, the challenge of this experience impacted programs at all levels. The challenges experienced by the participants moved the focus away from teaching nursing to dealing with the public health emergency. The students needed to engage in the learning experience while faculty and administrators were constantly pulled to address issues inherent in policy and procedures, which highlighted the fostering of salience through turmoil.
Theme 3: New Horizons for Healthcare
Nothing in this century has impacted healthcare in such a way as the COVID-19 pandemic, and these experiences have opened the possibility for a new understanding to emerge.
discusses the concept of horizon, which suggests the breadth of vision that a person needs in order to see a new way of understanding. The horizon is a way to expand upon how the participants came to understand nursing education during the pandemic.
Previous Ways of Knowing
Participants discussed ways they have learned from COVID-19, both through previous understanding and new perspectives. One participant shared some of what was lost because of the pandemic:Carol (faculty):[Students] miss the role modeling in what it looks like to be a professional, what it looks like to be a nurse. How do you talk to a patient who’s in distress? How do you talk to a crying family member? These are all pieces they missed. And you only get that with experience. You cannot get that on a screen of a computer. You get that from being in the moment with people going through hard things and that’s how you learn how to maneuver and navigate that situation.
The faculty shared that the students lacked communication techniques and professional role development with learning to be a nurse. Progression through a nursing program and the process of learning to become a nurse should be a transformative experience, but without in-person interactions, certain aspects were lost. Administrators also discussed how change was inevitable at this point. As one participant put it:London (administrator):I’m not sure that our education system is ever going to go back to full-time classroom successfully. I have a feeling that online teaching is here to stay, even in nursing and at least partially or some hybrid. And I really hope that schools at least keep that open as a possibility in these programs where nursing is probably one of the hardest to teach online. How do you do this?
As this administrator shared, some hybrid modality was beneficial to nursing education. The one question that this administrator rhetorically asked was, “how do you do this?” referring to content delivery being hybrid and presented in an effective way. Students also commented on transforming nursing to meet the new challenges confronting the healthcare industry. Many of the most experienced nurses have retired early or left positions in many healthcare systems, leaving new graduates with less experienced nurse mentors. Student participants shared an experience of learning on their own or missing some of the key aspects that nurses with more experience had to share about practice. One student summed up the experience in the following way:Val (student):[N]ursing education does need to change now with the world that we’re in…because what ends up happening is we learn best-case scenario…and that never happens in real life…. It is this gap of “how do you nurse in the 21st century.”
The participants shared how the pandemic made impacts in nursing education, from letting go of previous ways of knowing while learning to function in a transformed healthcare system.
New Beginnings
In times of extreme change, new lessons are learned. Participants focused on many ways to change nursing education. Several faculty were continuing to work on research projects, improving approaches to teaching and learning, and securing tenure appointments during the pandemic on top of all the other challenges. Faculty related experiences that expanded students’ thinking skills, such as the following:Sage (faculty):Instead of debriefing at the end [of clinical], we’ve instituted it in the midpoint of the shift…. It really worked after the pandemic because they were allowed to grow. We could assess their strengths and weaknesses in clinical by doing these structured questions and help them with prioritization and steer their thinking…. Significant growth occurred because they were allowed to come back after lunch and…asked better questions.
The faculty experienced challenges, but lessons were learned about ways to expand thinking in teaching and learning, such as debriefing in the middle of a clinical day like this faculty member shared. Several students commented on the experience of working in healthcare while in school during the pandemic, which provided an expanded way of understanding a different horizon related to learning. They thought certain teaching strategies were helpful for their learning, as one student relayed:Morgan (student):Just having those resources readily available that you can just hop on and access at any time are really great. Some of our zoom lectures were recorded, so having the ability to go back and re-watch was helpful. Then I think having PowerPoints and things that you could look at and reference was really helpful to me.
The accessibility of having videos was not a common practice in nursing education prior to the pandemic. The expectation was to be in-person for class and video recordings were not provided because hybrid learning was not standard practice. The pandemic shaped the content delivery for an optional web-based format and recordings for students who were sick or caring for family, and this change was perceived as beneficial.
Participants shared how the delivery of clinical instruction was experienced during the pandemic. The following quotations reflect some of the common perspectives shared among participants:Adrian (student):I just feel like nursing school does need to get with the times. And don’t be afraid to send your students into whatever’s going on if there is ever a future pandemic. That’s what we signed up to do. So that’s what we need to learn how to do.Ainsley (administrator):[New graduates are seeking roles] in primary care, so the federally qualified health centers and things like that, but some into more traditional public health and community health, home healthcare; agencies are offering new grad residencies around here and so that’s been an interesting transition. It’s not a huge number, but it’s definitely an increase from the number of students we used to see…. We have more students not choosing acute care as new grads.
The perspectives the participants shared about future possibilities create an opening for new horizons that reshape the previous ways of knowing and doing things as they always were done into potential new beginnings for nursing education.
Discussion
The three main themes and six subthemes revealed a sequencing of events from personal interactions to the chaos experienced during the pandemic and ultimately to the development of new ways of thinking about nursing education. The subthemes provided a depiction of the experience that describes what it was like for the participants going through the pandemic. The previously untold stories of the participants were revealed and new ways of approaching nursing education were discovered.
The Humanness of Nursing Education theme was discovered over the integration of technology via the internet. The participants adapted to the change in content delivery during the pandemic as a way to create order. Heidegger discussed an approach to order from his seminal work in The Question Concerning Technology (
): “Everywhere everything is ordered to stand by, to be immediately at hand, indeed to stand there just so that it may be on call for a further ordering. Whatever is ordered about in this way has its own standing” (p. 322). The object that was standing by to be used during the pandemic was the internet. The essence of technology is not reduced to a material object; rather, it is a phenomenon that provided a more profound understanding of the experiences of participants during this time of disarray. The use of the internet could be viewed as what
referred to as ready-to-hand. The internet was used as a tool to provide order (enframing) to educational tasks, like a hammer is a tool used for building objects. Through the integration of the internet, obstacles were met, and
referred to this as being un-ready-to-hand or obtrusiveness. The participants used the internet to deliver content because it was available as a means to an end, but through the overwhelming obstacles, the humanness of the experience was lost.
By bringing forth the essence of technology in nursing education at the time of the COVID-19 pandemic, we revealed a truth (aletheia) through the experiences that was not known at the time. These understandings cannot be measured or quantified but experienced at a point in time, thus fusing together the past, present, and future during the experience. It was not about the instrument used but the human connectedness of faculty to expand students’ thinking through and about clinical situations, thus leading students to the comportment of being nurses. In this study, there were aspects of preparation in terms of the technicalities of the role, but as identified by
, more research is needed related to humanistic characteristics of the experience. In nursing education, there was much to be lost if the dialogue and interactions between students and nurse experts were not emphasized during the program.
The theme Fostering Salience Through Turmoil was discovered as ultimately preparing students to be ready for practice during the challenges of the pandemic. A great deal of confounding factors created obstacles to engaging in learning, and many of these factors were outside of participants’ control. Through these challenges, opportunities were generated for understanding how salience was fostered in a nursing program.
The concept of salience included how the new graduate nurse is prepared through educational experiences. The idea of being ready to practice is based on a multidimensional concept that includes personal, professional, clinical, and industrial factors (
) and continues to develop after graduation as a nurse. Factors that can influence the student being ready to practice include the effectiveness of academic clinical experiences and paid healthcare positions while in school (
). The quality of the educational experience (didactic, clinical, and simulation) is not completely understood, and it has been identified that there is a need for more robust research for how to prepare the next generation of nurses for the clinical setting (
performed a virtual educational intervention on nursing students and determined that effective use of both virtual and face-to-face characteristics in addition to effective educational interventions can lead to the development of clinical reasoning. More research examining these hybrid approaches to teaching and learning along with summative and formative measures will help expand the robust strategies needed in nursing education.
recognized that students need realistic expectations of nursing to be prepared to practice, including the physical, emotional, and social aspects of the role. The results of this study emphasized those aspects of professional values in the role development of the student in addition to a need for learning clinical reasoning.
The students in this study were challenged during the pandemic to engage in learning and are now entering the nursing workforce—a workforce that has changed dramatically due to the pandemic, including a decline in healthcare safety (
identified that the practice environment receptiveness to the new graduate, with less emphasis on the education leading up to the transition, made a tremendous impact on being ready to practice. An urgent demand exists for practice partners to respond to the needs of new graduate nurses and their transitions, as they will need to develop a sense of salience in this new healthcare world. As one student asked, “How do you nurse in the 21st century?” Support is needed for a student to become confident, satisfied, and competent in the role of the nurse (
introduced the fundamental ways of knowing in nursing that included empirical, aesthetic, personal, and ethical tenets. While these concepts were prevalent in this study, much more was experienced by the participants that extended well beyond those characteristics. Administrators were pulled to address policies and process-oriented concerns, faculty were challenged with new ways of delivering content, and students were balancing multiple competing priorities; all of these demands created a sense of loss of what was previously known in nursing education and an opportunity for new beginnings. Through this process, both the historical horizon built on past experiences and the present horizon fuse to form a new level of understanding (
). The participants changed their horizons through this experience by adjusting to changes in nursing education and realizing how the new world of healthcare is different. The tremendous navigation of program, state, and regulatory changes were a major part of the transitions and program stability, leading to a different understanding of the horizons. The pandemic challenges resulted in numerous losses in nursing education, but through loss, a new horizon was discovered that has opened future possibilities.
Limitations
While the sample size provided a wide net of participation in this qualitative work, generalizability may be limited. The purposive sample of selection within the current study was not randomized to the general population. Some participants communicated less than others, which may have prevented all participants from sharing their perspectives. The limitations of the focus group should be acknowledged, as some participants may have been more open if they were in a personal interview. Since this was a focus group, some participants may have been hesitant to share the full disclosure on their experience.
Conclusion
The value of the communal experience in nursing education was expressed by the participants. The engagement through a convergence of conversations and coaching by faculty through a series of questions can help guide students to a sense of salience and clinical reasoning (
). Students need to be challenged in how they think through clinical situations by avoiding repetitive, predictable modes of learning. Faculty can help students by challenging them to think through multiple perspectives while remaining supportive and present through the experience. The experience during the pandemic was one that revealed the way of “being in the world” of nursing education and brought new perspectives to the humanness of the experience.
A call for transformation in nursing and nursing education has been widely published in the literature (
), but nothing has impacted change quite like the COVID-19 pandemic. Participants learned to challenge assumptions in nursing education during the pandemic, and through those experiences, they brought new perspectives to what was important to faculty, students, and administrators in nursing education. Participants often focused on creating order, but it was quickly realized how order can get in the way of progressing thinking. The participants expressed the value of hybrid modes of learning and the usefulness of debriefing; however, the importance of effective implementation within those delivery systems was stressed. A great opportunity exists to enact change in nursing education. A new healthcare system with resilient approaches during a crisis aimed at maintaining high levels of safety and infection control, both of which were adversely affected during the pandemic in the United States (
Continued increases in the incidence of healthcare-associated infection (HAI) during the second year of the coronavirus disease 2019 (COVID-19) pandemic.
), are desperately needed. These approaches must begin by effectively educating the next generation of nurses entering this dynamic workforce. Now is the time to disrupt nursing educational models and expand the robust educational research needed to effectively develop the next generation of nurses who are ready to practice in this new healthcare landscape.
Summary
The onset of COVID-19 in the United States in early March 2020 (
) and significantly disrupted traditional educational models (AACN, 2020). Compounding these issues further were the deleterious effects of the well-intentioned and often necessary policies put in place to mitigate the spread of the virus on prelicensure nursing education programs (
). This comprehensive four-phase longitudinal study provides substantial evidence on prelicensure RN students’ academic, engagement, and early career performance over the past 2.5 years. By systematically tracking the outcomes of nursing students in the spring 2022 cohort, NCSBN effectively captured in real-time the experiences of those undergraduates entering the core of their didactic and clinical nursing coursework during the COVID-19 pandemic.
The 51 prelicensure RN programs that participated in the study hailed from 27 states. They ranged from smaller private not-for-profit institutions with fewer than 20 nursing students to large flagship public institutions with nursing program enrollments in the hundreds. The summary results underscored the geographic, programmatic, and demographic diversity of our retained sample. This was evident both in terms of program characteristics, including program type (54.9% BSN, 45.1% ADN), setting (45.1% urban, 51.0% suburban/rural), and tax status (68.6% public institutions), as well as the racial (63% White) and ethnic (9% Hispanic) composition of their nursing student populations. Incorporating feedback from more than 1,100 student and faculty participants, including more than 4,000 course observations, this important work captures the breadth, scale, and ever-evolving nature of prelicensure RN programs’ changes to their course delivery formats during the public health crisis. In doing so, it illuminates the many innovative ways prelicensure RN programs sought to address the unparalleled challenges they confronted and provides the mechanisms for measuring their efficacy and possible implications for patient safety.
The Shifting Landscape of Prelicensure RN Education
Seemingly overnight, nursing programs were forced to hastily pivot their lecture content to online course delivery formats and their patient care clinicals to computer-based simulation or virtual/augmented reality settings (
). Nonetheless, our study found that most programs generally worked within the confines of long-established and evidence-based guidelines. In July 2020, prelicensure RN programs reported significant increases in the anticipated use of face-to-face simulation (+15%), but a majority did not report plans (n = 499, Mdn: 30%; IQR: 20%–50%) to exceed established guidelines (
The NCSBN National Simulation Study: A longitudinal, randomized, controlled study replacing clinical hours with simulation in prelicensure nursing education.
). Within the sample of programs that participated in our study, the proportional use of face-to-face simulation, as reported by faculty participants, confirmed an even more subdued application of related delivery formats (Mdn: 10%, IQR: 0%–25%).
The ascent of virtual clinical simulation also predated the pandemic (
). During the early stages of COVID-19, however, the relatively low cost, general availability, and range of virtual options presented a particularly appealing option to nursing educators and administrators (
). Thus, the need for rapid adaptation coupled with limited resources led many prelicensure nursing education programs to increasingly rely on virtual modalities to counteract clinical site restrictions (
). While 80% of programs that participated in our summer 2020 baseline survey planned to incorporate virtual simulation instruction to some degree, the change in the actual usage thresholds was relatively mild given the extent of the crisis (+20%). Still, the magnitude of this shift is perhaps made more evident by the proportion of programs that offered no virtual simulation hours pre- and postpandemic onset (fall 2019 n = 130 vs. fall 2020 n = 11). Nonetheless, the evidence suggests that most prelicensure RN programs moderated their use of this instructional medium (Mdn: 25%, IQR: 15%–50%), perhaps due to the absence of evidence-based guidelines. Within the sample of programs that participated in our study, the proportional use of virtual simulation was even lower (Mdn: 10%, IQR: 0%–25%).
Shifts to online delivery of lecture content were even more pronounced. This was true across the prelicensure nursing education landscape (+60% compared to fall 2019) and within our institutional sample (+48%). Strikingly, the number of programs that offered no online lecture hours decreased from 167 in fall 2019 to just 21 in fall 2020 (p < .001). By contrast, the number of programs that offered all of their lecture hours online increased from 10 in fall 2019 to 153 in fall 2020 (p < .001). Taken together with evolving modes of clinical instruction, the evidence paints a picture of a significant shift in the ways in which prelicensure RN students were educated over the past 3 years.
Within-Program Student Results
Unfortunately, as scholars have documented (Kardong-Edgren, 2019;
), the rapid adoption of virtual clinical simulation in nursing education has not been without growing pains. A consistent trend that emerged from this study was the superior outcomes documented, by both students and faculty alike, for in-person clinicals or face-to-face simulations vis-à-vis virtual simulated environments. These results manifested time and again in a variety of ways, including observed patterns in the reported results by program setting, type, students’ Pell Grant status, and the timing of students’ clinical rotations. In each instance, programs and students that fit certain profiles, such as students who self-identified as non-White and attended urban-based BSN programs, were significantly more likely to rely on pronounced increases in face-to-face simulation or virtual clinical simulation. Perhaps this was due to both the severity of the COVID-19 infection rates in those areas and the restrictions put in place to address them. Similarly, clinical rotations that were more likely to occur during the 2021–2022 academic year, such as advanced medical surgical and maternal-newborn, often confirmed students’ greater mastery of associated proficiencies compared to frontloaded topics, such as fundamentals, when various modes of simulation use peaked.
Interestingly, one contrast that emerged between the student and faculty self-report data was the effect of the passage of time. For students, the effects of the pandemic appeared to dissipate somewhat over time as the pronounced use of virtual simulation abated. By contrast, faculty observations and ratings of clinical competence gradually declined over the reporting period, with notable reductions in student engagement (32.0% much/less engaged) and work quality (21.6% much/poorer quality) documented over the 2-year period. The dissonance between these trends may perhaps reflect the point-in-time difficulties for students navigating the constantly shifting landscape, particularly during the first year of the pandemic. Overall, these trends may suggest links to early disruptions driven by surges in COVID-19 infections and hospitalizations, the loosening of restrictions over time, and simply students’ acclimation to new conditions. By comparison, faculty perceptions of students’ inability to scale their knowledge may signal the end of a “grace period” for relatively inexperienced (median 3 years’ experience, 2.4% CHSE certified) simulation educators’ evaluation of students’ work quality and progression.
Overall, evidence from early in the pandemic suggests some prelicensure RN programs employed unproven virtual modalities for traditional clinical hours and strayed from even the few foundational elements underpinning virtual simulation (
). While improved student ratings over the 2-year period stand in apparent contrast to the inverse trend observed for faculty CCEI evaluations, these seemingly divergent patterns may speak to one general truth: Programs and faculty learned from their initial trial and error during the first few academic terms of the COVID-19 pandemic and simultaneously recognized the likelihood that sustained reliance on virtual teaching methods would be necessary. As a result, the gradual return to normal, including once again relying on more traditional modes of learning, during the second of the 2 years of within-program data collection, may have led to improved student ratings. At the same time, the faculty’s prolonged use of and acclimation to new teaching methods may have increased their proficiency with such resources and in turn their expectations regarding students’ performance.
Furthermore, the near wholesale shift in online delivery of lecture content inevitably impacted students’ learning (24.6% much/poorer quality) and engagement (35.9% much/less engaged) outcomes as well. In-person (affective, psychomotor) and hybrid (cognitive, affective, psychomotor) learning consistently surpassed reported outcomes for online lectures. Similarly, students who enrolled in in-person and hybrid lecture courses also reported higher levels of engagement compared to those in online learning environments. Interestingly, these were among the few patterns that held for standardized examination scores as well, with programs that reported a larger online presence more likely to report lower NCLEX-RN results, albeit still well above the national average.
As other studies have documented, emergency guidance from BONs likely played an essential role in supporting and directing prelicensure nursing programs’ activities during this turbulent period (
) and regular communication with nursing programs early in the pandemic, BONs not only provided the necessary flexibility to ensure the continuity of student learning (
The NCSBN National Simulation Study: A longitudinal, randomized, controlled study replacing clinical hours with simulation in prelicensure nursing education.
). As a result, although this study confirmed that pandemic disruptions to traditional academic teaching models led to significant shifts in students’ self-reported learning and engagement, as well as faculty assessments of student competencies, the effects may have been somewhat mitigated.
Postgraduation and Early Career Outcomes
Given the systemic shock presented by the pandemic and the range of strategies employed by nursing education programs to counter it, it is unsurprising that emerging evidence on student outcomes has been mixed. Perhaps the most alarming trend documented to date is the decline in first-time NCLEX pass rates (down 7% – 8% since 2019) for U.S.-educated graduates (
). By comparison, programs enrolled in our study outperformed the national standard in this category, with little variation documented across within-program and NCLEX-RN standardized examination scores by aggregate institutional characteristics. In those instances where some preliminary distinctions emerged, patterns were consistent with faculty and student self-report data. Namely, programs that reported pronounced increases in their utilization of virtual simulation, particularly those that indicated no additional institutional resources to support such a transition, documented declines in their ATI, HESI, and Kaplan results. In addition, programs that fell short of the 80% first-time NCLEX-RN passing threshold often relied on higher levels of virtual simulation.
By contrast, programs that increased their utilization of more established face-to-face simulation within preestablished evidence-based guidelines documented higher ATI, HESI, and Kaplan results and often met or surpassed the 80% first-time NCLEX-RN passing threshold. Despite the noted limitations and availability of standardized examination measures in this study, results appeared to confirm that adherence to established evidence-based guidelines on face-to-face simulation use aligned with stronger student outcomes. In addition to seldom exceeding the simulation thresholds put forth by NCSBN (
), programs in our sample also required a median of 681 hours of clinical training. Both elements provide a critical lens for interpreting the potential impact of the utilization trends observed in this study, as it did in the National Simulation Study (
The NCSBN National Simulation Study: A longitudinal, randomized, controlled study replacing clinical hours with simulation in prelicensure nursing education.
Similar findings have resulted in increased calls for future work assessing how possible learning deficiencies inform new graduates’ early career outcomes (
). In this study, early career performance and diagnostic critical thinking metrics did indeed reveal consistent results, albeit with some nuances. Although changes to programs’ course delivery formats did drive most observed differences, domain-specific results varied based on the type, magnitude, and timing of the changes. For instance, a pronounced reliance on face-to-face simulation in fall 2020 led to lower initial ratings across select domains (technical skills, communication, and management of responsibilities), but these effects dissipated over the 2-year period. This perhaps reflects programs’ learning curve even in implementing evidence-based—but still new to programs—clinical simulation practices.
The inverse was true of virtual simulation use. Programs’ sustained use of virtual simulation over the 2-year reporting window led to lower mean ratings on critical thinking and management of responsibilities. An issue of scale also emerged, as graduates of programs that reported increased use of simulation or virtual simulation for more than 50% of the clinical rotations were more likely to report lower proficiency for their technical and communication skills. As the pandemic recedes, virtual clinical simulation now appears to be an established component of the nursing educational landscape (
). Given the consistency of our findings, and with broader adoption seemingly inevitable, further research on the best tools and methods to ensure comparable student learning and engagement outcomes in virtual clinical environments is critical.
Shifts to online delivery of lecture content were also associated with drop-offs in students’ self-reported proficiency across clinical knowledge, technical skills, critical thinking, communication, professionalism, and management of responsibilities. Despite the bulk of literature focusing—and rightly so—on applied clinical education, it’s imperative to remain cognizant of parallel disruptions to didactic course delivery methods. Course-for-course, the disruptions documented in this study were even more pronounced and widespread for students’ lecture content. In part, this appears to have contributed to students’ self-reported perceptions of learning loss, lower engagement, and reduced proficiencies across early career outcomes.
For the most part, additional differences observed in early career results (e.g., program type) tied back to course delivery format; however, pronounced and consistently divergent patterns also emerged by participant race. New nurse graduates who identified as non-White reported lower proficiencies across six of the nine tracked performance and diagnostic critical thinking domains: (a) clinical knowledge, (b) technical skills, (c) critical thinking, (d) communication, (e) management of responsibilities, and (f) decision making. In part, these differences are likely attributable to other documented patterns. For instance, non-White new nurse graduates tended to disproportionately attend urban-based programs (+11.5% compared to suburban and +16.2% rural), which in turn were significantly more likely to rely on remote and simulation-based learning environments. However, unlike other student outcome measures, few other criteria that displayed similar degrees of overlap with changes to course delivery format earlier, such as region, program type, etc., presented on early career metrics. Coupled with lower faculty CCEI evaluation scores and evidence regarding the barriers to online learning broadly encountered by underrepresented minorities (barriers that existed long before but were exacerbated by the pandemic) (
Disparities in remote learning faced by first-generation and underrepresented minority students during COVID-19: Insights and opportunities from a remote research experience. Journal of Microbiology & Biology.
Disproportionate impacts of COVID-19 on university students in underrepresented groups: A quantitative and qualitative descriptive study to assess needs and hear student voices.
Journal of Higher Education Management.2021; 36: 144-153
), these self-reported deficiencies are likely more nuanced and warrant further investigation by racial category.
Conclusion
The students in this study were forced to engage in learning in a manner inconsistent with their prior education and are now entering a nursing workforce that has changed dramatically due to the pandemic (
). Forthcoming results of NCSBN’s 2022 National Nursing Workforce Study estimate that nearly one-quarter of the RN workforce is now aged 34 years or younger, representing a generational shift in the nursing profession and perhaps an unprecedented loss in institutional knowledge. Furthermore, the report also confirms that nurses are experiencing heightened levels of burnout and stress due to the pandemic (Martin, Kaminski-Ozturk, O'Hara, & Smiley, 2023). When one also considers that this study and others (
) have clearly documented recent graduates’ perceived clinical deficiencies, today’s new nurses may feel they are in a more precarious position than their predecessors. Thus, an urgent demand exists for practice partners to respond to the needs of new graduate nurses and facilitate transitions to early career practice that develop a sense of salience in this new healthcare landscape.
Participants in this study learned to challenge assumptions in nursing education during the pandemic, and through those experiences, they brought new perspectives to what was important to faculty, students, and administrators in nursing education. They focused on creating order, but this frequently came at the expense of other critical aspects of the process of learning to become a nurse. Participants in our study expressed the value of hybrid modes of learning and the usefulness of debriefing; however, the paramount importance of effective implementation with those delivery systems was stressed. A great opportunity exists to enact change in nursing education. A new healthcare system with resilient approaches aimed at maintaining high levels of safety (
Continued increases in the incidence of healthcare-associated infection (HAI) during the second year of the coronavirus disease 2019 (COVID-19) pandemic.
). Gaps in nursing education were revealed during the pandemic. Moving forward, there is a need to expand and improve disaster and public health emergency education and training at a minimum (
Association of faculty perceptions of work–life with emotional exhaustion and intent to leave academic nursing: Report on a national survey of nurse faculty.
) and resulted in relatively inexperienced (median 3 years of simulation, 2 years online teaching) and unprepared (2.4% CHSEs) instructors shouldering much of the burden in guiding students through these challenging times. Now is the time to disrupt nursing educational models and expand the robust educational research needed to effectively develop the next generation of nurses. This will likely necessitate that colleges and universities bolster their faculty recruitment, training, and retention strategies. It will also require nursing programs and faculty to reevaluate their use of educational technology to facilitate experiential learning opportunities.
This important work captures both the breadth and scale of prelicensure RN programs’ early and sustained changes to their course delivery formats. It correlates programs’ ever-evolving responses to the pandemic and an array of student learning, engagement, and early outcomes. Given the scope of this work, the results not only document the many innovative ideas and strategies employed by nursing faculty and administrators who participated in our study, but they also provide a detailed evaluation of the associated results to identify best practices. In turn, this allows researchers across a range of disciplines, including, but not limited to, regulation, nursing, and education, a unique opportunity to identify and critically examine the many ways in which prelicensure RN programs around the country sought to address the nearly unparalleled challenges they confronted on a daily basis over the past 3 years. This study stands as the most comprehensive assessment of prelicensure nursing education in the United States since the onset of COVID-19. It extends knowledge by linking potential deficiencies in students’ didactic and clinical education during the pandemic and their early career preparedness and clinical competence, and in doing so illuminates the possible implications for patient safety moving forward.
APPENDIX A
A1. Initial Outreach Program Survey
Dear Colleagues:
The following survey will take 5-10 minutes to complete. All results will be reported in the aggregate and no identifying information will be disseminated or reported in any way. We very much appreciate your participation in the survey.
Sincerely,
Brendan Martin, PhD | Associate Director, Research | bmartin@ncsbn.org
National Council of State Boards of Nursing (NCSBN) | 111 E. Wacker Drive, Ste. 2900, Chicago, IL
Please indicate below which prelicensure nursing programs your school offers:
o
Traditional Bachelor of Science in Nursing (BSN)
o
Accelerated BSN (students with a non-nursing undergraduate degree)
o
Associate's Degree in Nursing (ADN)
o
Diploma
o
Other (please specify)______________
2.
Please provide your traditional BSN or ADN program enrollment information:
o
Fall 2019 enrollment:__________________
o
Estimated fall 2020 enrollment:___________
3.
Does your school use standardized exams to measure student performance?
o
Yes
o
No
3a.
[If 3. = Yes} Does your school use standardized exams to measure student performance?
o
ATI
o
HESI
o
Other (please specify)_______________
3b.
[If 3. = Yes} When do you administer them?
o
At the end of each nursing course.
o
A comprehensive exam is administered at the end of the entire nursing program.
o
Both
Clinical Course Curriculum
The following items focus on the percentage of clinical hours offered through simulation.
4.
With regard to your clinical courses, what percentage of clinical hours were completed in simulation during the fall 2019 semester and what percentage of clinical hours in simulation are anticipated during the fall 2020 semester.
During the fall 2019 semester, the percentage of clinical hours in simulation was:
During the fall 2020 semester, the percentage of clinical hours in simulation are anticipated as:
5.
With regard to your clinical courses, do you plan to offer virtual simulation instruction (e.g. computer-based simulation, virtual reality, virtual simulation, virtual reality simulation, augmented reality, etc...) during the fall 2020 semester?
o
Yes
o
No
5a.
[If 5. = Yes ] WWith regard to your clinical courses, what percentage of clinical hours were completed in virtual simulation during the fall 2019 semester and what percentage of clinical hours in virtual simulation are anticipated during the fall 2020 semester.
During the fall 2019 semester, the percentage of clinical hours in virtual simulation was:
During the fall 2020 semester, the percentage of clinical hours in virtual simulation are anticipated as:
5b.
[If 5a. % is 1 > 0] Please elaborate on which virtual tools, or instruments you will be using to evaluate students' learning outcomes:
__________________________________________
__________________________________________
__________________________________________
__________________________________________
__________________________________________
__________________________________________
Didactic course curriculum
The next questions are related to the percentage of online instruction associated with your didactic course curriculum.
6.
With regard to your didactic (lecture-style) courses, what percentage of the didactic curriculum was completed online prior during the fall 2019 semester, and what percentage is anticipated to be completed online during the fall 2020 semester.
During the fall 2019 semester, the percentage of lecture hours completed online.
During the fall 2020 semester, the anticipated percentage of lecture hours completed online.
Study Outline
We are interested in investigating the effect of increased simulation, and screen-based tools within the nursing curriculum. More specifically, we hope to build on previous studies, and compare students' outcomes at varying levels of simulation, and online education.
7.
Are you interested in potentially participating in a multi-site longitudinal survey-based study about how the Coronavirus (COVID-19) pandemic has impacted BSN/ADN program instructional format and student outcomes?
o
Yes
o
No
Survey Completion and Follow-up
Thank you for your interest in the study. Please provide the necessary contact information below to facilitate further outreach.
7a.
{If 7. = Yes} Please share the contact information of the most appropriate program personnel to discuss details of the study, including eligibility criteria and incentives for participation:
Name ____________________
E-mail ____________________
Phone ____________________
Institution __________________
City ______________________
State ______________________
7b.
{If 7. = Yes} Please indicate the start date of your Fall 2020 term below in the following format (mm/dd/yyyy):
____ / ____ / ________
8.
Do you have any questions or concerns?
______________________________________
______________________________________
______________________________________
______________________________________
A2. Study Induction Survey
Assessing the Impact of the COVID-19 Pandemic on Nursing Education: A National Study of Prelicensure RN Programs
Institutional Questionnaire
This questionnaire will be used to gather information about your (1) school or program, (2) faculty, (3) curriculum, and (4) student population. Please review each item and respond as appropriate.
If you have any questions regarding this questionnaire, please feel free to reach out to us at research@ncsbn.org. We thank you for taking the time to complete this questionnaire.
School or Program Characteristics
1.
Please describe your institution below:
Full name of program______________________
Mailing address of program__________________
City____________________________________
State___________________________________
County__________________________________
2.
What best describes the program’s geographic location?
o
Urban
o
Suburban (not rural and not within the core city boundaries)
o
Rural
o
Other (please describe)
3.
Is the nursing program nationally accredited?
o
Yes
o
No
4.
In what year (yyyy format)* was the [nursing] program founded?
* Year the nursing program started (might be different than the year the college/university was founded.)____________________
5.
Is your institution publicly or privately funded ?
o
Public
o
Private, not-for-profit
o
Private, for-profit
5a.
What is the yearly in-state tuition rate for your program?
(Please use only numbers and commas to denote tuition, e.g. 67,000 refers to $67,000)
What best describes the program’s academic schedule?
A quarter system divides the academic year into four sessions. A trimester divides the academic year into three sessions. A semester system divides the academic year into two sessions.
o
Quarters
o
Trimesters
o
Semesters
o
Other (please describe)_______________________
11.
How many clinical hours do students complete before graduation?
How many clinical rotations are completed in one term?
o
1
o
2
o
3 or more
12a.
For the fall 2020 term, please indicate when (in mm/dd/yy format):
the first rotation ends: ____________________________________________
the second rotation begins: ________________________________________
the second rotation ends: __________________________________________
the third rotation begins: ___________________________________________
the third rotation ends: ____________________________________________
Provide additional dates as needed:__________________________________
12b.
For the fall 2020 term, please indicate when (in mm/dd/yy format):
The term begins: ___ /___ /___
The term ends: ___ /___ /___
13.
When comparing the fall 2020 term, to the previous fall 2019 term, has your institution found it more difficult or easier to obtain clinical placements for your prelicensure students?
o
Much more difficult
o
Somewhat more difficult
o
Similar level of difficulty
o
Somewhat easier
o
Much easier
13a.
If you have found it more difficult to obtain clinical placements in Fall 2020 relative to the previous term, what actions have you taken (please check all that apply):
o
Delay graduation
o
Increase the number of clinical hours completed in simulation
o
Decrease the number of clinical hours a course normally has
o
Lengthen the term
o
Contact the appropriate state board of nursing (BON)
o
Other (please specify)______________________________
o
None of these
14.
Please specify the number of students to one clinical faculty member*.
(* All levels of faculty (full-time, part-time, and clinical adjunct) in all types of clinical experiences.)
If your program is transitioning to a higher proportion of online education, and/or clinicals completed in simulation or virtual simulation, have you received additional resources to support your transition?
o
Yes
o
No
o
We are maintaining similar levels of online education and clinicals completed in simulation
15a.
Please indicate which resources your program has received below: (check all that apply)
o
Additional funds
o
Formal training
o
Materials or instructional guides
o
Other (please specify)____________________________
16.
As it relates to education, what challenges do you anticipate for your program during the fall 2020 term?
This survey will assess how well your learning needs have been met throughout this course in traditional clinical, simulated clinical, and virtual clinical environments.
Please take the time to fully complete the survey. The table contains a list of learning needs and three rating sections. In each of the three sections, please circle the number corresponding to how well each learning need was met in the specified clinical environment. The choices range from “well met” (4) to “not met” (1). If the statement does not apply to any of your personal experiences, circle NA (not applicable).
Years of experience teaching online courses (total): __________________________
11.
What best describes the instructional format of the course you will be teaching?
o
In person only
o
Hybrid
o
Completely online
12.
Did you teach this course during the fall 2019 semester (please choose one):
o
Yes
o
No
13.
[If 12. = Yes] Please describe the course delivery format in fall 2019 (please choose one):
o
Entirely online
o
In person
o
Hybrid (both online and in person)
14.
Course delivery format anticipated for the fall 2020 semester (please choose one):
o
Entirely online
o
In person
o
Hybrid (both online and in person)
15.
[If 14. = Hybrid] If your course is a hybrid course, please estimate the percentage of the course that will be offered:
Online: ____%
In person: _____%
16.
[If 14. = Entirely online or hybrid] If your course has an online component, will you complete your lectures in synchronous (real-time) or asynchronous (pre-recorded) format (please choose one)?
o
Synchronous
o
Asynhronous
o
A mixture of both
17.
[If 14. = Entirely online or hybrid] If your course has an online component, how would you rate your familiarity with teaching courses online:
o
(1) Not all familiar
o
(2) Somewhat familiar
o
(3) Moderately familiar
o
(4) Extremely familiar
18.
How many courses have you taught online? ___________________________________________
19.
[If 14. = Entirely online or hybrid] If your course has an online component, has your institution provided materials or resources to assist in your transition:
o
(1) No, none at all
o
(2) Some materials
o
(3) Sufficient resources
o
(4) A great deal of resources
20.
[If 19. ≠ No, none at all] Please rate the usefulness of the materials provided:
o
(1) Not all useful
o
(2) Somewhat useful
o
(3) Moderately useful
o
(4) Extremely useful
21.
[If 13. ≠ 14.] If the delivery model for your course has changed, please discuss how you prepared for your course: __________________________________________________________________________________
22.
What challenges do you anticipate teaching this course during the fall 2020 semester?
Did the anticipated format of your course switch during instruction due to the COVID-19 pandemic (e.g., originally planned to offer in-person course, but during the semester switched to a hybrid format) (please choose one):
o
Yes
o
No
2.
[If 1. = Yes] If your course format switched, please describe your course delivery format (please choose one):
o
Mostly online
o
In person
o
Hybrid (both online and in person).
3.
[If 2. = Hybrid] If your course switched to a hybrid course, please estimate the percentage of the course that was offered:
Online: ____%
In person: _____%
4.
[If 2. = Entirely online or hybrid, OR Pre-survey 3. = Online or Hybrid] If your course maintained or switched to an online component, did you complete your lectures in either synchronous (real-time) or asynchronous (pre-recorded) formats (please choose one)?
o
Synchronous
o
Asynchronous
o
A mixture of both
5.
[If 2. = Entirely Online, Hybrid, OR Pre-survey 3. = Online, Hybrid] If your course maintained or switched to an online component estimate the percentage of lectures were completed:
Synchronously (real-time) ____%
Asynchronously (pre-recorded) ____%
In person (if applicable) ____%
6.
In terms of engagement, how would you rate your students during the fall 2020 course (engagement refers to students’ attention, curiosity, interest, and passion) (please choose one)?
o
Not at all engaged
o
Somewhat engaged
o
Generally engaged
o
Very engaged
7.
[If Pre-survey = 12. Yes] Were students in your fall 2020 course more or less engaged, relative to students enrolled in your fall 2019 course (please choose one)?
o
Much less
o
Less
o
No change
o
More
o
Much more
8.
In terms of your students’ quality of work, how would you rate your students during the fall 2020 semester (please choose one)?
o
Very low
o
Low
o
Neither good nor bad
o
Good
o
Very good
9.
[If Pre-survey 12. = Yes] Did students in your fall 2020 course produce poorer or higher quality work relative to students enrolled in your fall 2019 course?
o
Much poorer
o
Poorer
o
About the same
o
Better
o
Much better
10.
In terms of meeting learning outcomes, how would you rate your students enrolled in the fall 2020 course (please choose one)?
o
Did not meet learning outcomes
o
Partially met learning outcomes
o
Met learning outcomes
o
Exceeded learning outcomes.
11.
[If Pre-Survey 12. = Yes] Did students in your fall 2020 course meet more or fewer learning outcomes than students enrolled in your fall 2019 course?
o
Much fewer
o
Fewer
o
No change
o
More
o
Much more
12.
How would you rate your quality of instruction overall:
o
Very poor
o
Poor
o
Acceptable
o
Good
o
Very good
13.
[If Pre-survey 12. = Yes] How would you rate your quality of instruction this semester relative to the fall 2019 iteration of the course (please select):
Did the anticipated format of your course switch during instruction due to the COVID-19 pandemic (e.g. historically offered in person, but during the term switched to a simulated or virtually simulated format) (please choose one):
o
Yes
o
No
1a.
[If 1. = Yes] If your course format switched, please describe your course delivery format (please choose one):
o
Mix of clinical setting and simulated clinical scenarios
o
Mix of clinical setting and virtually simulated clinical scenarios
o
Mix of clinical setting, simulated, and virtually simulated clinical scenarios
1b.
[If 1. = Yes] If your course switched to a hybrid format, please estimate the proportion of the course that was offered:
Simulation: ____%
Virtual simulation: ______ %
Clinical setting: _____%
2.
In terms of engagement, how would you rate your students during the Fall 2020 course (engagement refers to students’ attention, curiosity, interest, and passion) (please choose one)?
o
Not at all engaged
o
Somewhat engaged
o
Generally engaged
o
Very engaged
3.
Were students in your fall 2020 course more or less engaged relative to students enrolled in your fall 2019 course (please choose one)?
o
Much less
o
Less
o
No change
o
More
o
Much more
4.
In terms of your students’ performance, how would you rate your students during the fall 2020 semester (please choose one)?
o
Very low
o
Low
o
Neither good nor bad
o
Good
o
Very good
5.
Did students in your fall 2020 course produce poorer or higher quality work relative to students enrolled in your fall 2019 course?
o
Much poorer
o
Poorer
o
About the same
o
Better
o
Much better
6.
In terms of meeting learning outcomes, how would you rate your students enrolled in the fall 2020 course (please choose one)?
o
Did not meet learning outcomes
o
Partially met learning outcomes
o
Met learning outcomes
o
Exceeded learning outcomes.
7.
Did students in your fall 2020 course meet more or fewer learning outcomes than students enrolled in your fall 2019 course?
o
Much fewer
o
Fewer
o
No change
o
More
o
Much more
8.
How would you rate your quality of instruction overall:
o
Very poor
o
Poor
o
Acceptable
o
Good
o
Very good
9.
How would you rate your quality of instruction this semester relative to prior iterations of the course (please select):
On average, how many hours do you work in a typical week?
_______ hours
8.
On average, how many hours do you work in a typical shift?
_______ hours
9.
Which of the following best describes your current work schedule? (Select one)
o
Day (7am-3pm)
o
Day {9am-Spm)
o
Day {12-hour shift) D Evening {3pm-11pm)
o
Night (11pm-7am)
o
Night (12-hour shift)
o
Rotating
o
Other:__________________________________
10.
Over the last week, what was the average number of patients each shift you were assigned to provide direct patient care?______________________________________
11.
In your opinion, over the las1 week your patient care assignments have been:
o
Not challenging enough
o
Just right
o
Too challenging or difficult
12a.
Have you worked as a charge nurse?
o
Yes
o
No
12b.
When did you start unsupervised charge nurse responsibilities?
__ / __ (mm/yy)
o
Not applicable
Orientation/Transition to Practice
A nurse residency program is a formal program of support, mentoring and orientation to the role of the professional nurse. The residency program supports the new graduate nurse as they transition from the educational program to the role of the professional nurse.
13a.
Are you in a nurse residency or transition to practice program at your facility?
o
Yes
o
No (skip to question 14)
13b.
When did your nurse residency/transition to practice program begin?
__/__ (mm/yy)
13c.
When did your nurse residency/transition to practice program end?
__/__ (mm/yy)
o
Still ongoing
Orientation is the process of introducing staff to the philosophy, goals, policies, procedures, and role expectations needed to function in a specific work setting.
14.
How long was your unit orientation? (Select one)
o
Still ongoing
o
≤ 4weeks
o
5-8 weeks
o
9-12weeks
o
13-16weeks
o
17-23 weeks
o
24+ weeks
Section 2: New Graduate Nurse Performance Survey
For the items below, rate your level of satisfaction with your proficiency in the following areas.
Tabled
1
I am satisfied with my proficiency in the following areas:
Strongly disagree (1)
Disagree (2)
Tend to disagree (3)
Tend to agree (4)
Agree (5)
Strongly agree (6)
Not applicable (NA)
Clinical Knowledge
Understanding of the principles of evidence-based practice
1
2
3
4
5
6
NA
Knowledge of pathophysiology of patient conditions
1
2
3
4
5
6
NA
Knowledge of pharmacological implications of medications
1
2
3
4
5
6
NA
Interpretation of physician and interprofessional orders
1
2
3
4
5
6
NA
Compliance with legal/regulatory issues relevant to nursing practice
1
2
3
4
5
6
NA
Understanding of quality improvement methodologies
1
2
3
4
5
6
NA
Technical Skills
Conducting patient assessments (including history, physical examination, and vital signs)
00041-8&id=gr1.jpg){kind=link}
00041-8&id=gr2.jpg){kind=link}
00041-8&id=gr3.jpg){kind=link}
00041-8&id=gr4.jpg){kind=link}
00041-8&id=gr5.jpg){kind=link}
00041-8&id=gr6.jpg){kind=link}
During the fall 2020 semester, the percentage of clinical hours in simulation are anticipated as: