The COVID-19 impact on reading achievement growth of Grade 3–5 students in a U.S. urban school district: variation across student characteristics and instructional modalities

The COVID-19 pandemic significantly disrupted education, leading to widespread concerns about learning loss. While national data exists, this study focuses on a single urban school district in North Carolina to understand the impact on reading achievement, particularly for vulnerable student groups and across different instructional modalities. The study aims to compare reading gains during the 2020-2021 school year with the 2018-2019 pre-pandemic year and explore how instructional modality influenced growth, especially for students from diverse backgrounds like low socioeconomic status (SES), English learners, and students with disabilities. Research consistently shows that extended breaks from schooling, like summer, lead to learning loss, and the pandemic's school closures exacerbated this. Early elementary students, students from low-income backgrounds, English learners, and students with disabilities are often disproportionately affected. Prior research on remote learning found a negative association with academic achievement, suggesting in-person schooling's importance. The 'faucet theory' proposes that schools equalize learning opportunities, which are turned off during closures, widening achievement gaps. This study hypothesizes that in-person schooling during the pandemic might have served as an equalizer for lower-achieving students.

The study utilized administrative data from 180 elementary schools in an urban North Carolina school district, covering the 2018-2019 (pre-COVID-19) and 2020-2021 (COVID-19) school years. The analytic sample included 52,525 Grade 3-5 students. Reading achievement was measured using the NWEA MAP Growth Reading assessment, a computer-adaptive test providing Rasch unit (RIT) scores that allow for longitudinal comparisons. Demographic characteristics like SES (low, medium, high based on census tract), language status (English learner vs. English-fluent), and disability status (students with disabilities vs. without) were obtained from district records. Instructional modality was defined as either in-person or fully remote, options offered to students in summer 2020. In-person students attended school physically for limited days in both fall and spring semesters, while remote students received virtual instruction exclusively. Data analysis for research question 1 (reading gains and variability) involved calculating 9-month MAP reading gain scores and standardizing differences. For research question 2 (instructional modality and reading growth), piecewise growth curve models were employed due to the non-linear pattern of reading progression across three assessment points. These models examined the main effects of instructional modality and demographics, as well as their interaction effects on reading levels at the beginning of the school year and growth rates over fall and spring semesters.

The study found that Grade 3-5 students in the 2020-2021 (COVID-19) cohort showed lower average 9-month reading gains compared to the 2018-2019 (pre-COVID-19) cohort. The learning loss effect size was 0.54, 0.27, and 0.28 standard deviation units for Grade 3, 4, and 5, respectively. Grade 3 students experienced the most significant reduction in gains, achieving only 48% of the pre-pandemic gains, while Grade 4 and 5 achieved 65% and 58%. Variability in reading gains also substantially increased in the COVID-19 cohort, particularly for Grade 3 students (56% increase in standard deviation). The negative impact of the pandemic was profoundly felt by students from high-poverty backgrounds, English learners, and students with disabilities. For instance, Grade 3 low-SES students made only 40% of typical gains, significantly less than high-SES students (61%). English learners in Grade 3 and 4 also demonstrated lower relative gains than their English-fluent peers. Students with disabilities experienced the lowest gains, with Grade 3 and 4 students achieving only 18% and 28% of pre-pandemic reading gains, respectively, and exhibiting stark increases in variability.

Students who opted for remote instruction started the 2020-2021 school year with higher MAP reading scores than those who chose in-person instruction across all grades. However, while fall semester reading growth rates were not significantly different between the two groups, significant variations emerged in the spring semester. Students receiving in-person instruction demonstrated stronger growth rates than their remote counterparts. Notably, for Grade 3, English learners and students with disabilities in remote instruction showed a decrease in reading growth during the spring, while those in person maintained steady or slight growth. For Grade 4, low-SES students in remote instruction started higher but made slower progress, with in-person low-SES students catching up by spring. English-fluent remote learners also experienced a slowdown. In Grade 5, remote low-SES students showed minimal gains during the fall, leading to the disappearance of their initial achievement gap with in-person low-SES students. Overall, in-person schooling appeared to act as an equalizer, helping lower-achieving groups catch up to higher-achieving groups over the school year, especially during the spring semester when in-person attendance increased. An exception was students with disabilities, whose reading growth stagnated or declined even with in-person instruction, possibly due to limited special education services.

The study confirms significant COVID-19-related learning losses in reading, particularly for younger students and vulnerable populations like those from low-SES backgrounds, English learners, and students with disabilities. The magnitude of loss, especially for Grade 3 students, suggests long-term consequences for foundational literacy skills. The increased variability in scores further exacerbates pre-existing achievement gaps. While initially higher-achieving students opted for remote learning, the study found that in-person schooling, particularly in the spring semester, supported continuous reading growth for many students, helping to narrow achievement gaps between instructional modalities. This supports the 'faucet theory' where schools play an equalizing role. However, students with disabilities did not show the same positive response to in-person instruction, possibly due to inadequate special education services during the pandemic. Limitations include the descriptive nature of comparisons, which prevents causal claims regarding the COVID-19 impact. The study also lacked detailed contextual information on in-person and remote instructional settings, which could provide deeper insights into the mechanisms of observed outcomes. Future research should prioritize causal intervention studies comparing in-person and remote instruction, coupled with qualitative data on instructional practices and student interactions, to better understand and address diverse learning needs.