Evaluation in a Changing Educational Landscape

Two examples of programs that can help schools better respond to changing conditions.

by Jena Barchas-LichtensteinElliott BowenJoanna Laursen Brucker
May 13, 2024

What do you think of when you think of a responsive organization? Perhaps words like "nimble," "flexible," and "proactive" come to mind. If pressed further, you might say that responsive organizations have the ability to quickly adapt to changing circumstances, or can respond in real-time to emerging challenges and opportunities. To this we could add that responsive organizations enable rapid learning, encourage the open and free flow of information, and are solutions-oriented.

Regardless of how you'd answer the above question, chances are you probably wouldn't think of a large bureaucracy – for example, a school district – as an example of a responsive organization. But by partnering with other organizations, schools can improve their ability to respond to emerging needs and opportunities in real time. We've seen multiple examples of this in our own work. For example:

The Covid-19 pandemic revealed that schools can change on a dime when they need to.

Four years ago, at the onset of the COVID-19 pandemic, US schools underwent a dramatic period of rapid change. Emergency remote learning was a challenging transition for everyone involved. Schools and districts scrambled to provide the social services that had been offered on-site. Only some students had access to devices and broadband internet at home, and adult caregivers did not necessarily have the time or skills to help their children learn. But schools made it happen. Though far from perfect, a close look at the data shows that districts that maintained remote learning throughout the 2020-21 school year had comparable outcomes to those that resumed in-person instruction as soon as possible.

As researchers, we aimed to be responsive to conditions on the ground in ways that would support our partners' responsiveness in turn. In early 2020, we were just starting a four-year project whose goal was to develop computational thinking (CT) interventions for upper elementary and early middle school students. When COVID hit, we had to totally rethink our plans. Though we were still focused on CT, we first and foremost sought to understand what was happening on the ground. How was the pandemic impacting teachers' working conditions? What were the new stresses and challenges they were confronting, and how might CT help them meet these? And what did CT lack that they needed in that particular moment? Asking and answering these questions helped the curriculum developers create a program that was responsive to the moment – particularly in its flexibility for in-person, hybrid, and remote instructions. Importantly, this curriculum appears to be effective for grades 3-5.

Emergency remote learning has long since given way to longer-term change: teachers are continuing to use more technology and are better equipped to teach remotely in circumstances like bad weather.

Not all change is rapid and visible. How can educators be responsive to a changing student body?

In New York City, where our organization is headquartered, public education is undergoing yet another wave of change, this one due to changing immigration patterns. About one-third of current New Yorkers were born outside the US, a percentage that has stayed fairly consistent since 2000.

What's new is large numbers of asylum-seekers, many of them bused in from the South. In the last two years, NYC has welcomed at least 180,000 asylum seekers, more than 30,000 of whom have enrolled in public schools. Unfortunately, not all of the schools receiving these students have the programs in place to support them. The city has struggled to hire bilingual educators, and nearly 4,000 potential students over age 16 have not been enrolled in schools at all. Newcomers' placement in temporary housing can make it hard for them to find stability, and many of these new students have had major interruptions in their prior educational careers.

One of our program partners, B-SEAL (Building Secondary Educator & Administrator Leadership) for Multilingual Learners, is helping schools respond. B-SEAL supports educators and school leaders to implement cognitive strategies instruction, which has been proven particularly effective for multilingual learners. B-SEAL's ultimate goal is for schools to build bilingual and dual-language capacity among content area teachers beyond English language arts, so that students can learn language skills alongside content area skills, rather than separately. The program, which predates the rise in newcomer students, has seen growing interest from schools and educators, particularly now that the district is incentivizing teachers to receive bilingual or ENL certification.

The program's initial focus was on schools that have long had large populations of students designated as English learners. In the last two years, they have shifted to supporting schools where these populations have grown rapidly. By developing a scenario-based assessment of teaching practice, we are helping our partners understand teachers' misconceptions about cognitive strategies instruction. More broadly, as "critical friends," we are helping our partners stay true to their goals while maintaining a nuanced understanding of changing circumstances on the ground.

About this Article

The projects discussed in this article are both supported by the U.S. Department of Education (DOE). B-SEAL is funded under DOE Grant No. T365Z210122, and INFACT under award U411C190179 from the DOE's Education Innovation and Research Program (EIR) program. The opinions shared here do not necessarily reflect the views of that department, or of any other officials or agencies with the U.S. federal government.

To learn more about our work in formal educational settings, see our contributions to a CT-based project called INFACT, along with our evaluations of the award-winning game "The Logical Journey of the Zoombinis," the Green Schoolyards initiative, PBS NewsHour's Student Reporting Lab STEM curriculum, and Children's National Research Institute's "Being Me" curriculum.

Photo by Ivan Aleksic @ Unsplash

Join the Conversation
What did you think of this? How did you use it? Is there something else we should be thinking of?
Support research that has a real world impact.