If you work with youth and have ever participated in a program evaluation, chances are, you've heard (or even uttered) some version of the following statement: "if your findings don't reflect what kids themselves are feeling and thinking, then how valid are they?" Answering this question is critical. Fortunately, in recent years, a number of promising ideas for centering children's voices in evaluation have been proposed.

Embedded data collection is one way to do this. In an embedded framework, evaluation can be integrated into program delivery — so much so that evaluators can be "part of the project team that is putting an intervention in place." The benefits of this approach are substantial: in addition to eliminating the time burdens associated with post-hoc evaluation activities like surveys and interviews, embedded data collection makes evaluation directly beneficial to participants, as it supports learning outcomes.

For children who may not be able to articulate what they've learned during an interview, having an evaluation activity double as a program component is especially important. It reinforces learning by giving them a chance to put the new knowledge and skills they've acquired to use. Bringing children's voices into an evaluation through embedded data can also be beneficial to them in other ways — especially in terms of "help[ing] evaluators better understand the local context children encounter and how to better protect them."

To illustrate the value of embedded data in projects involving youth, this blog post shares three examples from our work. Our hope in sharing these is to provide a range of options for how children's voices can be part of program implementation and evaluation.

Thinking Money for Kids

In 2019, the Financial Industry Regulatory Authority Investor Education Foundation (FINRA Foundation) and the American Library Association (ALA) created a traveling exhibition aimed at increasing patrons' understandings of financial subjects like spending, saving, and budgeting. Called "Thinking Money for Kids," the exhibit ran from 2019 to 2023 — during which time 50 US public libraries hosted it to teach children and their caregivers about money.

To determine the program's impacts (click here for our formal report), we used a variety of evaluation tools, including surveys, site visits, analysis of library reports, and a series of interactive wall activities. The last of these activities (an example of which can be seen above) was especially important, as it gave children a direct opportunity to tell us what they learned from the exhibition. Incorporated directly into programming, these unique visual instruments resulted in first-hand testimonials speaking directly to impacts. They offered clear evidence of knowledge and skills acquired through participation.

The Resilient Schools Consortium (RiSC)

In 2017, the National Wildlife Federation (NWF) and several New York-based partner organizations launched a youth climate education and empowerment initiative called the Resilient Schools Consortium (RiSC). The initiative resulted in the creation and use of an interdisciplinary curriculum designed to help middle and high school students understand and combat the inequitable impacts of climate change within Coney Island-a coastal community projected to experience permanent flooding by 2080.

As evaluators, our goal was to understand the curriculum's impacts on teachers, students, and community partners. To get at the perspectives of teachers and community partners, we conducted a series of in-depth interviews year after year. These conversations often alluded to student impacts, but to really determine the extent to which RiSC made youth more knowledgeable and more motivated to participate in climate resiliency efforts, we knew we would need access to more direct evidence. To enrich our evaluation, we reviewed samples of publicly available student work — specifically, a series of short statements participants shared during an Open House event.

Displayed on paper fish, these statements gave students a chance to communicate their understanding of climate change's causes and implications. The above image offers an illustration of what students learned about climate change's impacts on Coney Island."During my time in the RiSC program," this student says, "I have learned a multitude of essential things to take note of when it comes to the sea, such as thermal expansion and glaciers melting."

Discover SCIENCE with Dr. Bear

Another project that's succeeded in capturing children's voices is Discover SCIENCE with Dr. Bear. Currently being used in hospital settings, this innovative informal STEAM curriculum originated as an out-of-school-time (OST) program for youth in the Washington DC and Baltimore metro areas. Created by Children's National Research Institute, Discover SCIENCE developed and tested a collection of innovative, hands-on lessons-all of which are designed to pique children's curiosities about the human body, promote critical thinking and scientific literacy, and generate enthusiasm for STEM-related careers.

To evaluate the program's impacts, we used a combination of surveys, focus groups, interviews, and in-person observations. We also brought in data directly created by students — most notably, observations recorded in notebooks supplied by project leaders. During a summer camp version of the program, participants used these journals to complete specific assignments. Throughout the program, they were also periodically asked to draw images of what scientists look like. Incorporating these images into our evaluation was a powerful way of tracking changes in children's STEM identities. Specifically, they enabled us to see how the program was breaking down gender- and race-based stereotypes about who does science.

Let's Put it to Work

As the above examples make clear, there are lots of ways to make evaluation listen to and learn from children. Journals, notebooks, and public interactives are all effective tools for capturing children's voices. Other examples include multimedia materials like video diaries and recordings of children listening to / discussing educational content with caregivers in real time — two other techniques we're making use of!

Regardless of the approach taken, it's clear that evaluation needs to do more than simply seek adults' perspectives on children's experiences. By conducting evaluations that reflect children's own understandings of their needs, we can affirm children's status as legitimate social actors who have a right to shape the programs that directly impact their lives.

About this Article

This blog post is based on work funded through three separate projects.

Discover SCIENCE with Dr. Bear program was supported by the Science Education Partnership Awards of the National Institutes of Health (award #1R25Od023773-05). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Thinking Money for Kids was supported by the American Library Association and the FINRA Foundation. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the American Library Association or the FINRA Foundation.

The Resilient Schools Consortium was supported by the National Oceanic and Atmospheric Administration under Grant No. NA20SEC0080005, and by numerous grants from the Federal Emergency Management Agency (EMN-2022-CA-00018 (2022-23), EMN-2021-CA-00011 (2021-22) and EMN-2020-CA-00006 (2020-21). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Oceanic and Atmospheric Administration, the Federal Emergency Management Agency, or the Department of Homeland Security.

Photo courtesy of Lisa Marie Theck @ Unsplash