Behind the Research articles provide a view of the twists and turns that happen in research projects. When possible, we explain the steps in our thinking to make the social science research process more transparent. This is the second Behind the Research story about the project called A New History of Investment in Financial Education across the United States.

Social science research often involves quirky data. By “quirky,” we mean data that is inconsistent, incomplete, or idiosyncratic in some way. In our project, A New History of Investment in Financial Education across the United States, part of our goal was to build and study a database of states’ spending on financial education and outcomes of that spending. We encountered a lot of quirky data along the way and had to make decisions that would both help fill gaps and enable us to effectively use the data. This article builds on another Behind the Research article that describes the big-picture approach to this project.

One of the things we did early on in the research process was try to capture states’ education requirements from year to year. We began our search for legislation and regulations using LexisNexis, a provider of legal research information. But this frequently left us with gaps in a state’s timeline. At that point, we had to turn directly to state legislature websites. While not an insurmountable challenge, this meant navigating states’ different ways of organizing information, including when one statute or regulation superseded another.

We also saw ambiguity among states’ academic content standards that required us to make decisions in order to have precise hypotheses. One way in which this issue presented itself was the difference between the time at which content standards for a financial education course (or subject) were adopted, and when they were implemented. Sometimes this was explicit and sometimes it was not. Another facet of ambiguity was that some states might share standards that loosely guide an instructor, while others had detailed substandards. As a result, we had to consider whether we could make direct comparisons between financial education requirements between states. For example, would the value of a course that covers 4 out of 4 standards devoted to financial education for one state be equivalent to a course addressing 72 out of 72 standards devoted to financial education in another state?

We also had to make some adjustments to account for states that offer students a choice of several financial education courses rather than a single, specific course. Another consideration was whether or not students had taken the financial education courses offered by their states. Using other research that focused on individual school districts, we came up with a way to calculate the probability that students in the state took the classes that were offered in cases where they were not required.

We then turned to non-profits’ investments in financial education, where we focused primarily on the Council for Economic Education, a national leader in economic and financial education that supports affiliate councils in most states. Another challenge here was that some state-level councils had incomplete records available online. Since the Council for Economic Education has been around since the 1960s, we hadn’t expected this when looking at the time period that was the focus for the study, which was from around 1995 to 2020. In the end, we were able to work with the national headquarters of CEE to make this data more complete.

On the other hand, one of our other primary data sources -- the Federal Reserve’s Survey of Household Economics and Decisionmaking (SHED) -- is largely consistent in that it provides data from residents of all 50 states from one year to the next. But there are some variations in the questions and response options which made it challenging at times to know which questions were designed to collect the same information each year. To help with this, we devised a coding scheme, a “crosswalk” that allowed us to ensure that the same questions are matched across years. Further, for the SHED questions that we ended up focusing on most in depth, we also needed to make a similar scheme to account for the fact that when a respondent answered “A” in one year, this did not always match up with the “A” response option in all of the other years. The coding scheme we developed is free to use and can be accessed here.

The gaps that we encountered motivated us to build a stronger database of financial education investments. However, we think there’s still more to do. There is data that might have helped us answer our research questions, but it doesn’t yet exist in one place. The Jump$tart Coalition for Personal Financial Literacy supports a network of 51 affiliated state-level coalitions that advance youth financial literacy. Jump$tart’s state coalitions include a large fraction of financial education organizations with some data that could be usefully centralized. It would be useful to researchers and coalition members to understand the overall level of direct financial education spending in their state by all coalition members, the types of spending involved, and the effects of each type of spending.

About the Project

From 2019 to 2020, Knology led A New History of Investment in Financial Education across the United States, a research initiative funded by the National Endowment for Financial Education® (NEFE) and supported by a range of experts. Knology built a robust database of historical spending on financial education across all 50 states, as well as outcomes of that spending. The team then used the database to study how investments in financial education through public schools and non-profits contributed to indicators of financial health for U.S. residents, such as retirement savings and financial fragility. Learn more about the project. If you have questions about this project, contact us at info@knology.org.

The National Endowment for Financial Education is the independent, centralizing voice providing leadership, research, and collaboration to advance financial well-being. Find out more about NEFE at NEFE.org.

Photo credit: Lucas Gallone on Unsplash.