Can the news help you learn statistics? In this series of studies, we’re asking people to read, watch, or listen to different versions of a news report that contains numbers, visualizations, or both. Then we’re asking them a series of questions about the credibility of that news report and some of the inferences they make from that presentation. These are the two dependent variables common to all our studies. In addition, we’re asking people to assess the relevance of the story topic to four ever-widening social scales: me, my close family and friends, people who live near me, and society as a whole. For details about Numbers in the News and the hypothetical model that underlies this research, click here.

Quantification without Numbers

Not all quantitative information is presented using precise numbers, or even using numbers at all. For example, if 58% of respondents to a poll agreed with a statement, this might be reported as “over half” or “a narrow majority,” whereas 80% might be described as “most.” Here are a few more examples:

• Quantities may be described as “small” or “very large”;
• Percentages and ratios as “few,” “many,” “most,” or “nearly all”;
• Probabilities as “unlikely,” “improbable,” and “with a high chance,”; and
• Frequencies as “often,” “rarely,” or “sometimes.”

(If you’re interested in a deep dive, semanticists have described a wide range of natural language quantifiers across dozens of languages. Check out the chapters in Keenan & Paperno, 2012 and 2017.)

While some audiences may feel more comfortable with these plain-language explanations compared to numbers, not all readers will interpret the same term in the same way. For example, research shows that laypeople consistently interpret the standardized language developed by the Intergovernmental Panel on Climate Change (IPCC) differently than its creators intended (examples here, here, and here). Yet this language was developed precisely to avoid this problem. And we see a similar phenomenon in public health communication.

The case of “large groups”

Throughout the COVID-19 pandemic, the CDC has recommended exercising caution around “large groups” of people and avoiding them, if possible. During the early stages, many areas set maximum sizes for indoor and outdoor gatherings, either as specific numbers or as percentages of a building’s pre-pandemic capacity. However, there remains no official definition of a “large group” for COVID preventative measures, and the CDC has updated their definitions several times. Indeed, it would likely be impossible to give a one-size-fits-all answer, as the probability of COVID transmission in any space depends on ventilation, masking, distancing, and how cautious the people present are in general (affecting the probability that someone attending would be infected).

But the absence of numerical cut-offs may cause misunderstandings. Let’s say you and I are both avoiding large groups—but you think more than six people is a large group, whereas I think it’s more than 50. We would both say we’re taking this precaution, although we’re doing completely different things. So we asked the following questions:

• How do people interpret the phrase “large groups”?
• Do they think others around them share this interpretation?

What we did

As part of our study on social desirability bias, we showed participants a news story from March 29, 2022 including poll data on the extent to which Americans reported taking multiple COVID safety measures. The news text includes information about the percentage of survey respondents who say they are avoiding “nonessential travel” and “large groups” at different points in time. After participants read the story, we asked them how they would define a “large group” and how they thought the average American (reflected in the AP-NORC poll) would define the same term.

Individuals’ definitions of a “large group” varied widely

The median respondent said that a group of 20 or more people was “large,” and half of our respondents gave answers between 10 and 50 (inclusive). But 55 people (18%) offered numbers below 10, and 46 people (15%) said a large group had to be at least 100 people or larger. In fact, fifteen of those people gave limits of 1,000 or higher (Table 1).

Nearly all of our respondents provided round numbers: more than three-quarters of responses provided approximate numbers, with only one significant digit.

Table 1. Responses to How many people do you consider to be a large group?

Note. We did not require answers to be numeric, in case people wanted to give different numbers for indoor and outdoor groups or give a non-exact answer. However, most respondents did give a numeric answer.

Most people thought they were more cautious than average

Meanwhile, the median respondent said the average American defined a group of 30 or larger as “large.”

Most respondents (62%, or 196 people) gave a higher estimate for what the “average American” thought than their own definition of a “large” group. Another 63 people (20%) said the average American shared their interpretation (Table 2).

Table 2. Responses to How many people do you think the average American considers to be a large group? compared to How many people do you consider to be a large group?

As this table shows, only the people with the largest estimates (above 1,000) saw themselves as less cautious than the average American.

Recommendations

Journalists can’t solve ambiguities in poll questions or institutional communications, but they can help their audiences in two ways:

• Spell out the ambiguity.
  For example, in the context of “large groups,” point out that there is no accepted definition of large. Acknowledge that people may interpret this word very differently, particularly because it depends on a number of contextual factors.
• When numerical values are available, use both words and numbers.
  In the IPCC context, researchers consistently found that using both modes of communication helped people have the most consistent understanding.

Funding

These materials were produced for Meaningful Math, a research project funded through National Science Foundation Award #DRL-1906802. The authors are solely responsible for the content on this page.

Photo by Jon Tyson on Unsplash