This survey was conducted by the Media Insight Project, an initiative of the American Press Institute (API) and The Associated Press-NORC Center for Public Affairs Research. The survey was funded by API. Staff from API, NORC at the University of Chicago, and AP collaborated on all aspects of the study.
Data were collected using both probability and non-probability sample sources. Interviews for this survey were conducted from May 18 through June 8, 2022, with people ages 16 to 40 representing the 50 states and the District of Columbia.
The probability sample source is the AmeriSpeak® Panel, NORC’s probability-based panel designed to be representative of the U.S. household population. During the initial recruitment phase of the panel, randomly selected U.S. households were sampled with a known, nonzero probability of selection from the NORC National Sample Frame and then contacted by U.S. mail, email, telephone, and field interviewers (face-to-face). The panel provides sample coverage of approximately 97 percent of the U.S. household population. Those excluded from the sample include people with P.O. box-only addresses, some addresses not listed in the USPS Delivery Sequence File, and some newly constructed dwellings.
Adult panel members ages 18 to 40 were randomly drawn from AmeriSpeak, and 1,947 completed the survey — 1,941 via the web and 6 via telephone. Teen panel members ages 16 to 17 were drawn from AmeriSpeak Teen, and 202 completed the survey — 200 via the web and 2 via telephone. Interviews were conducted in both English and Spanish, depending on respondent preference. Respondents were offered a small monetary incentive ($3) for completing the survey. The final stage completion rate is 24 percent, the weighted household panel response rate is 24 percent, and the weighted household panel retention rate is 77.4 percent, for a cumulative response rate of 3.5 percent.
Lucid provided 3,826 non-probability interviews with people ages 16 to 40. The non-probability sample was derived based on quotas related to age, race and ethnicity, gender, and education. Interviews were conducted in English and via the web only. For panel recruitment, Lucid uses invitations of all types including email invitations, phone alerts, banners, and messaging on panel community sites to include people with a diversity of motivations to take part in research. Because non-probability panels do not start with a frame where there is known probability of selection, standard measures of sampling error and response rates cannot be calculated.
Quality assurance checks were conducted to ensure data quality. In total, 237 interviews were removed for nonresponse to at least 50% of the questions asked of them, for completing the survey in less than one-third the median interview time for the full sample, or for straight-lining all grid questions asked of them. These interviews were excluded from the data file prior to weighting.
Once the sample has been selected and fielded, and all the study data have been collected and made final, a raking process is used to adjust for any survey nonresponse in the probability sample, as well as any noncoverage or under and oversampling in both probability and non-probability samples resulting from the study specific sample design. Raking variables for both the probability and nonprobability samples included age by gender, age by Census region, age by race/ethnicity, and age by education. Population control totals for the raking variables were obtained from the 2021 Current Population Survey. The weighted data reflect the U.S. population of people ages 16 to 40.
To incorporate the nonprobability sample, NORC used TrueNorth calibration, an innovative hybrid calibration approach developed at NORC based on small area estimation methods in order to explicitly account for potential bias associated with the nonprobability sample. The purpose of TrueNorth calibration is to adjust the weights for the nonprobability sample to bring weighted distributions of the nonprobability sample in line with the population distribution for characteristics correlated with the survey variables. Such calibration adjustments help to reduce potential bias, yielding more accurate population estimates.
The weighted AmeriSpeak sample and the calibrated nonprobability sample were used to develop a small area model to support domain-level estimates, where the domains were defined by race/ethnicity, age, and gender. The dependent variables for the models were:
- Q1: In a typical day about how many hours do you spend online?
- Q24A: How concerned are you about each of the following? I have spread misinformation, even unintentionally
- Q18. Choose the statement that best describes you, even if it is not exactly right. In general, I actively seek out news and information or I mostly bump into news and information as I do other things or hear about it from others
- Q27B: As far as the people running these institutions are concerned, would you say you have a great deal of confidence, only some confidence, or hardly any confidence at all in them? Local news media
These were found to be key survey variables, in terms of model fit. The model included covariates, domain-level random effects, and sampling errors. The covariates were external data available from other national surveys such as health insurance, internet access, voting behavior, and housing type from the American Community Survey (ACS) or the Current Population Survey (CPS).
Finally, the combined AmeriSpeak and nonprobability sample weights were derived such that for the combined sample, the weighted estimate reproduced the small domain estimates (derived using the small area model) for key survey variables.
The overall margin of error for the combined sample is +/- 1.7 percentage points at the 95 percent confidence level, including the design effect. The margin of sampling error may be higher for subgroups.
Sampling error is only one of many potential sources of error and there may be other unmeasured error in this or any other survey.
Complete questions and results are available here.
Additional information on the TrueNorth approach can be found here.
Details about the Media Insight Project can be found on their site.
For more information, please email info@apnorc.org.
About the Media Insight Project
The Media Insight Project is a collaboration between the American Press Institute and The AP-NORC Center for Public Affairs Research with the objective of conducting high-quality, innovative research meant to inform the news industry and the public about various important issues facing journalism and the news business. The Media Insight Project brings together the expertise of both organizations and their respective partners, and involves collaborations among key staff at the American Press Institute, NORC at the University of Chicago, and The Associated Press.
About the American Press Institute
The American Press Institute advances an innovative and sustainable news industry by helping publishers understand and engage audiences, grow revenue, improve public-service journalism, and succeed at organizational change. We believe that for democracies to thrive, people need accurate news and information about their communities, the problems of civil society and the debates over how to solve them. That requires an economically sustainable free press that reflects the diversity of American society and understands the needs of its communities. API is a national 501(c)3 nonprofit educational organization affiliated with the News Media Alliance.
About the Associated Press-NORC Center for Public Affairs Research
Celebrating its 10th anniversary this year, The AP-NORC Center for Public Affairs Research taps into the power of social science research and the highest-quality journalism to bring key information to people across the nation and throughout the world.
- The Associated Press (AP) is an independent global news organization dedicated to factual reporting. Founded in 1846, AP today remains the most trusted source of fast, accurate, unbiased news in all formats and the essential provider of the technology and services vital to the news business. More than half the world’s population sees AP journalism every day.
- NORC at the University of Chicago is one of the oldest objective and nonpartisan research institutions in the world.
The two organizations have established The AP-NORC Center for Public Affairs Research to conduct, analyze, and distribute social science research in the public interest on newsworthy topics, and to use the power of journalism to tell the stories that research reveals. In its 10 years, The AP-NORC Center has conducted more than 250 studies exploring the critical issues facing the public, covering topics like health care, the economy, COVID-19, trust in media, and more. Learn more here.
Share with your network
You also might be interested in:
Interacting with your community and providing quality programming while providing the news may seem daunting, but it’s worth it.
Our belief in the brand and the business hasn’t wavered. When you’re a business with a mission, it becomes the only thing that matters.
Our mingles normally draw anywhere from 20 to 40 people. At a recent one, we had the mayor, a bank vice president, several retirees, a young entrepreneur and the owners of the bowling alley hanging out in our office, all chatting with each other and our newspaper staff.
