AmericaThinks Polling 101: A Big Step Forward
My goal for this 101 series is to present easily understood overviews of public opinion polling in order to show how AmericaThinks is re- imagining polling by combining the best of the past and the best of the future.
This series discusses:
Polling 101: How it works
Polling 101: Polling in Crisis
Polling 101: Literary Digest Revisited
AmericaThinks Polling 101: A Big Step Forward
AmericaThinks Polling 101: Asking about Issues, not predicting Elections
AmericaThinks Polling 101: Majority Opinions and Minority Rights
AmericaThinks Polling 101: A Big Step Forward
Other blogs have presented an overview focused on the big picture of traditional public opinion polling as well as some of the issues and crisis points that face modern polling.
AmericaThinks takes a big step forward in solving these issues and making public opinion polling reliable, free, and available to all registered voters.
This overview will focus on the big picture of AmericaThinks public opinion polling and how we plan to conduct these polls.
Issues Polling
Public opinion polls can be divided into three broad categories: election polls, issue polls, and marketing polls. AmericaThinks is exclusively focused on issue polls. Issue polls ask about the opinions and experiences of people to discover what people know about an issue, how it affects their lives, the importance of the issue, and what they think should be done. AmericaThinks does not concern itself with polls that attempt to predict the outcome of elections or collect marketing feedback.
Polling Methods
Traditional polling methodologies assume that it is not feasible to ask the opinion of everyone in a given population. This assumption is no longer valid. With the proliferation of smartphones, it is now possible for almost every registered voter to participate in election-grade public opinion polling on a daily basis.
An AmericaThinks poll can include all registered voters – 170 million participants – more quickly and with less expense than a typical 1,500-person poll. Each AmericaThinks poll is not just one poll but is actually 435 separate polls – one poll for every congressional district. Results are reported individually for each congressional district, each state, and the country as a whole.
How is that possible? Smartphones are the key. Data from the 2020 elections show that out of 252 million voting-age people, 168 million are registered voters. In 2021, 85% of voting-age American adults reported that they owned a smartphone. That means over 214 million voting-age people own a smartphone. We have more smartphones than registered voters in America!
In only a few short years, almost everyone of voting age will have a smartphone. The percentage of voting age adults that own a smartphone has grown 4% from 2019 to 2021. Very soon, every adult of voting age will have a smartphone capable of using AmericaThinks.
AmericaThinks provides election-grade public opinion polling free to all registered voters. This is a revolutionary advancement in public opinion polling, but will it create more accurate results?
AmericaThinks Polls
The goal of AmericaThinks is that every registered voter has a voice. This is a lofty and non-trivial goal. Before such a goal can be taken seriously, important questions need to be answered.
What percentage of the registered voters will choose to participate in AmericaThinks?
How long will take before most registered voters join?
How many participants will answer each AmericaThinks poll?
How will AmericaThinks demonstrate that a poll is comprised of a representative sample?
How will AmericaThinks accomplish reliable polling from the beginning when only a small portion of registered voters participate?
To answer these and other questions, AmericaThinks is adopting a hybrid solution that encompasses the best of the past and the best of the future.
AmericaThinks participants become part of a polling panel for which the results are reported in at least two ways. First, results are tallied using stratified random sampling of responses based on standard characteristic benchmarks for each district, state, and nationwide. Secondly, the results are tallied based on all responses and reported along with the associated characteristic participation data for each district, state, and nationwide.
As we go forward in time, the results of these methods will merge as the size and diversity of the participation grows. Comparisons will teach us a great deal about participation, reliability, fairness, and accuracy of these methods.
This methodology gives us the best of two approaches: we can publish large probability-based polls that reflect the opinions of the majority while publishing every opinion to ensure that the minority opinions are heard.
AmericaThinks makes a significant contribution to the state of the art by providing public opinion methodology that reduces the subjective influence in a poll. While the skill of creating good questions is subjective and the selection of demographic benchmarks is somewhat subjective, the remainder of the methodology is remarkably free from subjective judgements. This methodology will likely not require the subjective selection of samples, subjective administration by interviewers, subjective weighting of responses, or the subjective interpretation of the results.
The net result is the creation of a more objective representative of public opinion with less potential for bias. The practice of AmericaThinks polling follows a basic outline.
Create the questions to be asked.
Ask every registered voter to respond to the questions.
Use stratified random sampling to select a representative sample of responses.
Tally the results for a) all responses and b) responses selected by stratified random sampling.
Audit and publish full poll data and comparatively report the results.
First, create the questions to be asked. Questions are simple, right? Not so much. Misunderstandings are very common in our communication with each other. So, it is important that poll questions are as simple as possible, clear, and understandable by people from a wide variety of backgrounds. A previous blog discusses how we can ask good questions and stay away from asking bad questions.
Second, ask every registered voter to respond to the questions. All registered voters are encouraged to participate for free in every poll using their smartphone. AmericaThinks participation will grow from zero to hopefully, all registered voters in the years to come. We will work with local, regional, and national organizations to recruit registered voters, regardless of party or affiliation. We will work to maintain voter registration status to ensure that all participants are currently registered voters. Our motto could literally be, “No voter left behind.” Responses from all participants are cryptographically anonymous. The identity of the participant is removed from their response before their response leaves their smartphone. As participants come to embrace this confidentiality, response bias related to transparency, social pressures, or accountability concerns should be eliminated. Polls may be answered at the convenience of the participants, thus eliminating a source of non-response bias.
Third, use stratified random sampling to select a representative sample of responses. The benchmark for stratified random sampling is based on the demographics of each district polled. Once that a minimum number of responses are received in a district, responses within a target demographic characteristic are randomly selected from all responses to match the benchmark demographics. The result is a probabilistic representative sample of the population without the potential of weighting bias. Critics may content that the sample is not representative because it is randomly selected from an opt-in panel. However, other panel-based opinion polls are successfully using various implementations of stratified random sampling with notable success. Upcoming blogs will discuss this approach in more detail.
Fourth, tally the responses. Tallying simply translates a response into a measured outcome. A well-designed poll will easily correlate responses with measured outcomes. Responses are tallied using both the sampled and full set of responses. Responses are tallied for each district, state, and nationwide along with participating characteristics. Results may be reported in real-time as responses are received.
Fifth, audit and comparatively report the results. Each set of results are subject to a full internal audit. Since data is cryptographically secured in a chain and/or by digital signatures, an audit of the data is objective and repeatable. An audit report that includes detailed findings will be published along with the full data set for third party auditing. Results are tallied using stratified random sampling of responses based on standard characteristic benchmarks for each district, state, and nationwide. Results are also tallied based on all responses and the associated characteristic participation data for each district, state, and nationwide. The consumer may compare the results obtained by these two methods and draw their own conclusions.
Sample Size and Accuracy
The casual observer may note that AmericaThinks polls will contain millions of participants instead of the hundreds of participants typically found in current scientific polls. Conventional wisdom is tempted to remark that larger sample populations do not improve the accuracy of the poll. On the contrary, it is commonly understood that larger sample populations produce more accurate results as long as a representative sample is used for the poll.
In the past, notable polling failures have occurred with larger sample populations. For example, the failure of the Literary Digest presidential poll of 1936 is legendary. The Literary Digest poll will be extensively discussed in an upcoming blog post. To summarize, the Literary Digest poll did not fail because it was a large poll, but because it contained a non-representative sample, most likely due to non-response bias.
AmericaThinks has the core value of creating accurate polls with very large sample populations in order to support representative democracy. This core value will be unpacked in upcoming blog posts.
The summary is this: in America, every registered voter has a responsibility to speak into their self-government at election time. AmericaThinks believes that every registered voter also has a responsibility to speak into their self-government between elections.