PollyVote

The PollyVote project uses the high-profile application of predicting U.S. presidential election results to demonstrate advances in forecasting research. The project is run by political science professors and forecasting experts, one of which is J. Scott Armstrong. All procedures, data, and results are fully disclosed and freely available online.

The project started in March 2004 to demonstrate the benefits of combining forecasts. In averaging forecasts within and across different forecasting methods, the combined PollyVote forecast provided highly accurate predictions of the two-party popular vote shares for the last three U.S. presidential elections.^[1][2]

History

The PollyVote was created in March 2004 by marketing and forecasting expert J. Scott Armstrong and political science professors Alfred Cuzán and Randall Jones.^[3] The goal at that time was to apply the combination principle in forecasting to predict President Bush's share of the two-party popular vote (omitting minor candidates) in the 2004 presidential election. Until Election Day in November of the same year, the researchers collected data from 268 polls, 10 quantitative models, and 246 daily market prices from the Iowa Electronic Markets vote-share market. In each of the last three months prior to the election, they also administered a survey with a panel of 17 experts on US politics, asking them for their predictions. The forecasts were first combined within each component method by averaging recent polls, the IEM prediction market forecasts from the previous week, and averaging the predictions of the quantitative models. Then, the researchers averaged the forecasts across the four-component methods. The resulting forecast was named the PollyVote. From March to November, the forecasts were initially updated weekly, and then, twice a week. The forecasts were published at the Political Forecasting Special Interest Group at forprin.com.

In 2007, Andreas Graefe joined the PollyVote team and helped to launch the PollyVote.com website prior to the 2008 U.S. presidential election. For predicting the 2008 election, the general structure of the PollyVote remained unchanged; the PollyVote combined forecasts within and across the same four-component methods as in 2004. However, some changes were made at the level of the component methods. Instead of averaging recent polls, the PollyVote team used the RCP poll average by RealClearPolitics as the polls component. In addition, the advantage of the leading candidate was discounted (or damped) using the approach suggested by Jim Campbell. The first PollyVote forecast for the 2008 election was published in August 2007, 14 months prior to Election Day, and was updated daily.^[4]

For forecasting the 2012 election, a fifth component called "index models" was added to the PollyVote. This component captured information from quantitative models that use a different method and rely on different information than the traditional political economy models. In particular, the index models capture information about the campaign, such as the candidates' perceived issue-handling competence,^[5][6] their leadership skills,^[7] their biographies^[8] or the influence of other factors such as whether the incumbent government faced some scandal.^[9] The first forecast for the 2012 election was published on January 1, 2011, almost two years prior to Election Day. As in 2008, the forecasts were updated daily, or whenever new information became available.^[1]

In 2013, the PollyVote was launched in Germany to predict the German federal election of the same year.^[10]

For the presidential election in 2016, PollyVote augmented their reporting with computational writing for the campaign coverage campaign coverage, publishing articles for all of their (intermediate) predictions on their Blog.

Method

The PollyVote demonstrates the benefits of combining forecasts by averaging predictions within and across several component methods. In its application for the U.S. presidential election, the PollyVote is currently based on five component methods: polls, prediction markets, expert judgment, political economy models, and index models. The PollyVote predicts the share of the popular two-party vote achieved by the candidate of the incumbent party.

Accuracy of the PollyVote

The PollyVote published forecasts prior to each of the three U.S. presidential elections, the 2006 U.S. House of Representatives election, and the 2013 German federal election. In addition, one analysis tested how the PollyVote would have performed for the three elections from 1992 to 2000. As expected, the application of the forecasting principles has led to accurate forecasts. Surprisingly, however, across the three U.S. presidential elections, the forecast error was always lower than the error of each component methods.^[2] Comparisons have also been made with other methods. For example, forecasts of the 2012 election were also substantially more accurate than the closely watched forecasts from Nate Silver's model at FiveThirtyEight.com.^[1]

2004 U.S. presidential election

The 2004 PollyVote was launched in March 2004 and forecast a victory for President Bush over the 8 months that it was making forecasts. The final forecast published on the morning of the election predicted that President would receive 51.5% of the popular two-party vote, an error of 0.3 percentage points.^[3]

2008 U.S. presidential election

The 2008 PollyVote was launched in August 2007 and forecast a victory for Barack Obama over the 14 months that it was making daily forecasts. On Election Eve, it predicted that Obama would receive 53.0% of the popular two-party vote, an error of 0.7 percentage points.^[4]

2012 U.S. presidential election

The 2012 PollyVote was launched in January 2011 and forecast a victory for President Obama over the 22 months that it was making daily forecasts. On Election Eve, it predicted that Obama would receive 51.0% of the popular two-party vote, an error of 0.9 percentage points.^[1]

2016 U.S. presidential election

The 2016 PollyVote predicted that Hillary Clinton would win both popular vote and electoral vote. "Clinton will win the popular vote by 5.0 percentage points in the two-party vote (52.5% vs. 47.5%). Clinton’s chance to win the popular vote is above 99%. In terms of the Electoral College, Polly predicted Clinton to receive 323 electoral votes compared to 215 for Trump." They were right that Hillary Clinton would win the popular vote but failed to accurately predict the Electoral College winner. They were off by 4.2 percentage points for Hillary's popular vote percentage.

2020 U.S. presidential election

The 2020 PollyVote predicted that Joe Biden would win both the popular vote and the electoral college. "According to the PollyVote, it is virtually certain that Joe Biden will win the most votes. The final forecast is that Biden will win 52.2 percent of the popular two-party vote, which leaves 47.8 percent for President Trump." Polly predicted Biden to receive 329 electoral votes compared to 209 for Trump. They managed to predict the popular vote winner and the Electoral College winner. Joe Biden won 51.3 percent of the popular vote meaning PollyVotes error was just 0.9 percent. Although they did not get the electoral count right, they still managed to predict the winner correctly. They managed to predict the electoral count much closer than statistical models like Nate Silver's fivethirtyeight.

2006 US House of Representatives election

PollyVote predicted the outcome of the 2006 U.S. House of Representatives Elections, forecasting that the Republicans would lose 23 seats, and thus, their majority in the House. The Republicans lost 30 seats and the House majority in those elections.

Perception

The results of the PollyVote project are regularly published in the academic community. Prior to the past elections, forecasts were published in Foresight and the New Scientist.^[11] Analyses of the accuracy of the PollyVote were published in the International Journal of Forecasting and PS: Political Science & Politics. In addition, scholars have referenced the PollyVote as a benchmark when assessing the validity of U.S. presidential election forecasts.^[12][13][14]

To date, the PollyVote predictions have been rarely cited in the popular press. In their IJF paper, the PollyVote team discusses several reasons why this might be the case: (1) people have difficulties understanding the benefits of combining, (2) people wrongly believe that they can identify the best forecast, and (3) people think that the method of calculating averages is too simple.^[2] Another possible reason is that the PollyVote predictions are very stable and rarely change, whereas election observers and journalists are interested in excitement and newsworthiness.

References

1. Graefe, Andreas; Armstrong, J. Scott; Jones, Randall J. Jr; Alfred G. Cuzán (2014). "Accuracy of combined forecasts for the 2012 Presidential Elections: The PollyVote". PS: Political Science & Politics. 47 (2): 427–431. doi:10.1017/S1049096514000341. S2CID 53974907.
2. Graefe, Andreas; Armstrong, J. Scott; Jones, Randall J. Jr; Alfred G. Cuzán (January–March 2014). "Combining forecasts: An application to elections". International Journal of Forecasting. 30 (1): 43–54. doi:10.1016/j.ijforecast.2013.02.005. S2CID 153677755.
3. Cuzán, Alfred G; Armstrong, J. Scott; Jones, Randall J. Jr. (2005). "How we computed the PollyVote" (PDF). Foresight: The International Journal of Applied Forecasting. 1 (1): 51–52.
4. Graefe, Andreas; Armstrong, J. Scott; Cuzán, Alfred G.; Jones, Randall J. Jr. (2009). "Combined Forecasts of the 2008 Election: The PollyVote". Foresight: The International Journal of Applied Forecasting (12): 41–42.
5. Graefe, Andreas; Armstrong, J. Scott (2013). "Forecasting Elections from Voters' Perceptions of Candidates' Ability to Handle Issues". Journal of Behavioral Decision Making. 26 (3): 295–303. doi:10.1002/bdm.1764.
6. Graefe, Andreas; Armstrong, J. Scott (2012). "Predicting elections from the most important issue: A test of the take-the-best heuristic". Journal of Behavioral Decision Making. 25 (1): 41–48. doi:10.1002/bdm.710.
7. Graefe, Andreas (December 2013). "Issue and leader voting in U.S. presidential elections". Electoral Studies. 32 (4): 644–657. doi:10.1016/j.electstud.2013.04.003. S2CID 154094231.
8. Armstrong, J. Scott; Graefe, Andreas (2011). "Predicting elections from biographical information about candidates: A test of the index method". Journal of Business Research. 64 (7): 699–706. doi:10.1016/j.jbusres.2010.08.005. S2CID 18855501.
9. Lichtman, Allan J. (2008). "The keys to the white house: An index forecast for 2008". International Journal of Forecasting. 24 (2): 301–309. doi:10.1016/j.ijforecast.2008.02.004.
10. Graefe, Andreas (January 2015). "German election forecasting: Comparing and combining Methods for 2013". German Politics. doi:10.2139/ssrn.2540845. S2CID 154898822. SSRN 2540845.
11. Giles, Jim (22 October 2008). "And the next president of the United States will be…". New Scientist. 200 (2679): 12–13. doi:10.1016/S0262-4079(08)62672-X.
12. Sunstein, Cass R. (2006). Infotopia: How Many Minds Produce Knowledge. New York: Oxford University Press. p. 40. ISBN 978-0-19-534067-9.
13. Lewis-Beck, Michael S.; Tien, Charles (October 2008). "The Job of President and the Jobs Model Forecast: Obama for '08?". PS: Political Science & Politics. 41 (4): 687–690. doi:10.1017/S1049096508081262.
14. Holbrook, Thomas M. (2010). "Forecasting US presidential elections". In Leighley, Jan E. (ed.). The Oxford Handbook of American Elections and Political Behavior. Oxford: Oxford University Press. pp. 346–371.

External links

• Official Site