Aggregative Contingent Estimation Program Explained
Aggregative Contingent Estimation (ACE) was a program of the Office of Incisive Analysis (OIA) at the Intelligence Advanced Research Projects Activity (IARPA).[1] [2] The program ran from June 2010 until June 2015.[3]
History
The broad program announcement for ACE was published on June 30, 2010.[4] ACE funded the Aggregative Contingent Estimation System (ACES) website and interface on July 15, 2011.[5] They funded The Good Judgment Project some time around July 2011.[6] ACE has been covered in The Washington Post' and Wired Magazine.[7] The program was concluded by late 2015.[8] The program manager was future IARPA director Jason Gaverick Matheny.[9]
Goals and methods
The official website says that the goals of ACE are "to dramatically enhance the accuracy, precision, and timeliness of intelligence forecasts for a broad range of event types, through the development of advanced techniques that elicit, weight, and combine the judgments of many intelligence analysts."[1] The website claims that ACE seeks technical innovations in the following areas:[1]
- efficient elicitation of probabilistic judgments, including conditional probabilities for contingent events
- mathematical aggregation of judgments by many individuals, based on factors that may include: past performance, expertise, cognitive style, metaknowledge, and other attributes predictive of accuracy
- effective representation of aggregated probabilistic forecasts and their distributions.
There is a fair amount of research funded by grants made by the IARPA ACE program.[10]
Partners
The ACE has collaborated with partners who compete in its forecasting tournaments. Their most notable partner is The Good Judgment Project from Philip E. Tetlock et al.[11] (winner of a 2013 ACE tournament)[12] ACE also partnered with the ARA to create the Aggregative Contingent Estimation System (ACES).[5]
Data from ACE is fed into another program, called Forecasting Science and Technology (ForeST), which partners with SciCast from George Mason University.[13]
Notes and References
- Web site: Aggregative Contingent Estimation (ACE). Matheny. Jason. Rieber. Steve. May 6, 2014. Intelligence Advanced Research Projects Activity.
- Web site: Aggregative Contingent Estimation. Office of the Director of National Intelligence, United States. May 6, 2014.
- Web site: IARPA's New Director Wants You to Surprise Him. Harbert. Tam. 2015-10-19. IEEE Spectrum. 2016-03-31.
- Web site: Aggregative Contingent Estimation System. June 30, 2010. May 6, 2014. Federal Business Opportunities.
- Web site: Intell site tests crowdsourcing's ability to predict future. Hickey. Kathleen. July 15, 2011. May 6, 2014. GCN.
- Web site: The idea behind the Good Judgment Project . July 27, 2011 . May 5, 2014 . The Good Judgment Project . dead . https://web.archive.org/web/20140506202243/http://goodjudgmentproject.com/blog/?p=4 . May 6, 2014 .
- Web site: Can Algorithms Find the Best Intelligence Analysts?. Drummond. Katie. April 22, 2010. May 6, 2014. Wired Magazine.
- Web site: IARPA's high-stakes intelligence experiment. Corrin. Amber. 2015-09-23. C4ISR & Networks. 2016-03-31. 2017-06-21. https://web.archive.org/web/20170621170049/http://www.c4isrnet.com/story/military-tech/it/2015/09/23/iarpa-anticipating-surprise/72632204/. dead.
- Book: Marc Prensky. Brain Gain: Technology and the Quest for Digital Wisdom. 7 August 2012. St. Martin's Press. 978-1-137-09317-2. 260 . The ACE program manager is Jason Matheny.
- Web site: Google Scholar listing of research funded by IARPA ACE. May 6, 2014.
- Web site: The Project . The Good Judgment Project . May 5, 2014 . dead . https://web.archive.org/web/20140506201121/http://www.goodjudgmentproject.com/about_project.html . May 6, 2014 .
- Web site: Good judgment in forecasting international affairs (and an invitation for season 3). Horowitz. Michael. November 26, 2013. May 5, 2014. The Washington Post.
- Web site: Forecasting Science & Technology (ForeST). Matheny. Jason. May 6, 2014. Intelligence Advanced Research Projects Activity.