Metaculus: a prediction website with an eye on science and technology
“Metaculus” things happen when you gather a wealth of well-reasoned scientific opinion and point it at the future.
That’s the premise and the promise of Metaculus.com, a prediction website that rates the likelihood of various scientific and technological outcomes. Yale astrophysicist Greg Laughlin launched the site a year ago with physicist Anthony Aguirre of the University of California-Santa Cruz and their former postdoc Max Wainwright.
“The whole idea of prediction markets, that groups of people with different perspectives can make more accurate predictions than the ‘experts,’ is almost like magic,” Laughlin said. “We wanted to harness some of that magic.”
Online prediction markets began cropping up in the late 1980s, focusing on everything from movie box office receipts to the outcome of presidential elections. They have included the Iowa Electronic Markets, PredictIt, Betfair, and Smarkets. Researchers have speculated that prediction markets are more successful when they include a diversity of informed opinions.
Yet such websites are not infallible. Perhaps the most famous recent example of this was the Brexit vote in Great Britain. Prediction markets said voters would opt to remain in the European Union; instead, voters chose to leave the EU.
As for Metaculus, it trains its forecasting eye on all things scientific. For example, a majority of Metaculus users correctly predicted that the LIGO research team would announce the discovery of gravitational waves by the end of March; a majority also correctly predicted that an Artificial Intelligence player would beat a professionally ranked human player at the ancient game of Go, and that physicists at the Large Hadron Collider would not discover a new, subatomic boson particle.
“We tend to get a core group of researchers who lay out strong arguments, with good discussions that follow,” Laughlin said. “We’re intentionally focusing on a narrow selection of topics. We’re not about sports betting, entertainment, or fashion.”
As for the forecast for Metaculus itself: A recent feature about the website in the science journal Nature prompted a wave of interest within the academic community. Metaculus now has about 3,000 registered participants.
But participation is not limited to scientists. Laughlin said the ideal Metaculus member is a well informed “science nerd” who has a passion for discovery and a desire to learn. “President Obama is kind of our target audience,” Laughlin noted.
The emphasis on science and technology is what separates Metaculus from other crowd-sourced prediction sites, Laughlin added. Typical questions include: “Will SpaceX launch for Mars in the 2018 window?” (only 39% of participants thought so at the time of this story’s publication) and “Will a species extinct for more than 1,000 years be brought back by 2025?” (70% of respondents say yes).
The site has questions about chemistry, robotics, nanotechnology, psychology, statistics, infectious diseases, and computer algorithms. The questions are accompanied by background information and commentary from participants, with multiple perspectives on a given topic.
By registering on the site, each participant is able to rate the likelihood of a prediction by attaching a percentage to it. Participants also are able to make comments, suggest new questions, and earn points based on their predictions.
“We have users who are particularly sophisticated,” Laughlin said. “When you have that kind of group weighing in, you get educated guesses and informed opinions. It brings together the full spectrum of viewpoints on issues.”
There are tangible benefits to Metaculus, Laughlin added. The site identifies promising areas of research for scientists and funders alike, while also being a good gauge for science and technology writers looking for trend stories, he said.
“And it’s fun,” he said. “We spend quite a bit of time researching and writing the questions, making them as provocative and cutting edge as possible.”
For more information, visit the Metaculus website.