Founded in March of 2008 by baseball analyst Nate Silver, FiveThirtyEight originally began as a polling aggregator, its name derived from the number of electors in the U.S. electoral college. Well known for its election predictions, it successfully predicted the outcomes of both presidential races in 2008 and 2012. Silver pioneered unique methods incorporating statistics and demographic data not used before. As the organization grew, it would go on to receive many awards as its focus broadened under the rubric of data journalism, but it received significant flak in 2016 and 2020 for projections that significantly diverged from electoral results. Today, over half of Americans indicate they are skeptical of polling. Given this mixed reputation, how reliable is FiveThirtyEight?
How Factual Is FiveThirtyEight?
The Factual’s news rating algorithm analyzes more than 10,000 articles a day along four metrics: author expertise, publication history, writing tone, and cited sources and quotes. (See our How It Works page to learn more.) For this study, we analyzed 1,000 articles each from 245 major news sources.
Over a dataset of 1,000 articles, FiveThirtyEight scored an average Factual Grade of 75.9%. This is significantly above the average of 61.9% for all 245 news sources that we analyzed. This puts FiveThirtyEight in the 97th percentile of our data set and in the top 10.
These high scores are not much of a surprise given FiveThirtyEight’s strong commitment to data. The site relies heavily on data and uses extensive sourcing to substantiate claims, meaning most stories include numerous links to high-quality external sources. Likewise, stories are generally authored by experienced members of a dedicated team of journalists. These individuals routinely cover the same topics, indicating strong author expertise.
How Opinionated Is FiveThirtyEight?
The Factual measures how opinionated an article is using a sophisticated natural language processing algorithm, producing a score we call the Writing Tone. For this metric, the algorithm looks for signs of subjective commentary (e.g., first person pronouns, unnecessary adverbs), as well as the emotional nature of selected words, and sees how prevalent they are for a given length of text. Text which is less opinionated gets higher ratings, with “0” being the most opinionated and “1” being the most neutral.
FiveThirtyEight had an average score of 0.47 placing it in the 31st percentile in our dataset. This suggests that articles from FiveThirtyEight are often highly opinionated, meaning they are likely to use emotionally loaded language. Partially, this can be attributed to the sports journalism that the site largely focuses on, which involves articles that generally involve more opinion and argumentation, as seen through titles such as “An Aggressive Kemba Walker Could Turn Around Boston’s Season.”
What Is FiveThirtyEight’s Political Bias?
The Factual classifies news sites by political bias as either Left, Moderate Left, Center, Moderate Right, or Right. This classification is derived from third-party assessments from media bias organizations such as AllSides and Media Bias/Fact Check (MBFC). Based on this data, The Factual assigns FiveThirtyEight a Moderate Left bias.
AllSides assigns a “Center” bias to FiveThirtyEight based on 8,266 community ratings and independent research. While AllSides notes this as an initial rating, it emphasizes a three-year partnership with the New York Times, which AllSides has designated as “Lean-Left” for the past three years. FiveThirtyEight’s coverage of the 2016 presidential election received criticism from both sides of the political spectrum, on one side for referring to Trump as not “a real candidate” and for downplaying Sander’s primary bid on the other.
MBFC assigns a “Left-Center” bias based on story selection that typically favors liberal causes. It notes a tendency to publish factual information that utilizes emotionally loaded headlines like “Is Joe Biden Toast If He Loses Pennsylvania?” and “What The Heck Is Going On With AstraZeneca’s COVID-19 Vaccine?” However, MBFC gives the site high scores for factual reporting, proper sourcing, and a clean fact-check record.
Who Owns FiveThirtyEight?
FiveThirtyEight had grown from a personal blog to a highly respected site with decorated journalists. After a three-year partnership (2010–2013) with the New York Times, Silver wanted to transform the site into a mature media organization. ESPN bought the blog in 2013, expanding the newsroom from two full-time journalists to a staff of 20. In April of 2018, ABC News acquired ESPN, making Disney the overarching owner of the FiveThirtyEight.
Disney is an immensely successful media conglomerate with interests in all corners of entertainment. It has an annual revenue of close to $67 billion — most of which comes from advertising. In 2019, during the launch of a slew of Disney media products, many critics ESPN network sports analysts for advertising the new Disney+ service. Many critics contended this degraded consumer confidence in the various networks’ news coverage. However, this does not appear to have implicated FiveThirtyEight. Overall, there is no evidence that ownership by ESPN, ABC News, or Disney has influenced the publication's output.
Please check your email for instructions to ensure that the newsletter arrives in your inbox tomorrow.
Why Does It Matter?
News articles are bound to have bias because all authors have some frame of reference within which they describe a story. Political bias ratings are helpful in understanding this framing. However, it can be more beneficial to know how factual an article is based on quantifiable metrics that can be seen across the media ecosystem, from cited evidence, to author expertise, to the writing tone. This is what The Factual ascertains.
Reading several, highly rated articles from across the political spectrum helps counter the bias of any news source or story. To have the day’s most factual news stories delivered to your inbox every morning, subscribe to our daily newsletter.