ARLINGTON, Va. — The last eight years have been quite a ride for Nate Silver.
Before he ever got into politics, Silver built player projection models for Baseball Prospectus. They still exist under the name he bequeathed them, , an inside joke aimed at only the most down-the-rabbit-hole of baseball nerds.
But when he took his statistical models and applied them to politics, his projection system crushed the analysts in the 2008 presidential election. His site was bought by The New York Times, where he further cemented his reputation by nailing all 50 state projections in the 2012 presidential election.
Silverās site, (or 538) has since moved under the ESPN umbrella, where it has expanded substantially both in size and scope, diving into the world of data journalism. But as election season heats up, many turn to Silver to find to help them make sense of it all.
Silver was in town Tuesday for at George Mason Universityās Mercatus Center. But before he took part, he sat down with Ā鶹¹ŁĶų to talk about the difference between the Times and ESPN, āhorse raceā politics, the trouble with polls, why NFL coaches keep kicking and punting, and why PECOTA hates the Kansas City Royals so much.
Noah Frank, Ā鶹¹ŁĶų: What’s been the biggest difference about being under the ESPN umbrella as opposed to when your site was owned by The New York Times?
Nate Silver: I mean, there are a few fairly obvious things, and one probably is the scale of the operation. Weāre 25 or 30 people now ā at the Times we were effectively two or three full-time equivalents with the ability to get help from other people.
You spend a lot more time doing editing and management, but the flip side of that is you also have people who can do things on their own. I mean, thatās really fulfilling, when I donāt know something is going on until the end, and then Iām like, this is really amazing work. Thatās more fulfilling in some ways than when I have to be, like, very micromanagy and hands on.
But theyāre different climates, too. I think ESPN is somewhat more entrepreneurial than the Times. I think also 538 has a much kind of more ambitious mission where we want to be a focal point to cover all sorts of topics and not just bring in the nerds at the end. So thatās different. And the culture is different.
Iāve made the comparison before where ESPN is more like a big state school where people are pretty good, but youāre always aware of the scale. But itās unpretentious and people reward success.
Where The New York Times is an Ivy League school. You have lots of fascinating conversations, almost everything they do theyāre very good, but thereās a bit of, I donāt know, a certain type of elitism, I suppose (laughing). A weird type of, not close-mindedness, but thereās a lot of confidence at the Times that the Times way is the right way to do things.
You’ve recently tweeted about 538’s steady growth. How much energy do you invest in analyzing your own data, and what do you think has helped you be successful in creating sustained traffic where other sites may have failed?
It is different. I think I practically never looked at any traffic data at all when I was at the Times. I think traffic is a useful measure, up to the point where youāre trying to manipulate your journalism to win traffic. Weād rather have a story that 200,000 people respond to than 20,000 or 2,000.
At the same time, I believe in the long run, differentiating yourself with being a site people go to and know thereās a certain baseline level of quality, they know theyāre going to get something different. We hope itās right, too, but at the very least, the perspective you get on the election at 538, or on the NBA, is different than youāre seeing at other places.
Weāve had fairly steady growth and I think the feeling right now in a lot of media circles is you must be either really tiny or really big. And weāre kind of a weird outlier ā itās weird to be an outlier when youāre medium, right? But there are not that many medium-sized sites. If weāre kind of medium on the verge of large, then economically thatās more helpful for us than medium on the verge of small. We think we kind of are medium on the verge of large now. But still, we want to stand out in an environment where there are dozens of media outlets competing for sort of the same turf.
You talked about covering a broad range of topics. I looked through the archives, and there have been only 10 articles on FiveThirtyEight with the āmusicā tag. Do you think itās possible to quantify musical trends and styles, and what are your thoughts on algorithms like Pandoraās and Spotifyās Discover Weekly?
So, Iād love to do more with music. More of the choices than you would think are the result of our main culture writer is a big movies guy and not as big a music guy. So things that people think are highly deliberate choices are not necessarily. But we would like to do more with music. We did a big thing on . And those stories do well, but yeah ā all the areas where people are like, āI wish you covered this more,ā I probably agree, for the most part.
Someone this same question, but Ralph Nader said back in 2004 that if Americans brought the same discerning eye to the political process that they do with sports ā understood the metrics and statistics and dynamics ā weād be much better off as a country. Is that essentially what youāre trying to do with 538? Is there a way to make politics exciting without cheapening it?
I mean, certainly some of the original conceit of 538, back when it was kind of just me back in 2008, was that I had written about baseball for several years during the early Moneyball era, if you were to call it that, and was frustrated that you had very little of that in politics. It was all about gaffes and momentum, where āmomentumā usually means what was happening a week ago and not what was happening next week, which is kind of what it should mean potentially.
The difference, though, was that in politics you always had people who were close to the campaigns who are very analytical. People know [about] Obama and his analytics team, or Ted Cruz this year. But Karl Rove and George Bush were pretty good at that stuff, too, in 2000 and 2004.
So, itās more really the way politics is covered by the media. And for better or for worse, this is the part that gets us in trouble sometimes, but thereās a fairly explicit critique of āhorse raceā coverage at 538. Itās not that we donāt care about the horse race, that we donāt care about the polls, but people are kind of taking data and looking it at ways that are not very revelatory and can be pretty superficial.
Your model has buffers built in to accommodate for and correct bias in polls. Have you seen any reaction from the more biased polls in the way they ask questions as a result of your ratings of them? If so, how have you adjusted your own model to compensate for those changes?
I was going to say you donāt want polls to try to be more accurate, but look ā thereās an issue called herding, which means some pollsters seem very concerned about being close to the average, close to what Real Clear Politics says, or 538, or pollster.com. Thatās really dangerous. It turns out that statistically, what happens when every poll tries to be close to the average, the average itself becomes less accurate. You have less independent information.Ā Itās as though thereās really just kind of one poll and everyone gloms onto that. And so, trying to avoid piling on a pollster when they show an outlier result.
Statistically, thereās way more variance in polling than you would think. If the race is really at, say, Clinton +5 in a state and you have 100 polls that say that, youāre going to have polls that say Bernie up 7 or Clinton up 21. And the impulse is, oh, the pollster must have done something wrong, but a good pollster will actually publish results that it expects are an outlier. Maybe it will go back and then survey the same state again. But yeah ā frankly I worry that sites like 538 have a slightly negative effect on the culture of polling.
But Iāve also kind of evolved myself a little bit, where in the earliest incarnations of 538 it was a little bit, āwe donāt care how you do your poll as long as you have accurate results.ā My view on that hasĀ changed a little, in part because some pollsters who were very good in 2008, 2010, the first couple years of the site, have been very bad since.
And Iāve also found that, to the extent you can quantify methodology, there are a couple of basic things you can do. For example, is a poll a member of major polling organizations that require basic levels of disclosure and transparency? That turns out to be predictive. That a poll can have a bad year, but if it applies a good methodology, it will probably end up being pretty good in the end, and vice versa, too.
I’ve , and 538’s own Benjamin Morris has , but the NFL is still rejecting basic probabilistic math every Sunday in regards to punting and 2-point conversions. I , but I’m curious to know whetherĀ you think we’ll see sweeping change and acknowledgment of this at some point and, if so, what will that look like. Will there literally be a staff member charged with advising the head coach of the percentage play given the situation?
I mean, you need to change the culture, and I think that a lot of cultural change is gradual, rather than something that happens overnight, necessarily. It is kind of perplexing how much the NFL is willing to give away in terms of expected value.
But I have a friend who advises an NFL team ā and I canāt mention which team ā and they were trying to give their coach a contract extension. And in the negotiations, [they] said āOh, by the way, we want you to apply better strategy on fourth down.ā And he said, āGive me my extension first, and then Iāll consider it.ā Because heās worried that even if he knows itās right, or could be led to believe that itās right, youāre swimming upstream against a lot of people who think itās wrong ā a lot of conventional wisdom. And ā I donāt know, I think people kind of underrate the importance of culture, and how hard it can be to overcome.
A lot of what happened in baseball is, you had people like Bill James, who back in the 1980s were laying the groundwork for this stuff. And so itās kind of this generational change, where you might have a general manager who grew up working for an analytics site and is at least fluent in that vocabulary. In that sense, if you saw change at the high school and college level ā although, even there, college coaches are probably a lot better about fourth down strategy than NFL coaches are.
Do you enjoy the irony of PECOTA’s hatred of the Royals (the primary team of former big leaguer Bill Pecota, after whom the system was named) consistently backfiring, and is this the year the system finally gets it right (projected 76 wins) and KC comes crashing back to earth?
If youāre not counting for how bullpens are used, and thatās a big part of it, I think there are things to roster flexibility that can matter a lot ā you saw how hard the Royals worked to exploit in the postseason. You know, I donāt know. I handed over the reins on PECOTA a long time ago. I would think when you have a pretty big miss a couple years in a row you then want to do due diligence on those questions at least, and say, āAre there things weāre not thinking about?ā
One problem, too, with making preseason predictions is that you consider a roster as being static and not how it might evolve over time. So a team with flexibility, and depth, and good management, there might be a way to measure that, potentially.
At the same time, weāve looked at over 12 years of data, and over 12 years of data, PECOTA has done a fair bit better than Vegas at team over/under lines. It gave something back last year, I think ā it was a pretty bad year for projection systems. But over time, to beat handicappers is a pretty impressive and high bar to clear.
Politics seems like such an all-consuming topic for 18 months before every presidential election. Do you find yourself wishing you had time to focus on other areas of analysis?
Yeah, for sure. Iāve become less of a baseball fan, because every other year the October baseball peak coincides with either a midterm or a general election peak, whereas with the NBA, you can watch playoffs in the spring, when I usually have a little bit more time to relax. It is tricky with politics becoming so all-consuming and how early it started this year. There was so much interest in July, even, about Donald Trump, and increasingly about Bernie Sanders, that itās pretty exhausting. Itās good if youāre running a website.
I had beers with my colleagues the other day, and Iām like, āI canāt believe theyāre making us work on Saturday.ā There was another debate. And theyāre like, āWell, Nate, youāre the boss.ā And Iām like, āF, yeah, I am, right.ā But if youāre trying to cover politics for a living, then itās amazing in one way. And this has become the most fascinating election weāve seen in a long time. But it is a little bit burnout-inducing, too, where every day there is some type of news.
