Did Stanford, even as a two-loss team, deserve a spot in the 4-team College Football Playoffs? At first, this seems like it should have an obvious answer; after all, for decades, the metrics for ranking teams were rather simple: if you were from a major conference and you had a worse record than another team from a major conference, odds are you were ranked beneath the team with the better record.
But is that really the best way to calculate rankings? Might it be that Stanford completing a very difficult schedule with just two losses was as impressive as --for example--a Michigan State completing a relatively softer schedule with one loss?
When I first approached this question, my instinct was to say "yes, what Stanford did was every bit as impressive as what any other team did." After all, Stanford faced a whopping five ranked teams in its 13-game schedule, and no team in either year of the CFP Rankings has ever faced more than five ranked teams in a single season. Furthermore, no team that had faced five ranked teams had ever performed better than Stanford's two-loss season.
But there's more: Stanford faced 10 bowl-eligible teams in its 13-game slate, meaning that Stanford had to face an absurd number of teams that were .500 or better (which is one of the statistics the CFP examines when determining its rankings).
All of this led me to hypothesize that, with the exception of perhaps Alabama and Oklahoma (the two favorites to win the national championship this year), there is literally nobody in the country who is likely to perform better than Stanford if they had to face Stanford's schedule.
But why speculate when we actually have the statistical tools to evaluate this objectively? So, instead of making a nice-sounding argument, I turned to statistics and was curious how many games each of the top teams would be projected to win under Stanford's schedule.
Before I show you the results, a quick word on...
Methodology:
For my inputs, I decided to use three different projection systems. The first is ESPN's Football Power Index (FPI) because it correctly predicted 78% of the games that it projected this season, making it the most successful of all the publicly-available models for this season. The second is the Sagarin Rankings because, although FPI was best this year, Sagarin's long-term track record is second to none and Sagarin has consistently performed among the very most successful prediction models for at least the past 15 years. Finally, I used the S&P model of our very own director of analytics right here at SB Nation, Bill Connelly, since he's been providing informative numbers and insights to this site for years.
From each of these three models, I was able to get a point spread for each team. I then made an adjustment for home field, and through a linear regression, I converted the point spreads into probabilities of each team winning. Once we knew how likely each team was to win against each opponent on Stanford's schedule, it was trivially easy to convert that into a win total for the season.
Got it? Cool. On to the good stuff.
Results
What record would each team be projected to have under each of these models? Let's find out:
Color coding: red = teams projected to have more than three losses, yellow = teams projected to have between 2-3 losses, green = teams projected to have fewer than two losses.
Interestingly, just about any team across the country would be projected to have around two or more losses should they have to endure Stanford's schedule. The team with the very best of it was Alabama, but even they were projected to have about two losses (1.57 losses) if they had to play Stanford's schedule.
And if your name isn't "Alabama" or "Oklahoma," you were projected to have more losses than Stanford actually finished the season with.
Hell, even if your name was "Stanford," you were projected to have more losses than Stanford actually finished the season with (Stanford actually performed about one win better than expectation).
What the hell does this all mean?!
I don't know! I just like numbers and put up a lot of numbers up for you to enjoy and examine yourself!
Okay, okay, okay. I won't cop out and I will give you my take on what this all means.
What this means is that Stanford's schedule was really difficult when compared to other teams' schedules across the country. It's not too surprising that over the past two years, no team that's scheduled five ranked opponents has ever finished with fewer than two losses. As you can see, the deck is stacked against just about any team in the country --and the deck's even stacked against the very top teams in the country-- if they had to play Stanford's schedule.
So, does that mean that Stanford had a rightful claim at a CFP slot? Maybe, maybe not.
The goal of a playoff in any sport isn't necessarily to have the best teams play against each other, but instead to reward the teams who had the best seasons. In any sport, there are always arguments from fans as well as analysts that so-and-so should've made the playoffs, but didn't. And that's fine because the goal of a playoff isn't to have the best teams, but instead to reward the teams with the best records.
But which teams in college football had the best records (and how do you even evaluate which record is best)? Well, that's a very subjective question. Some might say a team that survived a very difficult schedule (even with two losses) is more impressive than a team that survived a relatively easier schedule with only one loss. After all, this very argument has been used to keep even undefeated Group of Five teams out of the playoffs in favor of one-loss Power Five teams (and perhaps rightfully so). But where do you draw the line? If you're willing to make this argument to keep a Group of Five team out of the playoffs, shouldn't this argument also be extended to Power Five teams that played more difficult schedules over other Power Five teams that didn't?
THAT is what I find a really interesting philosophical question, and one that I still don't know the answer to (just because it's a very subjective question). In the absence of all teams playing against each other, how can you objectively evaluate who had the best season (and, therefore, who is most deserving of a postseason berth)?
How do you "know" that an undefeated team from a Group of Five conference is having a worse season than a one-loss team from a Power Five conference? And, for that matter, how do you "know" that a two-loss team that endured a brutally difficult schedule is worse off than a one-loss team that played a relatively softer schedule?
You really can't know for certain. And sometimes it's just easier to rank the two-loss team behind almost all of the one-loss teams.
But there is one thing that can be stated with absolute certainty: Stanford surviving five ranked teams with only two losses is, thus far, the high mark for any team. No other team has ever done better than that, and even the very strongest teams this year would've had a difficult time emerging with fewer than two losses had they played a schedule like Stanford's. Is it really that surprising, then, that Stanford--even as a two-loss team--was still on the cusp of earning a playoff berth?
Comments
Matt,
I think the difficult thing to measure is the cumulative effect of playing quality teams consecutively.
Look at what happened to Baylor when they had to play a tough opponent after getting pounded the week before and losing key people to injury.
They didn’t have the luxury of putting in the back-up against some cream puff team.
If there tough games were more spread out, their loss count might have been less.
So when Ohio State only played five P5 bowl eligible teams this year compared to Stanford’s ten, Stanford’s schedule was actually more than 2x tougher.
Hard to really quantify that though.
By Cardinal-in-TX on 12.09.15 12:13pm
nice analysis
love the projections of other teams on the stanford schedule!
I have my doubts that the committee will ever use strength of schedule to upgrade a team with more losses over a team with fewer losses, particularly to get a team in the top 4. we will see as its only been 2 years. I suppose it could have happened if UNC had beaten clemson.
also, in retrospect it seems like a big mistake to schedule NW…i dont see the reason to have a 2nd power 5 team when we have ND on the schedule and I dont know of any other teams in the top 25 that schedule 2 out of 3 non-conference games against Power5 opponents. Heck, most only schedule 1 out of 4 non-con games against power5 (though Michigan played Utah, Oregon State, and a decent BYU team)
We were also unlucky in that no other Pac-12 team had an "elite" record this year. Oregon losing to MSU and Utah dropping from the rankings did not help. Perhaps if we played a top 10 ranked 10-2 Utah instead of an 8-4 USC team we might have had a shot.
Will be interesting to see how this plays out in the future as some other conferences are increasing their schedule strength. Big 10 is moving to 9 conference games next year, and I think at least 1 power 5 non-con game and no FCS schools. Of course it doesnt help us this year.
By WhatDoestheTreeSay? on 12.09.15 12:47pm
They did put Stanford ahead of OSU
But I wonder how heavily the committee actually debated that since it didn’t really have an impact on the final top 4 and/or can be easily justified by OSU failing to make it to the Conference Championship game whereas Stanford smashed faces in their Conference championship victory. Now that I think about it 2-loss Stanford ahead of 1-loss Ohio State may not provide much extrapolation.
By nickdempsey on 12.09.15 2:29pm
I mean, the massive issue is that the SOS that the committee and ESPN use
is wholly based on opponent W-L records. That means that if you played a P5 Northwestern that finished 10-2, it’s the same as playing a Sun Belt Appalachian State that finished 10-2, when everyone knows that’s absurd.
Hence, any team in a conference that plays fewer conference games (cough, SEC, cough) will automatically have a higher relative SOS by DEFINITION of the stat, which changes your other metrics (like how your defense is ranked — all SEC teams get a higher "defensive" ranking because they’re presumed to be playing a "tougher" team by virtue of SOS). Let alone the actual "poll" ranking which is heavily biased towards pre-season domination by certain conferences (cough, SEC, cough).
To wit: last year, using these "SOS" calculations, Alabama’s defense had a ranking of something like 20, while Oregon’s defense was ranked something far worse (67 or something). Subsequently, Ohio State played both of them consecutively in the CFP, and put up the EXACT SAME amount of points on Alabama as Oregon — and actually had LESS time of possession against Alabama.
By Visionary_ensemble on 12.10.15 11:37pm
Interesting Observation
I have been wondering how Stanford could play more ranked teams and more Power 5 teams going to Bowl games yet have a strength of schedule decidedly lower than Alabama, for example. If what you are saying is true (and I have no reason to doubt it), the SOS used by BCS and ESPN are total crap. Though, to be honest, looking through all of the rankings of SOS it is hard to understand SOS differences (even within conferences with many common opponents). While there is a measurable difference between say #25 and #90, is there actually that much difference between say #4 and #14?
I think the analysis up top feels right to me. Michigan State, Iowa and UNC are probably not as good as Stanford, Fla State, Notre Dame and Ohio State ….. whatever their current record and rankings show. Unfortunately, the optics of wins and losses trumps advanced stats. Also, the BCS "policy" of not penalizing a loser in a conference championship game seems wrong for those teams that have just not played anyone throughout the year (Iowa and UNC). It may not be relevant for the playoff, but it leaves Stanford with a sub-optimal bowl game vs. Iowa rather than Ohio State.
By hoyaparanoia on 12.11.15 9:23am
I mean think about it this way:
If Stanford simply played Charleston Southern instead of Northwestern, and basically was of the Big 12 or SEC OOC mindset, they’d be 12-1 with a last second 2 point conversion fail away from undefeated. And I’m pretty sure we’d beat Charleston Southern.
Are we suddenly a way better team, and should be in the CFP? We, quite literally, would probably be ranked higher than Alabama. Such a switch off is redonkulous.
Yet teams from certain conferences (cough, SEC, cough) play teams that are artificially awesome relative to SOS contribution which allows them to play teams like Charleston Southern but still keep their SOS strong. It’s silly.
By Visionary_ensemble on 12.11.15 5:54pm
So if I'm reading this right..
You think the rankings should be:
Is that right?
By thunsicker on 12.15.15 11:28am