College Football Playoff: Analyzing Stanford's strength of schedule

Did Stanford, even as a two-loss team, deserve a spot in the 4-team College Football Playoffs? At first, this seems like it should have an obvious answer; after all, for decades, the metrics for ranking teams were rather simple: if you were from a major conference and you had a worse record than another team from a major conference, odds are you were ranked beneath the team with the better record.

But is that really the best way to calculate rankings? Might it be that Stanford completing a very difficult schedule with just two losses was as impressive as --€”for example--a Michigan State completing a relatively softer schedule with one loss?

When I first approached this question, my instinct was to say "yes, what Stanford did was every bit as impressive as what any other team did." After all, Stanford faced a whopping five ranked teams in its 13-game schedule, and no team in either year of the CFP Rankings has ever faced more than five ranked teams in a single season. Furthermore, no team that had faced five ranked teams had ever performed better than Stanford's two-loss season.

But there's more: Stanford faced 10 bowl-eligible teams in its 13-game slate, meaning that Stanford had to face an absurd number of teams that were .500 or better (which is one of the statistics the CFP examines when determining its rankings).

All of this led me to hypothesize that, with the exception of perhaps Alabama and Oklahoma (the two favorites to win the national championship this year), there is literally nobody in the country who is likely to perform better than Stanford if they had to face Stanford's schedule.

But why speculate when we actually have the statistical tools to evaluate this objectively? So, instead of making a nice-sounding argument, I turned to statistics and was curious how many games each of the top teams would be projected to win under Stanford's schedule.

Before I show you the results, a quick word on...

Methodology:
For my inputs, I decided to use three different projection systems. The first is ESPN's Football Power Index (FPI) because it correctly predicted 78% of the games that it projected this season, making it the most successful of all the publicly-available models for this season. The second is the Sagarin Rankings because, although FPI was best this year, Sagarin's long-term track record is second to none and Sagarin has consistently performed among the very most successful prediction models for at least the past 15 years. Finally, I used the S&P model of our very own director of analytics right here at SB Nation, Bill Connelly, since he's been providing informative numbers and insights to this site for years.

From each of these three models, I was able to get a point spread for each team. I then made an adjustment for home field, and through a linear regression, I converted the point spreads into probabilities of each team winning. Once we knew how likely each team was to win against each opponent on Stanford's schedule, it was trivially easy to convert that into a win total for the season.

Got it? Cool. On to the good stuff.

Results

What record would each team be projected to have under each of these models? Let's find out:

Color coding: red = teams projected to have more than three losses, yellow = teams projected to have between 2-3 losses, green = teams projected to have fewer than two losses.

Interestingly, just about any team across the country would be projected to have around two or more losses should they have to endure Stanford's schedule. The team with the very best of it was Alabama, but even they were projected to have about two losses (1.57 losses) if they had to play Stanford's schedule.

And if your name isn't "Alabama" or "Oklahoma," you were projected to have more losses than Stanford actually finished the season with.

Hell, even if your name was "Stanford," you were projected to have more losses than Stanford actually finished the season with (Stanford actually performed about one win better than expectation).

What the hell does this all mean?!

I don't know! I just like numbers and put up a lot of numbers up for you to enjoy and examine yourself!

Okay, okay, okay. I won't cop out and I will give you my take on what this all means.

What this means is that Stanford's schedule was really difficult when compared to other teams' schedules across the country. It's not too surprising that over the past two years, no team that's scheduled five ranked opponents has ever finished with fewer than two losses. As you can see, the deck is stacked against just about any team in the country --€”and the deck's even stacked against the very top teams in the country-- if they had to play Stanford's schedule.

So, does that mean that Stanford had a rightful claim at a CFP slot? Maybe, maybe not.

The goal of a playoff in any sport isn't necessarily to have the best teams play against each other, but instead to reward the teams who had the best seasons. In any sport, there are always arguments from fans as well as analysts that so-and-so should've made the playoffs, but didn't. And that's fine because the goal of a playoff isn't to have the best teams, but instead to reward the teams with the best records.

But which teams in college football had the best records (and how do you even evaluate which record is best)? Well, that's a very subjective question. Some might say a team that survived a very difficult schedule (even with two losses) is more impressive than a team that survived a relatively easier schedule with only one loss. After all, this very argument has been used to keep even undefeated Group of Five teams out of the playoffs in favor of one-loss Power Five teams (and perhaps rightfully so). But where do you draw the line? If you're willing to make this argument to keep a Group of Five team out of the playoffs, shouldn't this argument also be extended to Power Five teams that played more difficult schedules over other Power Five teams that didn't?

THAT is what I find a really interesting philosophical question, and one that I still don't know the answer to (just because it's a very subjective question). In the absence of all teams playing against each other, how can you objectively evaluate who had the best season (and, therefore, who is most deserving of a postseason berth)?

How do you "know" that an undefeated team from a Group of Five conference is having a worse season than a one-loss team from a Power Five conference? And, for that matter, how do you "know" that a two-loss team that endured a brutally difficult schedule is worse off than a one-loss team that played a relatively softer schedule?

You really can't know for certain. And sometimes it's just easier to rank the two-loss team behind almost all of the one-loss teams.

But there is one thing that can be stated with absolute certainty: Stanford surviving five ranked teams with only two losses is, thus far, the high mark for any team. No other team has ever done better than that, and even the very strongest teams this year would've had a difficult time emerging with fewer than two losses had they played a schedule like Stanford's. Is it really that surprising, then, that Stanford--€”even as a two-loss team--was still on the cusp of earning a playoff berth?