The most important question regarding the crowning of college football's champion has always been whether the system got it right. In the poll days, it was a question of whether the votes were cast correctly, during the BCS era, whether the polls and computers combined to determine the two best teams for a Championship Game and now in the playoff era it's whether the committee has selected the four best. Unfortunately, this question is also the most ambiguous and difficult to answer objectively. While proponents of teams one through four surely believe the committee has done a spectacular job, fans from teams five and on are likely less enthusiastic about the results. The committee's current report card is below, and next up we'll be attempting an objective discussion on whether the committee got it right.
Entertainment Value: A+
Effect on Scheduling: A-
Transparency: C+
Committee Members: C-
Consideration of Past Champions: D-
The Four Best Teams
Weighing Conference Championships
The Four Best Teams
The Selection Committee's stated mandate is to find the four best teams to compete in the college football playoff. Whether picking the four best teams is preferable to the four best conference champions remains debatable, nevertheless, "best" is the stated goal of the Selection Committee.
In the first two season of the playoff era, however, the four best seems to have given way to the four most deserving.
In the first season of the Selection Committee, Alabama, Oregon, Florida State and Ohio State made the cut. The two Big 12 co-champions, TCU and Baylor, were shut out of the post season playoff, despite being statistically superior teams than Florida State. This goes back to the previous discussion of the Seminoles getting a boost from their Past Championship, but is worth examining closer.
If the mandate is simply to find the four best teams, the most literal translation of this is the four teams that would be favored in games over every other team. There are numerous statistics that are used to identify this, and as the successful enterprise that is Vegas bookkeeping can attest to, they are quite accurate. Some of these statistics include FPI, F/+ and Jeff Sagarin's rankings, all of which held TCU in the top 6, while Florida State's highest metric was 14th.
Even the essence of subjectivity that is the "eye test" demonstrated clearly that Florida State struggled terribly to close out the season while TCU was playing some of its best football.
The argument can be made that because Florida State was an undefeated champion of a Power Five conference that they deserved to prove themselves in the playoff, and it's a solid argument to be sure. But again, that is not the mandate of the committee.
The four best.
That's what they claim to be looking for, and it is the phrase that had the SEC thinking they might sneak several teams in the door when all this was first being debated. Despite the push for only conference champions, the objective remained open ended, the four best.
And yet, despite the fact that TCU was also a conference champion, or at least a co-champion, and was statistically a better team than Florida State, the Seminoles made the playoff, as what can only be described as one of the four most deserving teams.
Numerous articles have been written with regard to how the committee ought to be selecting these teams, and there are strong points to be made about conference championships, undefeated seasons and teams that are more deserving due to strength of schedule or conference difficulty. While these are all fair points, we can only evaluate the committee based on their own stated objective, which is finding the four best teams.
In this past season, a similar situation occurred. In this case Michigan State, despite being ranked outside of the top ten in FPI, F/+ and the Sagarin Rankings, was considered one of the four best teams. Meanwhile Stanford, a team who sat within the top ten in all of these statistics, was left out. Stanford's two losses compared to Michigan State's single loss was likely the difference here, but if the goal is to find unequivocally the four best teams, the penalty of a loss shouldn't be as severe as the benefit of a far more difficult strength of schedule.
The Committee could have considered a common opponent, Oregon, in this situation, who the Spartans beat early in the season and Stanford lost to towards the end, if not for the massively important Vernon Adams factor, the Ducks star quarterback who was injured for the game against Michigan State.
As evidence of the discrepancy in the ability of these two teams, the Cardinal proceeded to overwhelmingly thrash Iowa in the Rose Bowl 45-16, the same Iowa team Michigan State went to the final minutes with in the Big Ten Conference Championship and edged out 16-13, while the Spartans served as a doormat for eventual National Champion Alabama, to the tune of 38-0.
Again, while it can be argued that Michigan State's one win better record made them the more deserving team, the committee should recognize that four most deserving and four best teams are not the same thing.
Stanford certainly cost themselves a shot in the playoff by inexplicitly losing to Northwestern the first game of the season, but that doesn't change the fact that they were one of the four best teams at the end of the season. Upsets happen, and when they do, they should have an impact, but the committee should also be clear and straight forward about what they are looking for.
While both seasons had one questionable selection, it is one out of four, so the success rate has been fairly good overall, and there will always be controversy near the cutoff line, it's one of the best aspects of the sport. The Committee, however, has not been particularly challenged in the past two seasons. We have not had a season that has multiple undefeated Group of Five teams, Notre Dame as a national power or a combination of Power Five Conference Champions that should not be overlooked as well as two from a single conference that are overwhelming. These seasons have occurred in the recent past, however, and are sure to spring up to challenge the committee in the years to come. It should be interesting to see how they react.
If college football is searching for the four most deserving teams to fill the playoff, the Committee has succeeded both years, if they are looking for the four best, they have failed.
Four Best: B-
Follow us on Social Media:
facebook: facebook.com/RuleOfTree/
twitter: twitter.com/RuleofTree
Comments
Really interesting analysis
I’ve very much enjoyed reading these articles. I do disagree with you, though, that Stanford should’ve been included in the playoff this year (and I’m a diehard Stanford fan). Let’s assume for the moment that there’s no BCS buster—no undefeated team outside the Power 5. It seems to me that if you have 5 P5 champions and 4 of them are undefeated or have only one loss, and the fifth has more than one loss, it’s absolutely an uphill argument that the last team should be included in the playoff. Maybe in very rare situation where the schedules are so different but the results so similar—say the one-loss team has played a ridiculously weak schedule, beaten no ranked teams, and lost to a very bad team, while the two-loss team has played the hardest schedule in the country, beaten multiple ranked teams, and lost in a close game to a very good team—could you justify taking the team with more losses. But that’s just not what happened this year; the Pac-12 and Big 10 were both very good conferences, and both Stanford and Michigan State played tough schedules. The advanced metrics were split on which team had a tougher schedule, but IIRC, more had Michigan State with a tougher schedule.* The advanced metrics are useful and often have predictive value, but the ultimate metric of a good football team is winning football games. That’s why we play the national championship games rather than just compare advanced metrics and call it a day. Michigan State won more games than Stanford before the bowl season, and there wasn’t enough disparity in their schedules or qualities of wins and losses at that point to take Stanford over MSU. So I don’t think it’s fair to say that "Stanford certainly cost themselves a shot in the playoff by inexplicitly [sic] losing to Northwestern the first game of the season, but that doesn’t change the fact that they were one of the four best teams at the end of the season." Stanford hadn’t proved that it was one of the four best teams prior to the Rose Bowl.
That said, even though I’m defending this as the right way to pick the playoff teams, it clearly got the wrong result this year. Stanford was undoubtedly a better football team than MSU, as indicated by the final rankings and the fact that Stanford shellacked the same Iowa team MSU had barely beaten on a last-second touchdown in the Big 10 Championship, while MSU got embarrassed by Alabama. Unfortunately, every game reveals something we didn’t know before. And in football, where there are so few games relative to other sports, that effect is magnified. So sometimes even the fairest process is going to get the wrong result. Not that the CFP is necessarily the fairest process, but I think it got the right result this year.
* Quick check seems to confirm this: Sagarin has Stanford with a tougher schedule, but FEI and S&P have MSU with the harder schedule.
By randalthor on 05.03.16 12:04pm
Great comment!
First of all, thanks very much for a well thought out and composed comment, I’m glad you got something from the article. Secondly, I understand what you’re saying, and I admit, Stanford versus Michigan State was a much tougher call to make prior to the post season. The metrics were close in some areas, but two major factors stuck out to me in favor of the Cardinal.
One, their schedule was one of the best in the country, 12 out of 13 games played against Power 5 teams and five against teams ranked at the end of the season. While Michigan State’s schedule stats were similar, I would hardly call the Big Ten a great conference this year. In full disclosure, I grew up a Penn State / Big Ten fan, and still catch a midwest game once a year with my friends, so maybe we both have higher standards for our die hard affiliations, but I wasn’t overly impressed this season. There were definitely some good teams, but no real great ones. Ohio State had talent, but couldn’t put it all together, Michigan is trending up, but not quite there, Penn State is still a mess and the other side hasn’t been elite for a while – Northwestern, Wisconsin and Iowa all good, but no great wins amongst them. I think Michigan State’s best win overall would be against Oregon, if not for the fact that Vernon Adams was injured.
And two, just watching them play. While the styles are a contrast for sure, the combination of Hogan’s leadership and McCaffery’s flat out talent was staggering. He was the Heisman winner in my book for sure, and I think if you compare where he falls in the draft next year to Henry’s 45th overall selection this year that will be abundantly clear. So for me, and as much as I hate this term, I guess the eye test plays in as well.
Regardless of those points though, I think what you’re getting at is similar to what I was discussing in this article, as well as the previous installment. The selection committee claims to be picking the four best teams, not the four most deserving teams. And while I agree wholeheartedly that as the one loss Big Ten champion, Michigan State was more deserving than the two loss Pac-12 champion, who lost to a Big Ten team no less, I disagree that Michigan State should have been perceived as a better team.
But who knows, maybe I’m just bitter that I didn’t get to see Stanford vs Alabama, wow that would have been something!
By Mike_Francis on 05.04.16 11:57am
Wrong comparison in '15-16
It wasn’t Michigan State who was the questionable choice, it was Oklahoma. They played a complete garbage OOC schedule and didn’t play a single quality team until the final three weeks of their regular season.
The question I ask when looking at the candidate teams is what would team A’s likely record have been if they had played team B’s schedule? I think Stanford goes at least 11-1 with Oklahoma’s schedule and has a good shot to be 12-0. I don’t see Oklahoma most likely 10-2 and not winning the North division with Stanford’s schedule.
By winflop on 05.06.16 5:51am
And The Good Teams They Played At The End of the Season....
…..had injured QBs.
By hoyaparanoia on 05.06.16 9:58am
In The Relatively Short College Football Season.....
Losses will kill you, but they do not necessarily say a whole lot about the team. Stanford had several games last year where they almost won, almost lost and won on the final play of the game. If those things play out differently, Stanford has a very different record. But it is still the same team that thrashed Iowa. Weather, injuries, travel (particularly cross country to the East Coast) can impact single game outcomes. It would be nice to sit back and try to pick the four best teams at the end of the year without regard to record but that is a really tough thing to do. I think the playoff committee is prepared to take into account schedule strength in accommodating a team that has one loss more than another team with a weak schedule…………….but once you catch two losses it is really hard to prevail – particularly in a year when quality teams have one or zero losses.
By hoyaparanoia on 05.06.16 10:04am
Four best teams
If you really want the four best teams at the end of the season, most likely you won’t have the four teams that accomplished the most during the season. You can’t leave a Florida State team out that hadn’t lost a game in two years, and you can’t put a two-loss Stanford team in, because they did "inexplicitly" lose to Northwestern. (sounds like a complaint about a porn flick that didn’t seal the deal, think the word you want is "inexplicably".)
By vk on 05.06.16 10:05pm
Four most accomplished
Haha, good catch vk, and to your other points, and some of those above, I agree, those teams earned their way into the playoff. My issue is, if you’re going with the four teams that accomplished the most, let’s be honest and call it that, and not say we’re picking the four best. And if it’s the four most accomplished, then a conference championship should be a requirement for those teams that play in a conference.
And while I agree with the Oklahoma analysis in terms of record, I do think it’s important when knocking Big 12 schedules to keep in mind that they play nine conference games. While more conferences will be doing that in the upcoming years (other than the SEC), last season only the Pac-12 and Big 12 played nine. So when comparing a non-conference slate against teams from the Big 10 or SEC for example, the Big 12 essentially has a built in Power 5 opponent in addition to their three cupcakes, which is similar to a non-conference schedule for most schools, one tough game and three easy ones.
By Mike_Francis on 05.10.16 12:17pm