ROFL @ The BCS
October 27, 2009
The first BCS rankings of 2009/10 have brought a nice opportunity to examine some problems that will perpetually befuddle the NCAA's BCS arm.
One problem is that the BCS Committee prohibits the use of margins of victory in any computer rankings system they utilize. Some may not see this as a problem at all, as the philosophy behind the policy seems sound and reasonable. That is, the BCS does not want to give teams a reason to run up scores needlessly (which might pad ratings if margins of victory were utilized). Also, many people seem to like the "a win is a win" adage.
That sounds nice, but this approach can yield terrible rankings. I have already written a lengthy article about what might have happened in 1997 under the present BCS set-up (click here for that). Since that article is about what might have been, it seemed like something about the current situation would lend more to the argument. Here is my next effort...
I am not going to go into tremendous detail. All we have to do is look at two sets of rankings devised with and without margins. The rankings will speak for themselves.
The first set of rankings we shall examine are made with a system devised as what I would do if tasked with making a BCS-complaint method. In this system, all games are deemed to have the same score: 2-0 victories for the winner. The number two is arbitrary - any number could be used. I used 2 so the program could be allowed the intermediate score of 1 to be used when applied to leagues where ties are possible (all ties are assigned the score of 1-1).
The ranking procedure starts with writing an equation for every game. If Michigan beats Notre Dame, then the equation derived is: Michigan = Notre Dame + 2. Every game produces an equation like this, and then the huge system of equations is solved with linear algebra. The principles are simple, but solving such a large set of equations is no trivial task. I have written many programs to solve large groups of equations. I hope my past work is enough to inspire trust in readers. (For any who doubt the results, verification is left as homework!)
Note that I have not mentioned the "home field advantage" ("HFA" for short). If one so chooses, HFA can be added to the system by adding its value to the score of every road team. However, I believe that if the BCS wanted true idealism, they would prohibit the implementation of HFA correction.
What we are left with is the purest and most elegant ranking system I can imagine for a large group of linked teams. I call it the "WLT" system (for win-loss-tie). Here are are the WLT rankings after the 8th week of the 2009 season: click here.
Well, I'm sorry, but despite the ideal theory and sound logic, these rankings are terrible. Boise State is #12, and Oregon #3, even though Boise State soundly beat Oregon (19-8), and has thrashed all their other opponents. Arizona, with two losses, is above undefeated Cincinnati. What is going on here?
First of all, notice that Iowa is #1. Five of the six BCS systems also have Iowa at #1. This is a strong indicator that my simple system has fairly well replicated what is bound to happen when rankings are made without consideration of margins of victory. This fact is the primary reason I have taken the liberty of using these rankings to criticize the BCS.
Next, let's examine rankings that involve margins. If computer ratings where all wins are treated the same do not produce appealing rankings, one would hope that adding the actual game scores to the mix would help. There are any number of ways to calculate rankings based on scores, and again, I do not want to go into great detail, so I am going to again use a system that is easily explained, but complex in action. I call it the "networked transitive comparison," or "NTC" for short.
For the NTC system, if Michigan beats Notre Dame by eleven points, write down this equation: Michigan = Notre Dame + 11. Do the same for all games, with the number added to the loser being the game's margin of victory. Then solve the full system of equations to find a rating (solution) for each team (variable).
For the NTC system I have used a home field advantage of exactly three points. Any value could be used, and a more realistic value for college football is probably about 3.5 points. However, for the sake of simplicity and to not have to justify any other number, I have chosen to use HFA = 3, as many people seem to have this simple estimate firmly planted in their minds.
Now, I do not assert that NTC rankings are all that great. However, some obvious improvements over the WLT system can be seen. Click here for the NTC rankings.
In the NTC rankings, Boise State is #7, just 0.06 points behind Oregon - quite an improvement and obviously more realistic, in my opinion. However, here Iowa plummets to #16. Iowa deserves to be higher. Texas, Alabama, and Florida seem better candidates for the top slots than Iowa, and I'm confident most oddsmakers would agree with this statement. Iowa barely got by Northern Iowa and Arkansas State. (Iowa fans, please keep in mind I have nothing against Iowa. If they go undefeated, they will rise in the NTC rankings, and only after the regular season finishes would I give an opinion on their worthiness for the BCS title game.)
So, what have we found? Neither system can do justice for all the undefeated teams. Personally, I don't think being undefeated automatically means a team should be rated highly, however, this year is different. In the past, teams like Boise State could more easily be dismissed for their weaker schedules. This year Boise State (beat Oregon), Texas Christian (won at Clemson), and Cincinnati (won at Oregon State) are all legitimate contenders. If the BCS's rankings are not fair to these teams, it will be a travesty. So far it appears that Boise State is going to be unfairly held down by the BCS.
Does Montana deserve to be a Top 25 teams? Probably not, but BCS-style rankings make this claim. Idaho was just blasted by Nevada, but the WLT rankings have Idaho #35, and Nevada #55. A long list of problems could be made.
To some extent, any ranking system will produce some oddball results at this point in a season. Seven or eight games are just not enough to even out the effects of upsets and highly dissimilar schedules. Therefore, some might object that this is not a good time to criticize the BCS. My response to that would be that my whole point is there are no perfect ranking systems, no matter how many games have been played. This just happens to be a time when the point can be made exceptionally well.
There is really no way to prove any of this, though. The best evidence lies in the fact that year after year, a lot of teams seem to have legitimate complaints that the system is unfair.
Do I have a solution? No. I just think putting a fascade of science on the process of determining which teams are worthy of a championship game is folly, and as long as it continues, I will try to help others appreciate the comedy of it all.
I greatly enjoy making and analyzing power ratings. However, after tinkering with them for over fifteen years, I can confidently say... There is no "right answer" with power ratings. No team can be encapsulated with a number. Power ratings are nothing more than a game about games.
Note: This was written sort of off-the-cuff. I might turn it into the basis for a running commentary on the BCS's futile efforts, including updates of the NTC and WLT ratings. If I do, a link will appear here. Bookmark this page and check again later if you may be interested.
Click here for the continuation...