Richard Billingsley, curator of the Billingsley Report, which is a component of the BCS, responds to the recent article by Yeshayahu Ginsburg.  His written rebuttal follows in its entirety.

Here we go again………………

College Football is a unique sport, unlike anything else on earth, and because of that it possesses some of the most unique fans on earth. Passion beyond belief. Because of college football the entire state of Alabama comes to a standstill when the Crimson Tide meets the Auburn Tigers. That passion plays itself out every Fall Saturday in some corner of America, whether it’s USC and Notre Dame or New Mexico and New Mexico State the passion for the pigskin has been playing itself out for 142 years.

It’s because of the uniqueness of this sport, over 120 teams playing within 11 separate athletic conferences, across a broad continent with absolutely no chance of round robin scheduling, that defining and national champion is a daunting, and some would say impossible task. Still, attempts have been, and continue to be made. Some would say, as the Associated Press did in 1936, that human observation is the best way. The American Football Coaches Association would agree, they just believe they’re panel of voters is perhaps, more qualified. Mathematicians believe algorithms are the only way to settle such a complex question. And then there is Richard Billingsley, a devout college football fan who realizes that for the most part, the sportswriters and coaches have done a pretty darned good job, save one issue….inherent human bias. Regardless of how hard a sportswriter or coach tries, somewhere, somehow, a little bias is going to come out. Maybe regionally, maybe by conference, maybe by team, but at some point it’s going to influence a decision, maybe critically, like in selecting teams for a national championship.

When I created my system 1970, my goal was to follow the lead set by the AP and UPI (who administered the Coaches Poll at that time), but to do it in an unbiased way. Mathematics was the answer, but I wanted to use math within the framework already in place by the human polls. I wanted an improved human poll. In order to accomplish my goal I established these important parameters, a solid starting position, a balance between wins, losses and strength of schedule (where one does not completely overshadow the other), and most importantly, a powerful head to head rule that would be fair, one that would adjust the starting position if needed, quickly, on a weekly basis. The end result was a ranking system easy enough for the average fan to understand and appreciate, accurate enough for author and ESPN Editor Michael MacCambridge to call brilliant and impressive enough for Roy Kramer, creator of the BCS, to use in determining the National Championship in college football.  All in all, not bad for a country boy from Oklahoma with only a high school education.

Mathematicians have called for me to be thrown out of the BCS and fans love me or hate me, depending on how I rank their favorite team that week. That goes with the territory, they’re just being fans. I’ve had California fans burn me in effigy and LSU fans hail me as the greatest pollster ever. I’ve had Oklahoma fans call me a traitor and Texas fans send me death threats.  I’ve had Brigham Young fans send me the blessings of Joseph Smith, and Auburn fans tell me, not so politely, that I don’t know Didley.

That, my friends, is the passion of college football. Fans are not always easy to deal with, but once I talk them through to a greater understanding of the system and they eventually come around…..usually.

But not everyone comes around. Occasionally I run across someone who has their mind made up beforehand that my rankings are worthless, and no amount of dialog is going to convince them otherwise. Such is the case of Yeshayahu Ginsburg who recently wrote an article for Pollspeak.com . I remember my offseason interview with Mr. Ginsburg quite well, I enjoyed it, but by conversations end I knew I had not made a lot of headway in changing his mind about anything. After reading his article I realized my suspicion was correct. Mr. Ginsburg made the same mistakes so many critics of the Billingsley Report make, he tried to critique a system without fully understanding the rules, looked only from his predetermined negative view, and refused to see that the very points he considers to be flaws are key, positive points within the BCS. The Billingsley Report is NOT like other BCS computers and that is a good thing, not the bad thing he tries to convince readers it is. Roy Kramer, the creator of the BCS, specifically brought my rankings into the mix because of the very things Ginsburg calls “flaws”.

I’ve never understood why critics want me to conform to the other 5 BCS computers. What would be the point of duplicating 6 programs? I was brought in BECAUSE I’m different, I’m outside the box. So, when Yeshayahu Ginsburg says Billingsley is the “outlier” more than anyone else (30% of the time), yes, that is very true and I’m damn proud of it! But Billingsley is the outlier from what? The answer: The other 5 BCS computers. The real comparison that should be made is the high correlation the Billingsley Report has to the human polls, (check out the final polls here ), and the high correlation Billingsley has to the BCS standings. (Check out the Final BCS Standings here). That’s what the readers on PollSpeak need to be made aware of. Why would I strive to correlate with something that brings me LESSER results than the official BCS standings? The BCS has made it crystal clear they respect human polls more than computer polls. Why do you think they are headed towards a human selection committee? The fact that the Billingsley Report conforms more to human polls than any other BCS computer speaks volumes, positively, about my rankings. The Billingsley Report is, in essence, a computerized human selection committee. Think about that.

Also, I need to be very clear about something, just because my rankings correlate well with human polls does not mean I consider the Billingsley Report to be “better” than the other 5 BCS computers. We all look at things from a different perspective. I have a very healthy respect for all 6 BCS computer programs and believe every single designated number one team is worthy of being called a National Champion, even if it differs from mine. These guys believe in their programs, and put so much work into it, and have devoted so much time to the sport. They deserve the respect of every college football fan in the world. They helped maneuver college football through the greatest transformation in its history. Without computers there would have been no BCS. Without the BCS there would have been no college football playoff. Do you think we would have ever seen a playoff without first having the BCS?

Let me point out a few facts that readers need to be aware of as counter points to Yeshayahu Ginsburg’s article. First of all, to say my system is “flawed” because it does not agree with the other 5 BCS computers is absurd, certainly, a bad choice of words. I can assure you, my differences to other computer programs are very appreciated within the BCS, and not considered “flaws”.

In regards to what he calls my “fatal flaw”, nothing could be further from the truth. Having a starting position has been proven to create a much more accurate strength of schedule, and creates a more accurate ranking system on the season overall. It is an accepted, and in fact, encouraged practice by the greater majority of computer polls and prognosticators.  Ginsberg says I’m great at picking the top two teams but the starting position “throws the whole thing off” for the rest of the teams. If the rest of the rankings are “off” then they would be less accurate right?  Let’s look at the facts. If we describe “accuracy” as higher ranked teams beating lower ranked opponents, then according to an outside independent monitoring service called the Prediction Tracker, the Billingsley Report has been the most accurate BCS computer program over the last 3 years. Here are the documented percentages, and by the way, for programs that do NOT use margin of victory, these are remarkable winning percentages for all 6 systems:

• Billingsley- 73.2%

• Anderson/ Hester- 73.1%
• Sagarin Elo-Chess- 70.7%
• Wolfe- 70.6 %
• Colley- 69.8%
• Massey BCS - Not Tracked

• Billingsley- 71.8%

• Sagarin Elo-Chess- 69.4%
• Anderson/Hester- 67.6%
• Wolfe- 67.3%
• Massey BCS- 67.1%
• Colley- Not Tracked

• Sagarin Elo-Chess- 71.6%

• Billingsley- 70.9%
• Anderson/Hester- 70.5%
• Colley- 70.5%
• Massey BCS- 70.0%
• Wolfe- 70.0%

In looking at those results, I certainly would not say the Billingsley Report is “off”.

Let’s also look at the 2009 season that Ginsburg highlighted. He says that the 2009 starting positions of Ball State and North Texas threw everything out of whack for the entire 2009 season, so completely that it destroyed any credibility in later rankings, even giving my eventual National Champion Alabama “an unfair boost” in the rankings. Indeed, North Texas did benefit from an opening week win over what turned out to be a slumping Ball State team. What Ginsberg failed to recognize (even after my repeated explanation to him) is that because of the internal checks and balances in the program, after the next game, if the new ranking was proven not to be justified, ratings of both teams would be adjusted. North Texas started #119 and jumped to #77 after beating Ball State (hardly unfair and far from “skyrocketing” as Ginsburg declares. The next week the Mean Green lost to Ohio and fell to #93. There is the quick internal needed adjustment. Alabama got a onetime credit for playing #93 North Texas, in doing so Alabama received a weekly total of 1.424 points. Compare this to the 16.090 points they received for playing #2 Texas and tell me, does it look like Alabama received an unfair boost for playing North Texas?

My work is completely available to the public. My suggestion to you, as you read this, is take time to review my work for yourself. It’s all there, 142 years of data in Billingsley’s Online College Football Encyclopedia.  After looking at the All Time Scores and the Final Ranking comparisons, decide the validity of the Billingsley Report for yourself, don’t rely on someone else’s opinion.

Richard Billingsley

August 28, 2012

Comments (4)

Subscribe to this comment's feed

Billingsley Preseason Rankings and CFB Turnover
written by T1DEJP, August 28, 2012

Mr. Billingsley, your theory would be more applicable to professional sports, where a team can build around a nucleus of veterans (see Boston Celtics, Chicago Bulls, NY Yankees).

In college, 25% of the team--usually the best players--leaves every single season. Assigning the rank earned by the previous year's team to the current year's team is just asinine.

Look at Auburn in 2010. That team, led by Cam Newton, turned college football on its ear. They barely squeaked by Utah State in the game immediately following the BCS NCG.

Perhaps the Billingsley report should begin after Week 1 and use one of the other polls as its starting point?

• 
• +2
• 
• 

...
written by Yeshayahu Ginsburg, August 28, 2012

First of all, I would like to thank you for the response. I was disappointed that you felt the need to make part of this disagreement personal, but you did respond to what I believe is a flaw, which is all that matters.

I am well aware of what others (such as Death to the BCS) have said about you and your system. I intentionally did not reference any of that. I merely wished to discuss your system on its own merits and pointed out what I still feel is a problem.

I enjoyed our interview as well. I believe that I understand the system as well as anyone who does not actually know the formula can. And if you recall, there was what I believed to be a second flaw in the system. Your explanation of that was more than satisfactory and I did not reference it at all in the article either.

You are correct in that the system has checks. A team who is overranked due to a mistaken preseason rating will lose more points for a loss (because of their current ranking) and get less for a win. But that is reactionary. The system never can solely judge a team on their merits. It has to react to what a team earns or loses. If there were many more games in a season then those little overrankings and underrankings would eventually even themselves out so that the preseason mistakes get weeded out. But these differences don't have enough time to even themselves out with only 12 (or 13) games. But with so few games, each game has such an impact that one team being misranked will affect every team they play, which will then affect every team that they play.

Just as a note, I happen to appreciate your system the most of any of the computers. Part of it is that your system is so complex yet you still explain the basic details of it more than any other computer (with the exception of Colley who publishes his formula) does. It is precisely because yours is the computer that I enjoy delving into most each week that this problem with it bothers me so much.

• 
• +1
• 
• 

...
written by Yeshayahu Ginsburg, August 28, 2012

And I might add that your comparison of the Texas game to the North Texas game is a bit unfair. The later-season games are worth more and there are more points involved later in the season. And the fact is that even a few extra points is too many. Let's say that a third team ran the table in 2008 and fell behind Alabama by .5 to end the season. That North Texas game, because of Ball State's overrating, could very well have affected who ended in what position. The fact that it didn't doesn't change the fact that it could have, which is where I believe the problem in the system lies.

• 
• +0
• 
• 

...
written by buby74, August 29, 2012

I felt the original article was unfair to Billingsley. His system is very different to the other computer systems but that is not necessarily a flaw, it is a difference. It is not my favourite system but to speak to someone at length and then write about the “flaws” in their system struck me as unfair. If he took the time to explain to you how he ranks all the teams then you should apply the same level of criticism to the other computers and most importantly the human voters.

Billingsley seems to have attempted an objective way of replicating the voting behaviour of a poll voter who moves from week to week and is influenced by the ranking of teams at the time the games are played. If you consider Billingsley flawed are not all the human polls flawed as they use the same or similar logic?

What is the correct way to run a computer ranking system in your view?

In comparison the Colley and Wolfe systems treat all games equally which potentially misses out on momentum or a change in team form or injuries. So a team that goes 8-0 and then 0-4 is treated the same as a team that goes 0-4 and then 8-0 all else being equal. Is this “flawed” in your view?

That said his system is quite odd so here is a test for Billingsley’s system. I think he should re-run the last few seasons REVERSING the initial ranking. Or start all teams at a ranking of joint 60th place. That will give some idea of how crucial his pre-season ranking is.

I think an opportunity was missed not to improve the computer rankings. The BCS should have properly evaluated any computer system that was interested according to the following rules.

1.Fully published and explained so everyone knows what they have to do and how hey will be ranked.
2.Allowed to use MOV (the biggest criticism of the BCS computers)-Human voters take it into account and I'm sure the selection committee will so why ban the computers?
3.Allowed to use home field advantage.
4.Explain how they treat FBS and lower opponents. Do they ignore them, merge them into a single super team or rank all the teams that play football.
5.Explain if all games are treated the same or if they weigh more recent games more heavily.
6.Explain if they use pre-season rankings and show what effect they have on the final results.

In my view there is no single best way to rank teams using computers. But if a large enough selection of systems was chosen and published then that would be the best way to rank teams or at least give a guide to the selection committee. I think lower football divisions uses either a group of computer polls or the RPI in the selection process of their playoffs so I dont know why the BCS playoff will be different.

To declare an interest I have a computer system which is on the Massey comparison site which uses MOV and home field advantage, treats all the games the same and has no pre-season rankings and includes all teams down to division II.

James

• 
• +1
• 
•