Statistical Analysis: Projecting NFL QB Failure Using College Stats

[The Franchise]

I originally posted this in the Official Geno Smith thread.....but I was told that this needed it's own thread.

It was brought up a while ago that Matt Stafford was a lock for the Lions at #1 back in 2009 from the very beginning. I didn't think that was true....so I did some research to see what the "experts" were saying about Stafford in advance of the draft. And I found this article (http://espn.go.com/blog/nfcnorth/pos...tafford-debate).

This is from the 1st half of the article.

The experts are hedging. The fans are sweating. The team is making clear it is considering all of its options.

There are 47 days remaining until the 2009 NFL draft, giving the Detroit Lions some 1,125 hours before they are required to make the No. 1 overall pick. The Lions might need every minute of that span, especially if their internal discussion at all reflects the raging public debate on Georgia quarterback Matthew Stafford.

A classically built, strong-armed quarterback, Stafford has not yet caught on as the consensus No. 1 pick. ESPN.com draft analyst Todd McShay, for example, said recently the Lions face a "nightmare" decision because Stafford is "not mentally ready" to take on the pressures of being the No. 1 overall pick. McShay said that scouts from at least 10 teams agreed with that assessment and added: "I just don't feel great about building my organization around him."

NFL Network draft analyst Mike Mayock told a Detroit radio station that "there are some things about him that bother me," and even Stafford's biggest supporter advocates with a negative argument. Yes, Mel Kiper Jr. said the Lions should select Stafford primarily because "there is nobody else to take."

Even fans are getting into the act. On the day the Lions announced they were playing host to Baylor offensive tackle Jason Smith, Jim of Cincinnati wrote:

Matthew Stafford? Why are people thinking he is a good fit for an 0-16 team? I have seen Stafford play. He gets rattled easily. His arm is ok but his leadership skills lack. Next year they can get a much better QB. This year they need to fill in the holes on defense and on the line.

Why all of this generalist hate against Stafford, who by all accounts offers fine character as well as the draft's strongest arm?

So....typical bullshit that we're hearing right now. Not worth the #1 overall pick....blah blah blah.

What I found really interesting was the 2nd half of the article....which is this.

Our friends at ESPN Research have developed a method for fleshing out the debate with statistical analysis. Using time-honored performance standards to predict future success for "blue-chip" quarterbacks, the formula placed Stafford between Akili Smith and Cade McNown in a category reserved for busts.

Does this mean Stafford is guaranteed to crash and burn? Of course not. But this evaluation documents in specific fashion the previously ill-defined criticisms of Stafford, helping to explain why there is so much disagreement about him with the draft little more than six weeks away.

The formula takes into account three statistics: Career starts, completion percentage and touchdown-interception ratio. The theory is that experience, accuracy and production versus mistakes can provide substantive indicators for college quarterbacks.
Formula Explanation
ESPN Research developed this formula to measure quarterbacks relative to a baseline completion percentage of 60 and a touchdown-interception ratio of 2.25. The multipliers allow each figure to have equal weight with career starts, which provides an important measure of experience.

The total score is the sum of the three adjusted figures.

The separate parameters for BCS and non-BCS quarterbacks help level the statistical playing field. They are based on the assumption that NFL-caliber quarterbacks playing against non-BCS opponents are going to have inflated numbers.

For those mathematically inclined -- it took me 10 readings to get it after having nightmare flashbacks to eighth-grade algebra -- below is the formula itself. (Note: This is the updated, corrected version. The formula in the original post was incomplete. Thanks to SwampThing86 and a few others for the heads-up.)

For BCS quarterbacks
(Career Starts x 0.5) + [(Career completion pct. - 60)x5] +[(Career touchdown-INT ratio - 2.25)x10]

For non-BCS quarterbacks
(Career Starts x 0.5) + [(Career completion pct. - 60)x2.5] + [(Career touchdown-INT Ratio - 2.25)x5]

(For a complete explanation of the formula, see the text box on your right.)

To test the formula, ESPN Research plugged in the 31 quarterbacks taken in the first round over the past 12 drafts, dating back to 1997. The results are below.

You'll see the quarterbacks broken into three categories. If their college statistics translated into a value of 20 or more, there was a strong likelihood for success. (Alex Smith and Tim Couch notwithstanding.) A value between 1 and 19 essentially meant "iffy."

Group I: Strong likelihood of success
Player School Draft year Score
Matt Leinart USC 2006 64.04
Philip Rivers NC State 2004 48.44
Tim Couch Kentucky 1999 47.64
Alex Smith Utah 2005 44.88
Aaron Rodgers California 2005 40.58
Peyton Manning Tennessee 1998 39.47
Jason Campbell Auburn 2005 38.75
Byron Leftwich Marshall 2003 36.39
Ben Roethlisberger Miami (Ohio) 2004 33.85
Chad Pennington Marshall 2000 33.53
Mark Sanchez USC 2009 32.63
Daunte Culpepper Central Florida 1999 30
David Carr Fresno State 2002 23.97
Joe Flacco Delaware 2008 23.92
Eli Manning Ole Miss 2004 23.14
Donovan McNabb Syracuse 1999 21.62

Group II: Hit-or-Miss
Player School Draft year Score
Brady Quinn Notre Dame 2007 18.93
JaMarcus Russell LSU 2007 18.64
Rex Grossman Florida 2003 18.39
Vince Young Texas 2006 18.21
Carson Palmer USC 2003 16.35
Matt Ryan Boston College 2008 9.14
Patrick Ramsey Tulane 2002 9.06
J.P. Losman Tulane 2004 7.86
Jay Cutler Vanderbilt 2006 2.39
Josh Freeman Kansas State 2009 1.94

Group III: Busts
Player School Draft year Score
Akili Smith Oregon 1999 0
Matthew Stafford Georgia 2009 -4.55
Cade McNown UCLA 1999 -6.41
Joey Harrington Oregon 2002 -6.85
Michael Vick Virginia Tech 2001 -11.32
Ryan Leaf Washington St. 1998 -16.92
Jim Druckenmiller Virginia Tech 1997 -20.25
Kyle Boller California 2003 -50.67

I decided to do some research and throw together the college stats of the QBs drafted since 2009 and see where they fall.

Here are the numbers of the last 3 years based on their stats and that formula.

Geno Smith 83.1
Matt Barkley 75.4
E.J. Manuel 50.7
Tyler Wilson 19.1


Andrew Luck 68.7
RG3 74.4
Ryan Tannehill 23.65
Brandon Weeden 56.35
Russell Wilson 36.3
Nick Foles 48.3

Cam Newton 54.3
Jake Locker -17.9
Blaine Gabbert 19.7
Christian Ponder 20.3
Andy Dalton 34.6
Colin Kaepernick 28.1

Sam Bradford 74.5

Keep in mind these things when comparing those numbers to the list from 2009.

1. How much do you knock down the numbers of QBs that come from a "spread offense"?

2. These numbers aren't 100% cut and dry on whether those players will succeed or fail. I mean ****....is Locker worse than Leaf? Probably not. Is Alex Smith better than Aaron Rodgers? No way. But it's basing everything off of the number of starts, the completion percentage and the TD/INT ratio they had during college.

3. These numbers ONLY factor in passing stats. They do not include rushing statistics.

4. These numbers don't factor in extenuating circumstances or outside factors that they can't account for (David Carr having no offensive line or Matt Leinart being a playboy that didn't care about football). This is only based on stats.

When McShay, Mayock and Jim from Cincinnati express their concerns about Stafford, it's primarily for these reasons: College quarterbacks don't typically improve their accuracy in the NFL. If his decisions were at all suspect against SEC opponents, then it's reasonable to wonder how he will react to professional defenses.