The Coaching Class of 2004 | by Mike

With Year 2 of the Charlie Weis tenure officially in the books, the search is on for metrics by which to judge his performance. What numerical data can we use to check our biases and provide context? Over the long term, every Notre Dame coach will be judged by whether or not he delivered a national championship. However, while five different coaches have won national championships at Notre Dame, none of these championships was won in the coach's first two years. Thus it's too early to apply this standard to Weis.

One of the obvious measures by which to judge a coach in the first few years of his tenure is his performance relative to his predecessor. In his first two years, Charlie Weis guided Notre Dame to 19 wins. In the two preceding years, Tyrone Willingham guided Notre Dame to 11 wins. While Irish fans certainly appreciate those eight extra wins (and may appreciate the eight fewer soul-crushing losses even more), it's not clear how much this tells us about Weis at Notre Dame.

To provide context, we decided to look at the change in win totals for all the Division I coaches hired following the 2004 season. Twenty-two new head coaches were hired that offseason. Because there was such a large amount of turnover (changes at almost one-fifth of the D-I schools), we should have a decent-sized sample to examine. A Dennis Dodd article from early 2005 provided a comprehensive list of the new hires at that time, and that list is reproduced below. (Dodd's ranking of the hires is also included, for entertainment purposes only.)

Dodd's Rank	Coach	School		Dodd's Rank	Coach	School
1	Urban Meyer	Florida		12	Frank Solich	Ohio
2	Steve Spurrier	South Carolina		13	Dave Wannstedt	Pitt
3	Ed Orgeron	Mississippi		14	Walt Harris	Stanford
4	Mike Gundy	Oklahoma State		15	Dick Tomey	San Jose State
5	Les Miles	LSU		16	Hal Mumme	New Mexico State
6	Kyle Whittingham	Utah		17	Terry Hoeppner	Indiana
7	Tyrone Willingham	Washington		18	Bronco Mendenhall	BYU
8	Charlie Weis	Notre Dame		19	Shane Montgomery	Miami (OH)
9	Ron Zook	Illinois		20	Skip Holtz	ECU
10	Greg Robinson	Syracuse		21	Brent Guy	Utah State
11	Mike Sanford	UNLV		22	Bill Cubit	Western Michigan

Now let's compare the win totals at these programs in the two years preceding the new coach's arrival and in the two years following his arrival. The following list ranks the coaches by this change in win total:

Coach	School	Wins (03-04)	Wins (05-06)	Delta	Dodd's Rank
Bill Cubit	Western Michigan	6	15	+9	22
Skip Holtz	ECU	3	12	+9	20
Bronco Mendenhall	BYU	9	17	+8	18
Charlie Weis	Notre Dame	11	19	+8	8
Dick Tomey	San Jose State	5	12	+7	15
Frank Solich	Ohio	6	13	+7	12
Urban Meyer	Florida	15	22	+7	1
Terry Hoeppner	Indiana	5	9	+4	17
Steve Spurrier	South Carolina	11	15	+4	2
Ron Zook	Illinois	4	4	0	9
Tyrone Willingham	Washington	7	7	0	7
Les Miles	LSU	22	22	0	5
Brent Guy	Utah State	6	4	-2	21
Walt Harris	Stanford	8	6	-2	14
Hal Mumme	New Mexico State	8	4	-4	16
Mike Sanford	UNLV	8	4	-4	11
Dave Wannstedt	Pitt	16	11	-5	13
Mike Gundy	Oklahoma State	16	11	-5	4
Greg Robinson	Syracuse	12	5	-7	10
Kyle Whittingham	Utah	22	15	-7	6
Ed Orgeron	Mississippi	14	7	-7	3
Shane Montgomery	Miami (OH)	21	9	-12	19

What do these numbers tell us, and how much significance should we attach to these deltas? Let's dispense with the low-hanging fruit. The first conclusion we can draw is that Dodd's prognostication abilities are lacking, and in this respect I think the numbers are fairly reliable. Dodd ranked just three of the nine coaches that improved on the prior two years' win total in the top half of new hires. Even worse, three of the six worst totals were produced by coaches Dodd ranked among the six best hires. While (for reasons that will be discussed below) one cannot look solely at the delta in evaluating coaching performance, the numbers are not flattering to Dodd.

Moving on to the more substantive analysis, the numbers suggest that one cannot look at the delta in isolation. The programs that had the highest win totals in 2003-2004 generally garnered fewer wins in 2005-2006 following the coaching change. Conversely, the teams with the fewest win totals in 2003-2004 generally improved their win total in 2005-2006 following the coaching change. I believe there are two phenomena at work here. First, the three programs in this group that had 20+ wins in '03-04 were left searching for new coaches in the winter of 2004 because their coach had left for greener pastures. Thus the new coaches at LSU, Utah, and Miami (OH) were all following coaches who had exceeded the historical performance standards at their program. One would expect matching such standards would be difficult.

Second, I believe it is harder to improve on a mediocre win total than an abysmal win total for three reasons: (1) when the win total is higher in the previous years, there are fewer opportunities to garner additional wins; (2) as the previous win total increases, the quality of teams against whom additional wins can be garnered will tend to increase; and (3) increases on truly bad performances may simply represent regression to the mean. This suggests that an individual coach's delta will be "inflated" relative to his peers where his absolute total of wins in 2005-2006 is middle-of-the-pack, and an individual coach's delta will be "undervalued" relative to his peers when his absolute total of wins in 2005-2006 is high.

A few examples illustrate this point. Les Miles, Tyrone Willingham, and Ron Zook all failed to improve on the preceding two years' win total. However, even if Miles had won every game the past two years, he would only have improved on Saban's record by four wins. Saban's record in his final two years at LSU was 22-4 and Miles matched this record in his first years. Yet if Miles had notched these four wins, he would have won back-to-back national titles, and would be regarded more highly than any other coach on this list. Conversely, Willingham matched Keith Gilbertson's 7-16 record and Ron Zook matched Ron Turner's 4-19 record. Willingham and Zook inherited bad situations, but there are many losses to bad teams in Willingham and Zook's tenures at Washington and Illinois. Would it have been too much to ask Willingham to beat, say, Air Force? The truth is that matching Nick Saban's performance is simply a greater achievement than matching Gilbertson or Turner's, even if the result is the same delta.

Urban Meyer's numbers tell a similar story. Four coaches improved their school's record by more wins than Meyer, but it is Meyer that has reached the mountaintop. That Florida's players were able to win 15 games in 2003-2004 despite Zook's mismanagement speaks volumes about just how much talent Meyer inherited. Yet to generate a +7 improvement in wins, Meyer had to defeat teams like Ohio State and LSU, teams that finished in the top five. Again we see that the difficulty of achieving marginal improvement increases with the previous success of a given program.

(I recognize that many attribute much of Miles' and Meyer's recent success to Bo Pelini, Jimbo Fisher, and Charlie Strong. While I can agree with this to an extent, a head coach deserves credit for hiring strong coordinators and must share the blame for hiring weak coordinators.)

In conclusion, while I think the change in a team's win total tells us something about coaching quality, such numbers must be viewed in light of the starting point. The raw delta is not as informative as it might initially appear. In this respect, I do see a positive sign for the Notre Dame faithful. As down as Irish fans are following the Sugar Bowl loss and the last-minute defection of a handful of recruits, the long-term trend for the program remains positive. Only three coaches in this sample were able to deliver improvement on a double-digit win total: former national champion coach Steve Spurrier (+4 on 11 wins), current national champion coach Urban Meyer (+7 on 15 wins), and our own Charlie Weis (+8 on 11 wins). Time will tell whether Weis can build the defense necessary to meet the real measuring stick of great coaches - a national championship.