Evaluating Catchers (October 22, 2003)
Hope you like it...
--posted by TangoTiger at 05:12 PM EDT
Posted 6:16 p.m.,
October 22, 2003
(#1) -
Nick S
I like it. You are determining catcher fielding runs in a similar manner to determining batting event values from BaseRuns by doing a small perturbation (e.g. take away 1 single, recalculate), except by removing one catcher from his career. This works because of the massive sample size (as you note, you manage to catch the average baseline, pretty much) and, as such, is not likely to be useful for evaluating individual seasons. It is terrific, though, for evaluating a career in retrospect (i.e. Hall of Fame arguments), and I'm sure everyone (both) people who are reading this section of the site would be curious to know how Mr.'s Rodriguez and Piazza have faired in this metric to date.
Posted 6:28 p.m.,
October 22, 2003
(#2) -
tangotiger
Unfortunately, I don't have access to the PBP files from 1993-1998. Not only am I missing out on a good chunk of years, but all that cross-referencing that I do would also be missed.
Posted 6:47 p.m.,
October 22, 2003
(#3) -
Bob Dobalina
Whether the work is "revolutionary" will be left to the historians to decide, but your common-sense approach seems to have yielded fruit. I heartily enjoyed the article. Please keep it up.
Posted 9:58 p.m.,
October 22, 2003
(#4) -
Michael Humphreys
Great work. It sets a new standard for rating catchers.
Posted 12:56 a.m.,
October 23, 2003
(#5) -
Hatrack Hines
Tango, where do you get PBP data for 1999 to 2002?
Posted 9:09 a.m.,
October 23, 2003
(#6) -
Tangotiger
Follow the link to "Ray Kerby" from the article.
Posted 9:32 a.m.,
October 23, 2003
(#7) -
tangotiger
Thanks for your kind words, guys.
Posted 5:23 p.m.,
October 23, 2003
(#8) -
FJM
Fascinating stuff! Here's another way you could use this data. For those catchers where you have plenty of data on both sides of age 30, treat them as two different players. You could then determine how much each catcher lost in each category as he aged.
Posted 5:47 p.m.,
October 23, 2003
(#9) -
ColinM
This is brilliant work Mr. Tiger, far and away the best evaluation of catcher defense I have ever seen. It seems to pass the Bill James 80% test, notice how the top ranked catchers are mostly the guys with the best reputation to begin with.
One small suggestion... If I understand your methodology right, it looks like you could run into sample size problems in the case where a pitcher spent the great majority of his career with one catcher. Take Charie Lea from your Gary Carter example. Lea only had 805 PA with other catchers. But this gets pro-rated to match the 3061 he had with Carter, producing large deltas of -12. Lea's "without Carter" numbers are given as much weight as Scott Sanderson's, even though Sanderson has 7 times as many PA with other catchers. To avoid this, maybe it would make sense to use the lower of the two sets of PA as the weight, and pro-rate everything at the end. So in the case of Lea, the deltas would be about -3 and the PA weight 805. Then you can add up the deltas, and multiply them by ActualPA/WeightedPA to get the total for a career. What do you think, am I insane?
And maybe it's just the lingering insanity, but couldn't this work for other fielders too? I mean, all of the stuff we adjust for: GB:FB, Lefty:Righty, this should all be accounted for in the selection of pitchers for each fielder. As with the catchers, the comparison environment should be the same as the fielder's actual environment EXCEPT for park factors, which is a big exception of course. But then PF could always be added afterwards.
Posted 7:30 p.m.,
October 23, 2003
(#10) -
Vinay Kumar
Excellent stuff, Tango.
Posted 10:57 p.m.,
October 23, 2003
(#11) -
Tangotiger
Thanks again guys!
FJM: hmmm... good idea.
Colin: almost always, when I do these types of studies, I take the "lesser of the two PAs". The primary reason I didn't do it in this case was that it was a little more work than I wanted to do. And, I would have to do that fudging you mentioned to bring the PAs back to their norm. However, all the other actuals for the catchers would not necessarily match. Because of the sheer number of pitchers, I feel pretty safe that any oddball effect would be drowned out mostly. However, I would certainly have no problems doing it as you suggest.
Posted 7:32 a.m.,
October 24, 2003
(#12) -
David Smyth
Nice work, Tango. I'm wondering, did you look at the unadjusted rates for thoses outcomes to see how much of a difference it really makes to adjust according to pitcher? Especially for the majority of pitchers who have little or no contact with knucklers.
Posted 7:33 a.m.,
October 24, 2003
(#13) -
David Smyth
I meant "catchers" in that last sentence, not "pitchers".
Posted 7:51 a.m.,
October 24, 2003
(#14) -
Tangotiger
That's a good point... no I did not.
Posted 1:38 p.m.,
October 27, 2003
(#15) -
Arvin Hsu
Great stuff, as always, Tango.
For 28 of the 29 catchers, the "other catchers" was between -1 to +1. Benedict was compared to a +2 baseline. In essence, by doing this huge meshing, we are just about capturing a league average baseline.
When you did this, did you do a weighted average? If so, what were the weights based on, IP(caught) for each pitcher, then weighted for the catchers the pitchers caught?
That's a good point... no I did not.
hehe. Doing the more difficult adjustment before looking at the easier one. :)
Colinm: . Then you can add up the deltas, and multiply them by ActualPA/WeightedPA to get the total for a career.
I'm not grasping what you're trying to do. Doesn't the multiplication undo what you did by taking the lower IP-weight?
-Arvin
Posted 1:47 p.m.,
October 27, 2003
(#16) -
tangotiger
In the example in the article, I would take Foote's rate, and apply it to the 2176 PAs with Rogers. I do this with all catchers that Rogers had. This tells me how good the "non-Carter" catchers were. And I repeat the step, etc, etc....
***
What Colin's trying to do is scale back up the # of PAs, so that the actual # of PAs is preserved.
Posted 1:49 p.m.,
October 27, 2003
(#17) -
Chris Dial
Really great stuff, Tango. I think it takes teh work Keith and I did to the next step - if catchers aren't lowering ERA, where is their value? ANd your work here is great.
Posted 2:42 p.m.,
October 27, 2003
(#18) -
Arvin Hsu
In the example in the article, I would take Foote's rate, and apply it to the 2176 PAs with Rogers. I do this with all catchers that Rogers had. This tells me how good the "non-Carter" catchers were. And I repeat the step, etc, etc....
wow. fully weighted and recursive. very nice.
Posted 5:07 p.m.,
October 27, 2003
(#19) -
tangotiger
In response to Colin, and looking only at Carter's PB and WP, and Carter's top 20 pitchers:
The total # of Carter PAs was 47,476. However, if I weight by the lesser of the two PAs (with and without Carter), the PAs is 40,724.
If I prorate the WP and PB down to the lesser of the 2 PAs, I get:
Carter is 52 WP and 36 PB better than his baseline catchers.
Increasing all the totals by 17%, so that we put the PAs back in line to 47,476, and Carter is 61 and 42 better. In the original process, he was 79 and 51 better.
Only 2 pitchers were affected by this process: Steve Rogers and Charlie Lea. As it turns out, the 2 of the 4 pitchers that benefited the most from Carter were.... Rogers and Lea.
The effect is 27 extra WP and PB over 47,000 PAs, or about 1 run per season.
So, what we have is that:
- the majority of the pitchers did throw to other catchers more than they did to Carter
- of all catchers, Carter would have this effect the most
- of the pitchers who's other catchers had their PAs prorated up, they were particularly better with Carter
And after all this, the net effect is 1 run per season (PB and WP only).
I would have to say that even though taking the lesser of the two PAs has some good aspects to it, you have to figure that the impact of not doing so will be pretty small.
Posted 8:21 p.m.,
October 28, 2003
(#20) -
ColinM
Thanks for checking that Tango. Once again, great study.
Posted 9:41 p.m.,
October 28, 2003
(#21) -
Charles Saeger(e-mail)
Good study, Tango. Awfully darn clever, I might add.