See copyright notice at the bottom of this page.
List of All Posters
MGL takes on the Neyer challenge (January 13, 2004)
Discussion ThreadPosted 1:54 p.m.,
January 16, 2004
(#29) -
villageidiom
In keeping with my new year's resolution to foster recursion, here's a link to Neyer's latest article, in which he references this study and also provides a link back to Baseball Primer.
Granted, his link is to Clutch Hits... but at least he was well-intentioned.
The Scouting Report, By the Fans, For the Fans - 1B Report (March 24, 2004)
Posted 1:52 p.m.,
March 24, 2004
(#1) -
villageidiom
Great stuff, as usual.
One danger in regressing to the fans is that, sometimes, the fans' evaluation could be based on reputation or distant skills than on current demonstrable skills. On the whole I'd expect it to work, but the danger comes once you start looking at individual players.
From a ballot perspective, I'd expect such reputational voting to come through on ballots where players on many teams were evaluated. We're all fans of baseball, but most tend to watch games of one or two teams often enough to make a judgment. If someone is evaluating players on 20 teams, in most cases they either (a) have TiVo and more than 24 hours in their day to watch it, or (b) have evaluated players based on a small sample of observation.
To test this, I suppose you could look at the average number of players evaluated per ballot that a given player appeared on. (Perhaps variance would be better?) For example, how many different teams are represented on each ballot where John Olerud was evaluated? The assumption is that higher averages (or variances) would indicate greater "reputational" evaluation.
As your sample of fans gets larger, I'd hope that actual witnesses will far dominate the guessers out there. Until then I'd fear that small samples will give nutty results at an individual player level. And, if that is happening, then regressing to the fans' rating would be akin to the Gold Glove award balloting.
BTW, could you add sample size to your table?
Once again, on the whole I think the fans' scouting report is fantastic, endlessly fascinating, and a reliable and independent confirmation of UZR. I'm just not sure how much credence should go to the fans' rating at an individual player level. IMO a simple 1/3 weight isn't enough. Perhaps the weight should vary by sample size, with a max weight of 1/3 and a min of 0?