Offensive Performance, Omitted Variables, and the Value of Speed in Baseball (November 6, 2003)
Thanks to Mike.
(I haven't read it yet, but I'll comment on it tomorrow.)
--posted by TangoTiger at 08:05 PM EDT
Posted 10:18 a.m.,
November 7, 2003
(#1) -
Nick S
I skimmed the paper, so maybe I missed his point, but it seems that he looked at calculated values of the SB/CS break even point and said "Teams seem to steal to much, why is that?" and offers that the calculated values are based on run value break-even points, whereas teams base decisions on win value break-even points. While this is a perfectly reasonable statement, I recall (though can't cite) work (I think by MGL) that SB attempts are randomly distributed throughout game situations.
Posted 10:23 a.m.,
November 7, 2003
(#2) -
tangotiger
I mentioned that the impact of the SB is rather randomly distributed. This was figured out using the Leveraged Index (LI) on the SB attempt. That is, rather than figuring out the LI for a reliever or starter, I figured it for an event. Things like "defensive indifference" had an LI of 0.01 I think, while the SB attempt was around 1.00 (I don't remember exactly, but it was somewhere between 0.90 and 1.10).
The win break-even point DOES vary wildly by game situations, anywhere from 60% to 80% if memory serves. (This will be covered in a few months).
Posted 12:26 p.m.,
November 7, 2003
(#3) -
Ted T
Two notes for those of you who decide to read the paper:
(1) Remember that it's an econometrics paper about baseball, not a baseball paper about econometrics. Because of the audience it's written for, some of the stuff in there is going to seem very basic to readers on this site. The main point of the paper is methodological. For many years, I took it on faith that regression on team statistics would give the same estimates as Pete Palmer got from his simulation; but when I read Albert and Bennett's book, I realized it wasn't true.
(2) It's certainly true that the equilibrium success percentages for win-probability teams jump around; but, as tangotiger notes, they stay in the range from the low 50s to the high 80s. A naive regression estimator such as the one in Albert and Bennett gives estimates that make no sense in the context of this. But, if we use this as a baseline for what the relative SB and CS coefficients must look like, we can use it to evaluate whether we've adequately instrumented for speed. There's tons of heterogeneity in this data, both in situations (as noted) and across players -- but the fact that most equilibrium success percentages cluster in the 60s and 70s means we still should -- and it appears we do -- get approximate aggregation.
So, I see the paper as being exactly the opposite of what Nick S got out of it: the relative values of Palmer's SB and CS weights (the original ones from the simulation) make sense if we bear in mind that basestealing is an elective play, and we can get the same result from "real" data if we control for quality of baserunners appropriately. The fact that passing from expected runs to winning percentage doesn't make a huge difference in strategy is one of the things that makes the regression work OK.
By the way, I'm interested in ideas for other instruments I might be able to use to get the SB3 and CS3 coefficients to make more sense; it'd really improve the paper. I haven't had any success yet, but am going to try various flavors of bases advanced on hits next. I wish I could use infield singles, but the Retrosheet data isn't good enough for that for most years. :(
Posted 12:50 p.m.,
November 7, 2003
(#4) -
tangotiger
Ted,
In the "How Runs Are Really Created" article, I show what the chances of scoring from each base, using Markov on the 1974-1990 data.
Here's the snip from that chart:
======================================================
Chance of scoring, from each base/out state
0 outs 1 out 2 outs
1B .38 .25 .12
2B .61 .41 .21
3B .86 .68 .29
======================================================
If we focus only on the "1 out" column, we see that a SB adds .16 runs from 1b to 2b, and .27 runs from 2b to 3b. I'll guess that errors might add another .02 runs or so to the SB event itself.
The CS figure can be similarly calculated. You lose the .25 runs from being on 1B (and another .20 runs for making the 2nd out... not listed in this table, but the breakdown is essentially .24/.20/.10 in terms of the cost of the out, when it's 0, 1, or 2 outs). So, the CS is worth about -.45 runs, if it happens at 2B.
If it happens at 3B, the CS is worth .41 + .20 = -.61 runs.
So, the breakeven of the SB at 2B (+.18 SB, -.45 CS) is 71%, and the breakeven of the SB at 3B (+.29 SB, -.61 CS) is 68%.
Also note that a CS does *not* always lead to an out, the many quirks in baseball recordkeeping.
Posted 12:59 p.m.,
November 7, 2003
(#5) -
tangotiger
btw, the old adage about "don't make the 1st or 3rd out at 3b" is dead-on. The breakeven in those cases jumps to 75 to 80%. Only really really really good baserunners should try to steal 3B, or if there's a really really bad catcher behind the plate.
Posted 1:02 p.m.,
November 7, 2003
(#6) -
Mike Emeigh(e-mail)
I recall (though can't cite) work (I think by MGL) that SB attempts are randomly distributed throughout game situations.
The last time I looked at this, it was my recollection that (a) more SB attempts occurred early in games rather than late in games and (b) it was usually the team that was ahead that attempted the SB. There are more data points available now, and maybe it's time to take another look.
-- MWE
Posted 2:10 p.m.,
November 7, 2003
(#7) -
tangotiger
I should amend my statement. SB are *not* randomly distributed by inning/score/base/out, but they *are* (more or less) distributed in such a way that the "leverage" of when they occur, on average, is 1.0 (a random situation).
What this suggests is that
a) SB are distributed such that it follows a normal distribution
b) SB are distributed such that alot more SB occur at the center
c) SB are distributed such that alot more SB occur at the tails
while all at the same time, distibuted around the average in all cases.
Since we really care about the leverage of the situation, and not necessarily the inning/score only, I'm happy to leave it at that.
Posted 4:16 p.m.,
November 7, 2003
(#8) -
FJM
Does anybody segregate true SB attempts from failed run-and-hit plays where the batter swings and misses? I'd expect they would have different distributions across the base/out states. Certainly the would have different success rates. When a runner is thrown out on a failed run-and-hit, shouldn't the batter be charged with the out?
Posted 4:25 p.m.,
November 7, 2003
(#9) -
tangotiger
In a play-by-play world, I would charge the batter with part of the out of the runner.
Posted 4:28 p.m.,
November 7, 2003
(#10) -
J Cross
So economic theory suggets that managers as a whole must be acting rationally? What if the decision of whether to steal was left up to the runner (to one extent or another)? Couldn't the runner rationally decided to steal more than regression coeffecients (assuming he's familier with Palmer's work) suggest he should in order to boost his numbers for his upcoming contract arbitration? Of course, GM's and arbitraters should be acting rationally (and with perfect information) too so this wouldn't help him. I'm assuming that economic theory doesn't suggest that ALL managers are making rational decisions (since some managers are more prone to steals than others, they can't all be acting rationally, right?) but rather that the irrational managers will either realize their mistakes or get fired. But what if the irrational managers tend to err in the same direction? Couldn't there be more (or fewer) steals than there should be?
Posted 4:39 p.m.,
November 7, 2003
(#11) -
tangotiger
I'm quite certain that Rickey stealing 130 and getting thrown out 42 (!!) times was not optimal. He probably could do 100/20, making the extra 32 SB and 22 CS a very bad thing.
Posted 6:37 p.m.,
November 7, 2003
(#12) -
FJM
In the regular season the Marlins stole 150 bases and were caught 74 (!!!) times. Then in the postseason they were 8-6. Did they steal too much?
Posted 8:55 a.m.,
November 10, 2003
(#13) -
Ted T
J. Cross: I think you have the science backwards here. The paper starts with a hypothesis that stolen base behavior is occurring according to some optimal scheme. There's a testable implication of that optimality: that the estimated net contribution of the marginal attempt is zero. The paper looks at the data, and finds that we cannot reject the hypothesis. That's a *much* weaker statement than *accepting* the hypothesis that there's rationality going on, which is what you ask. But it's the strongest statement science is able to make -- we can only reject or fail to reject hypotheses.
Of course, there are lots of other things that could go on and still give the result the paper has. The ideas you list are certainly relevant and could also well be going on. I thought about the agency question (stealing too much to pad numbers) early on in this project, but couldn't figure out a coherent model that had a testable implication. I still think it's an interesting area to look at.
But, overall, these non-optimalities, if they exist (and they probably do) couldn't be too systematic, or the resulting parameter estimates we get from the data wouldn't be what they are. So "on average" in some sense, the data suggest that overall basestealing behavior is likely not to be too suboptimal.
Posted 9:02 a.m.,
November 10, 2003
(#14) -
Ted T
tango - Are you basing your assertion on personal opinion or science?
The game-theory based paper I wrote a few months ago (which if I recall correctly you've seen, since I believe you had some comments for me) implies that the equilibrium attempt frequency goes up very rapidly for a marginal change in talent for the best basestealers. The differential equation looks like d(attempt frequency)/d(talent) is proportional to the square of attempt frequency, which means it's going up even faster than an exponential (the solution to the equation would be an exponential if it were the attempt frequency itself, not the square).
So theory expects that the handful of the best basestealers should steal noticeably more than the second-tier guys, which is what we observe in the data. In fact, the game in that paper does a pretty good job of predicting the cross-sectional distribution of both attempt frequency (very skewed) and success percentage (not skewed) in the population of players.
I'm not taking a position on Rickey per se, because figuring the "correct" steal frequency would require direct observation of "talent", which we don't observe. But both qualitatively and quantitatively, Rickey's numbers aren't out of whack with a plausible model of stolen base strategy for an elite baserunner.
Posted 10:21 a.m.,
November 10, 2003
(#15) -
tangotiger
Ted, actually, I don't think I ever read that paper, since I'm involved in similar research, and I didn't want to have any kind of conflict. I'm working on a model that shows what the optimal steal% should be given a given talent level. It's all theoretical, but, off-hand, I'd say it would be quite improbable that Rickey, in that season (1982?) was stealing optimally.
Posted 11:13 a.m.,
November 10, 2003
(#16) -
Ted T
tango -
The last public draft's on my website at
http://econweb.tamu.edu/turocy/papers/sbecon.htm
I'm redrafting a final version now. (I was in the process of working on it when I factored out the paper we're discussing here.) I don't think the version on the web has the comparative statics calculation I cite, but I can send that to your email if you'd like.