DIPS

At the very minimum, we should be looking BABIP-IFFB instead of straight BABIP.

I don't think anyone can argue with that. But the variability of BABIP can either be luck (ball getting hit right at players) or pitching better (weak contact) than normal.

Simply dismissing great starts because of FIP, SIERRA and luck is not valid IMO.  In the minors, that's scouting the box score.

Maybe this is just semantic (what exactly constitutes "simply dismissing?"), but I wholeheartedly disagree. You should absolutely use peripherals to differentiate between performances, and if you're just looking at surface stats like hits, runs, ERA, OPS, etc., you're going to make a lot of poor evaluations. A guy who gives up three runs but has a 10:0 K/BB absolutely had a better start than a guy who have up two hits and no runs but has a 2:2 K/BB. Especially in the high minors, you can draw very valuable conclusions based on stats (especially peripherals), including conclusions that scouts may miss. This is particularly true for a forum like this one, because most of us do not have the opportunity or training to scout minor league games, but all of us can learn to analyze stat lines and make accurate predictions based on that analysis.

(Also, it's SIERA. One R.)

A couple of notes.

Since Voros McCracken published his original article DIPS theory is probably the most misused theory on the internet. Not only do several so called "sabermatricians", misapply the research, but McCracken's research wouldn't have passed the strict scruitiny required in other fields. McCracken used a low Rsquared and only used two seasons of research.

First off the idea of DIPS theory makes absolutely no sense. A batted ball in play is by far the most common end to a plate appearance by far. I would estimate that roughly 2/3 of plate appearances end with a ball in play. To suggest that the speed, location, and movement of the pitch thrown has little or no influence on the ball landing safely is somewhat silly. In the article that jimei posted it was estimated that 30% of a the variance of BABIP could explained by factors influenced by the pitcher. Considering that it's around 65% of outcomes that's a lot. For those math geeks out there, because n is large, the standard deviations are going to be smaller, much smaller.

Second there are significant limitations on the research.

All the research suggested was that established major league starting pitchers had little influence on BABIP over an extended period of time. Any other analysis extending this research is a complete misuse of the ideas presented. You cannot use the theory to analyze a minor league prospect or to dismiss a single start.

Players who are much better than their competition are clearly able to influence BABIP. Even in AAA the competition is far more diffuse than it is in the majors. A AAA pitcher who is really doesn't belong there might be able to get minor league hitters to make consistently bad contact. Can you imagine a high school team making consistently bad contact against Felix Hernadez and some guy in the stands declaring that the weak contact was just bad luck? It's not.

A pitcher's influence on BABIP over long periods says nothing about his influence over short periods. Over one start the pitcher may just be facing a team swinging away early in the count. Or a pitcher may have exceptionally good movement that start. I wouldn't be surprised if the real reason that pitcher BABIP shows less constancy than K's or BB's is that BABIP is influenced by movement, which there is some evidence for, while K's are influenced more by velocity and BB's by control. It would make sense that a pitcher would have more ability to be consistent with velocity and control than movement. You would need to, analyze how hard the ball was hit, location etc. as opposed to scouting a box score.

Finally it should be mentioned that most statistical studies are influenced by survival bias. I remember reading a post once that suggested that Zach Stewart was actually a strong pitcher because he had a much lower XFIP than ERA. Simply put a pitcher who has a substantially higher HR/FB ratio or BABIP than average won't be on the mound very long and won't be part of the study. It's likely that Stewart simply wasn't good enough for the majors and that was the reason for the variance.

I really don't know why I posted this as it's not going to stop people from polluting the internet by misusing research that they do not understand. Some people love to scout the box score and pretend that they are smart. I am sure no one cares.

Inducing weak contact can absolutely be a skill. But before we go around accusing everyone who disagrees with us of "scouting the box score," can we at least provide evidence that Ranaudo is actually doing that? Otherwise everyone is just looking at the same box scores with a different set of biases.

Over 5-10 starts, peripherals are certainly a more reliable indicator of anything than runs and hits allowed. If you've been doing it as long as Matt Cain or Mark Buehrle or Tom Glavine, sure. But ten starts of Ranaudo outpitching his peripherals doesn't mean much. The more important thing is that he's producing a lot of grounders, which is going to keep the ball in the park, which will keep his ERA down whenever his BABIP regresses to whatever his mean is. Because it's not .202.

I noted repeatedly that he had pitched really well over those five starts. My point was just (a) that he pitched worse than his ERA would suggest (i.e., advanced metrics regarded him as having pitched more good than spectacular), (b) he still had things to work on in Pawtucket and was not major league ready, and (c) that I still liked Webster and Barnes more than him.

Part of the problem is that we use the word "luck" in different ways. I don't mean to suggest that he's actually a bad pitcher who just fluked into some good results. I mean that he pitched really well in that stretch but we shouldn't expect him to continue to pitch that well going forward. There are no value judgments there; rather, just the suggestion that that stretch probably doesn't represent his true talent level.

I'm generally suspicious anytime someone points to a small sample of great results and suggests that that prospect has turned a corner and is blossoming into the guy we always thought he was. Having followed the system for eight or nine years now, I can tell you that (a) folks do this all the time and (b) the vast, vast majority of the time, it's just a little blip in the radar rather than a breakout. You're right to suggest that it's certainly a possibility that this is Ranaudo reaching his potential, but the odds are very much against it, and I tend to think that we should avoid getting too worked up about that possibility until he sustains it over a longer period of time.

Re: Owens-- I think we agree more than we disagree. I think Owens' deception is a legitimate skill, as is his ability to get a good number of popups, and he will likely be able to outperform his peripherals. But the magnitude of that skill is very large. The best BABIP-outperformers can sustain something like a .280-.290 BABIP over the long haul, which means that they'll put up a 3.10 ERA as compared to their 3.30 FIP. Even if outperforming your BABIP is a skill in the minor leagues, it's one that translates less well to the majors and has a pretty narrow spread there. As such, I don't think we can just ignore his minor league FIP, as he's not going to be able to outperform it as much in Boston as he has in Portland and Salem.

SIERA does, FIP doesn't. If you have questions about these metrics, I would just google around and read more about them. You're going to encounter them a lot, and it's worth knowing how exactly they work.

Just wanted to briefly elaborate how on I evaluate small sample data for pitchers. I generally hold a rebuttable presumption that the high variance stats (think BABIP, LOB%, etc.) are driven by chance rather than skill, especially when we're talking small samples (if it's a really small sample, I add HR/9 to the list). As such, I think that ERA estimators (FIP, SIERA, etc.) are a much better estimate of a player's future performance in small samples than their ERA is.

If the player possesses skills that suggest that they have a true-talent ability to outperform a league-average BABIP (think deception, a high IFFB%, etc.), those ERA estimators might slightly underrate them, but the effect tends to be pretty small. Therefore, I still think their ERA estimators are a better estimate of their future performance than their ERA is, though their true-talent levels are likely a little better than those ERA estimators suggest.

(I generally follow the same process with hitter BABIP, although because hitter BABIP is much, much more of a skill than pitcher BABIP, I mentally regress less with hitters than I would with pitchers. This is especially true if the player has either the skills (elite batspeed, speed, etc.) to suggest the ability to maintain above-average BABIPs or has shown the ability to maintain above-average BABIPs over the course of a large sample in the minors.)

I do the above because it's just playing the odds. Study after study indicates that BABIP is highly variable and more often driven by chance than skill, especially in small samples. When it is driven by skill, we can roughly identify most of the factors that drive it, so we can identify those guys who might be able to sustainably outperform a league-average BABIP. As such, when those factors aren't present, we should regress BABIP (and HR numbers in small samples) to league-average marks when evaluating a pitcher. When those factors are present, we should still regress a clearly-out-of-line BABIP to what we estimate that pitcher's true-talent BABIP might be.

Here's an illustration. It's like knowing that nine out of ten coins are 50/50 odds, but one out of ten is 60/40 odds. Someone presents us with a coin of unknown providence and flips it four times, getting four heads in a row. Our reaction should be that it's probably a fair coin that just happened to get four heads in a row due to chance. Our reaction should not be that maybe it's a coin that flips heads every single time.

I understand these these statements are descriptively different, but practically speaking, because the BABIP spread amongst major leaguers is so narrow and determining true-talent BABIP skill is so difficult, I tend to think that we can regard the first statement as essentially accurate and thus can and should heavy-handily apply DIPS principles in evaluating pitchers.

It's like how classical physics is, strictly speaking, not entirely accurate (see quantum mechanics and relativity), but for for the purposes of teaching 11th graders basic science, it's accurate enough. Yes, in reality, it's more complicated than F=ma, but for our back-of-the-envelope internet discussion, it's close enough.

Did you read this link? The smaller the sample size, the more that you want to look at a guy's peripherals as opposed to his ERA. Remember, this isn't a dichotomy between having a scout at the game giving us a detailed report versus just looking at his FIP. There's that somewhat famous quote about how, in small samples, a good scout is always better than stats, which I absolutely agree with. In general, a good scouting report will be way more valuable for a minor league prospect than any statistical analysis, for many of the reasons discussed above.

But here, we're talking about the difference between looking at a guy's box score stats or looking at his peripherals. In general, few of us have the ability to see many minor league games, and the TV feeds are generally of poor quality (blurry, from a bad angle, don't show batted ball trajectory, etc.). Even fewer of us has any scouting background. In light of that, we're forced to predominantly look at stats and secondhand scouting reports to evaluate prospects. And if we're going to cite numbers to evaluate a player, there are certain ones that are more predictive than others, and we should be using the more predictive ones.

I mentioned this earlier-- we're using the word "luck" to mean different things. I'm not trying to take anything away from someone who had an great performance-- no-hitters and perfect games (and, to a lesser extent, shutouts or big wins, etc.) are awesome and worth celebrating. But they're also unsustainable in the sense that that guy is probably not going to continue to pitch that well, and that's the question I'm almost always intent on answering. I'm usually looking forwards rather than backwards, and for those purposes, looking at ERA estimators rather than ERA generally makes more sense.

But, more broadly, just because a guy gave up a bunch of shallow fly balls does not necessarily mean that he pitched well. The difference between a shallow fly ball and a home run is maybe a quarter of an inch and a fraction of a second. There just aren't many pitchers who are good enough at tricking major league hitters into consistently mis-hitting their pitches (unless they're getting them to swing-and-miss).

But "everyone still at BP" has since come to the conclusion that SIERA is actually completely awful at anything anyone could possibly want, right?