BP's new streamlined projections

6 years ago I discovered Baseball Prospectus: the quintessential baseball projection publication. Every March since has been an exciting time for me. I can't wait to see who's on the cover, what rookies are going to break out, which vets are going to fall off a cliff, how the top 100 prospect list has changed from last year, and of course, what is to be said about my boys on the South Side. This year's edition, however, is so different from previous years that I feel compelled to write a FanPost.

Now, I don't think there will ever be a perfect measure in this game. If you ask statisticians they will probably tell you the same thing. There are simply too many variables and too much chance. Physics plays such a large role in the game that the slightest change in atmospheric conditions can have dramatic effects on outcomes. It's for this reason that you will see stats from half a dozen corporations and thinkers that attempt to measure the same thing. And they all think their method is the best.

So, this year when I opened my BP and read the statistical introduction to refresh my memory on the lingo, I wasn't surprised to read about them touting their "Deadly Accurate PECOTA Projections". I was, however, surprised to learn that they combined many of their key stats into single, simpler measures, and completely changed how they evaluate defense. I turned to the players and saw a completely re-done profile. For hitters, at least VORP and EqA were now gone. Something new, FRAA was there. For pitchers, Stuff score was gone and EqERA was replaced by FRA. There are probably even more differences than I'm not picking up on right this moment. Everything appears streamlined: there's less clutter and less overlap; less fat and more muscle. I'm honestly not sure how I feel about this. Perhaps some of you can enlighten me with your thoughts as to why this was necessary and  explain to me how their new defensive measure, FRAA, works.

Here's a breakdown of what they've done with defense from Colin Wyers at BP: (with my comments in brackets)

The general trend in the sabermetric community has been toward stats based on zone data, where human stringers [who?] record the type of batted ball (grounder, liner, fly ball) and its presumed landing location... Different data providers can come to very different conclusions about the same events, depending on the provider's recording practices and vantage points... [our old pal OC would agree with you on this] ... For example, we've seen that the quality of the fielder can bias the data. Zone-based fielding metrics will tend to attribute more expected outs to good fielders than to bad fielders, irrespective [what an archaic word to use] of batted-ball distribution. Scorers who work in parks with high press boxes will tend to score more line drives than scorers who work in parks with low press boxes, and so on. [Okay, now this makes some sense to me. The rest, however, is murkier:]

Because of the secrecy surrounding the underlying data, we've barely begun to scratch the surface of quantifying these problems and their effects. Consequently, we  have abandoned our efforts to produce our own zone-based metric for inclusion in this book. Simply put, no evidence shows that the inclusion of zone-based data improves defensive metrics over the short run, and much evidence shows that incorporating the data causes severe distortions over the long run. [What the Hell does that mean?]

Instead, we've revised our Fielding Runs Above Average (FRAA) to incorporate play-by-play data, which allows us to study defense at a much more granular level without resorting to the subjective data used in some other fielding metrics. [So you no longer go by what the human scorers say?] Simply put, we count how many plays each player made, as well as the expected number of plays for the average player at each position, based upon the pitcher's estimated ground-ball tendencies and the handedness of the batter. [Ground-ball tendencies? Handedness? Have they always used these?] There are also adjustments for park and the base-out situations: depending on whether there are runners on base and the number of outs, the shortstop may position himself differently, and we account for that in the average baselines. [I'd like to see the math on this]

[Following this is a chart showing our (other) favorite Gold Glove SS's defensive metrics from 2003-2010 as reported by DRS (Defensive Runs Saved), UZR (Ultimate Zone Rating), TZ (Total Zone), the old FRAA, and the new FRAA]

[I can't/won't reproduce this chart, but I will give you the totals. DRS: -114, UZR: -43.4, TZ: -42, Old FRAA: -91, New FRAA: -196]. -196!!! How does UZR and TZ only give -40ish and their new system gives -196? That's a difference in 155 runs over 8 seasons. That's -15.5 wins, or roughly -2 wins per year! Holy shit that's different!]

Because of a phenomenon called range bias, the "advanced" zone metrics [did he really have to put "advanced in quotes"?] all severely underestimate Jeter's inability to make plays on ground balls: if you simply look at the distribution of ground-ball location measured by any data provider, you can see clustering around the primary fielding positions. If we look at objectively collected data on batted-ball distribution, such as that offered by Sportvision's HITf/x system, which uses the same methods employed by the company's PITCHf/x system to track the location of a pitched ball within approximately an inch, we see a very different distribution, which is mostly smooth. This is explained by the tendency for an observer (absent better reference points) to use the location of the fielder as a guide to where the ball was hit. [so a fielder's instincts, manager preferences, and fielding shift patterns that determine where a fielder should stand influence your data and don't influence theirs? I'm confused]

As you can see, we've made a clean break from much of the rest of the sabermetric community on defense... [yes, yes you have]

So, you can see that this new defensive measure has me in a tizzy. To further show how drastic the change is, I give you some White Sox FRAA scores from last year:

• Alexei Ramirez: +21 for a total WARP of 5.3! (FYI: Alexei was -7 based on this same measure in 2009, a 28 run difference!)
• Juan Pierre: +6
• Brent Morel: -1
• Carlos Quentin: -10
• Paul Konerko: -6
• Andruw Jones -2 RF, -1 CF
• Ramon Castro: 0
• Gordon Beckham: -1
• A.J. Pierzynski: -2
• Mark Teahen: 0 3B, -2 RF
• Alex Rios: +4 CF, +1 LF
• Dayan Viciedo: -3 3B
• Omar Vizquel: -6 3B, -3 2B

Now... the fact that Mark Teahen was 6 runs better at 3B than Omar has me questioning this system. I didn't watch many games, but my impression from you guys was that Omar was pretty decent. At worst, he was average.

Their WARP system also is a little head scratching. As I mentioned earlier, TAv, which plays into WARP, is now modified. Strikeouts are more damaging than other kinds of outs, and sacrifice bunts are a bit less. They also removed BRR (baserunning runs) from TAv and these are now incorporated into a player's WARP. Neat-o. Now, why wasn't it before?

Some final observations. Carlos Quentin racked up 1.4 WARP last season. Tyler Flowers is projected for 2.1 WARP. Compare that to Q's Fangraph's WAR (0.0) and the Marcel/Bill James projections for the Flowers. It's quite the difference. What are your thoughts and observations?