(Hi, Dr. Ivan!)
A while ago, our resident Carp (not Mike Carp, who managed to get DFAed between the time I initially wrote this line and I published this Fanpost...) put together a pretty cool fanpost that looked at how the Braves top prospects have fared, given the benefit of hindsight. It was super-informative, but the one thing I thought it was missing was context about the league as a whole. Knowing some odds about our prospects is certainly useful, but if there's something systematic going on that leads to certain teams getting better outcomes from their top prospects relative to other teams, looking at just one team may not give you the whole picture. So, armed with that little bit of intellectual curiosity, Baseball Cube's collection of Baseball America top 10 prospect lists, by team, going back to 1992, and the well-intentioned sacrifice of some leisure time, I embarked on a journey to see if I could put together something edifying, or at least mildly interesting.
This fanpost is the culmination of those efforts. Before I go into the details, I just want to say two things. First, there is a Juan Francisco-sized load of caveats about this whole thing. I will try to summarize them in the first comment post, so as to not to derail the flow. But, before you start commenting about, "Damnit, Ivan, you didn't consider X!" well, believe me -- I did consider X. I considered X a lot. (Probably.) So yes, I'm aware, but I didn't find that those caveats were a compelling reason not to do this at all. So I persevered. Second, as you could probably tell by now, I'm incredibly long-winded. My girlfriend calls me "conversationally anal-retentive" (hat tip to Sports Night). So, this post is hyper-trimmed down from everything I want to say into a digestible format. I have way more information than contained here, and am happy to have an ongoing conversation about this in the comments, if anyone else finds it as interesting as I do. If not, well, I don't know, this will scroll off the main page eventually, and then we can go back to reading 25 fanposts about Fredi's new collection of boudoir paintings featuring Jose Constanza, or something.
What I Did
My methodology was pretty simple. I went through, team-by-team, pulling down a unique list of top 10 prospects for all years available from the Baseball Cube, starting with 1992, and ending with 2010. I ended with 2010 because a lot of the young guys still haven't hit the majors, or haven't played much, if they have. These guys give me no (useful) data, so going beyond 2010 wasn't that useful--a lot of the prospects listed in 2009/2010 I also excluded, due to their age and professional inexperience thus far. I broke every list down into pitchers and hitters, and used an algorithm to score them as follows:
4 - Platoon guy / OOGY - Hitters: More than 100 career games, but always fewer than 600 PAs in a season (450 for a catcher) | Pitchers: At least 50 career games but without 60+ IP in multiple seasons Braves examples: Wilson Betemit, Jung Bong
5 - A cup of coffee, or something like it: Did not play more than 100 games as a hitter, or 50 games as a pitcher, in the big leagues. If never made the bigs, made it to the AAA level in a minor league organization (no foreign leagues). Braves examples: Christian Parra, Eric Campbell
6 - Bust - Never made it past AA in a minor league organization.
Now, I know there are a ton of problems with this. I chose to do it this way because of two competing ideas, one of which may be based on an admittedly false premise. First, I wanted to see which prospects at least somewhat fulfilled their potential via a few great seasons, and see if that differentiated itself from guys that were consistently solid. On the other hand, I wanted to capture a sense of "value" to the team, not in the purely-value statistical "wins" sense, but rather in the mileage the team was able to get in plugging the guys into the lineup/rotation/bullpen rather than having to fill those holes from elsewhere. Naturally, these are both imperfect measures. Teams that aggressively promote prospects and give them lots of playing time are going to look "better," when really they may just have less to play for, or can't afford to pay to get better performance at some positions. There's also a separate problem in that my ratings were relatively rigid -- EOF got a "4" under this system, for example. But for every rigidity causing weirdness in one direction, there was the reverse happening, or something close. So, overall, the ratings reflect what they more or less mean to -- if there's any fault in the methodology as it pertains to this, it has to do with my scores themselves, and not my assignation of them. (One final note: in retrospect, the distinction between the 3 and 4 categories did not work out so well across pitchers and hitters. Few hitters have a seriously full-time role, so the 3 for hitters is probably too stringent, and too many get 4s instead. On the other hand, it's easy for mop-up guys and swingmen to rack up 60+ IP in a season, so lots of pitchers get 3s when really they weren't really integral parts of their teams. Things to think about for Round 2 of this, I guess.)
After doing this, I basically just threw together some stats (well, a lot of stats, but not that many featured here).
One interesting thing I found was that the BA top 10 rankings for a team tended to change by about half with each passing season, on average. In other words, as an example, I had data on 18 years for most teams (including the Braves). Over those 18 years, the Braves had 96 unique players on their top 10 lists, or around 5.3 per year. This figure is pretty consistent around the league: the average is about 5.6. Only the Rays fell under 5 per year at 4.84, and five teams exceeded 6 per year (most turnover award goes to the A's, with 120 unique top 10 prospects in 18 seasons, or 6.67 per year). So, for lots of reasons (trades, graduation, poor performance, outpaced by new/non-ranked prospects, etc.), about half of a team's top 10 list doesn't stick around into the next season. Not unexpected, perhaps, but interesting information (to me, anyway).
These three basic pie charts show the breakdown of all prospects to appear on a top 10 list for any team from 1992 through 2010. Starting at the 12 o'clock position and going clockwise, the different colors represent the different categories -- the small blue is the MVP category, the red is the "bust" category, and so on.
Overall, the results are probably what you would expect. Somewhere between 10 and 15 percent of top 10 prospects become awesome for any period of time (first and second slices). About 40 percent become either useful regulars or role-players. The single biggest group tends to be AAAA-guys - over a third of all top 10 prospects don't make any kind of real contribution at the major league level. Lastly, about a tenth of all top 10 prospects tend to bust and not even make it to AAA. So, your top 10 prospect, on average, has about even odds of having an All-Star/MVP season in the bigs, or failing to to play in the International League or PCL.
As the pies (mmm... pie) show, there's not all too much difference between pitchers and hitters, but hitting prospects tend to have better outcomes, however slightly. Pitchers are more likely to become AAAA guys or bust outright, where hitters are more likely to have an All-Star season or two. (Note: A basic t-test on the means of pitchers and hitters indicates no real difference between the groups, but isn't really appropriate given that the rankings are discrete, rather than continuous groups. I could do t-tests on the probability of being in a certain group, set of groups, though, but haven't yet.) As I mentioned earlier, the differences between the "3" and "4" groups across pitchers and hitters are more artifacts of the algorithm I used than anything else, so I wouldn't read too much into those, especially not that pitching prospects tend to become solid contributors at higher rates than batting prospects, or anything like that.
Braves vs. the World
Of course, since this is a Braves blog, we probably care more about our own guys, at least a bit. So, here's a couple of sets of those same pie charts, this time for the Braves, and then for the rest of the majors rolled up as one entity, excluding the Braves.
And now the rest of the league (RoL):
A few interesting things are going on here. In terms of pitchers, the Braves have had better success with getting MVP or All-Star performances out of their top pitching prospects. However, their "solid contributor" category falls pretty short of that reached by the rest of the league's prospects, and this is actually enough of a shortfall to more than make up for the gains from the first two categories (on a pure quantity basis). This might be because the Braves have tended to have solid pitching staffs more often than not over the last two decades, precluding the chance for many pitching prospects to have significant impacts. The remaining categories are more or less in line with the rest of the league -- a little lower rate of busts and AAAA guys, but not by much. (Again, no detailed statistical tests run... yet.)
One thing that the Braves have done really well at, given these charts, is getting All-Star seasons out of their former top prospect hitters. Of the 41 top 10 Braves prospects that happened to be hitters I examined, nine (~22 percent) had at least one All-Star season (and two more, the Jones boys, had MVP-type seasons). This rate is considerably better than the rest of the league. The Braves also reaped considerably fewer AAAA hitters than the league as a whole; however, this is balanced by the fact that their top prospect hitters busted at a rate greater than in the rest of the league.
Overall, the Braves prospects have been pretty good at paying dividends on the top end, relative to the rest of the league, and on the strength of those contributions, have a greater proportion of top prospects that made at least some impact at the major league level (first four slices on the pie chart). The Braves did experience prospects busting at a slightly higher rate, but it isn't really noticeable in the grand scheme of things, as the difference is pretty small.
To wrap this up, I put together a table that might be of some interest (click to pop, it's hard to read).
The table is pretty self-explanatory. The categories up to are cumulative, so it's not just the players in a given category, but in that category and all categories above it. So having the "worst" value in the coffee+ category is basically like saying you experienced the most busts, because that's the only category remaining (lol Phillies hitting prospects).
I added averages and medians just in case anyone was curious. The "all players" row is the true average, whereas the average and median rows do not account for the fact that some players have hopped teams while remaining top prospects. The "all players" row actually does the accounting on a player level rather than an average of the percentages of the 30 teams, but as you can see, the difference is very small and not relevant.
The Braves row is shaded green where the Braves are in the top 25% of the league based on their percentage for that column, and shaded red where they are in the bottom 25% of the league. These conclusions aren't any different than those I noted a few paragraphs earlier, but just displayed in a different way.
Thanks for reading this far and not posting a snarky tl;dr comment in the thread!
Have questions? Comments? Things you want to see me do with this data? Inappropriate limericks about Dan Uggla? As you all know, I spend way too much time on TC as it is, so I'll troll around the thread for a while answering any flights of fancy any of you have, if you're interested. The data is very much at my fingertips.
As for my own next steps, I'd like to do the following:
- Create a better rating system to get more accurate pie slices. I'm interested in potentially crowdsourcing it. If you don't suck at web stuff and want to throw this up somewhere, I think that would be really neat.
- Create a separate rating system based on WAR/season or a similar measure. I'm interested in what the results of that are, but it's a lot of work in order to find something that works.
- Give Freddie Freeman a hug.
So, as I noted elsewhere, the caveats are below, because I don't want to tack a bunch of additional stuff onto this post.