clock menu more-arrow no yes

Filed under:

Where do good starting pitchers actually come from?

New, 40 comments

What if, instead of looking at the average value of a draft pick, we applied a magnifying glass to the set of major league starters we already know and love?

Sean Newcomb's personalized glove, and apparently non-personalized shoes.
Sean Newcomb's personalized glove, and apparently non-personalized shoes.
Reinhold Matay-USA TODAY Sports

Note: This post will seem long and may take a bit of time to load, but it's actually not much longer than many others posts I've made. It may just seem that way since I'm embedding some large interactive charts.

The rebuilding Braves regime has made it no secret that they covet pitching prospects. The plethora of present value-for-future value trades made as part of the rebuilding effort targeted high-upside arms as the centerpieces of the returns, and the 2015 amateur draft (this regime's first) applied a pinpoint focus to high school hurlers, including first round picks Kolby Allard and Mike Soroka.

There's a lot of pontificating and blowharding that can be done to assess, with no evidence or backing, the merits of this strategy. In reality, we won't know how well it turns out until after the Braves are in the post-rebuild stage of hopefully competing for playoff berths, and not even then (because if the forecast says there's a 90 percent chance of rain, and it's a sunny day, it does not mean the forecast was wrong, just that the great Die Roll of Nature rolled in the 10 percent for that event). But, that does not mean we should not attempt to gain some insight into the process.

There have been a lot of studies about the values of draft picks, the success rates of various prospect types, and so on. I'm happy to discuss those in the comments, and there are some folks in the community who know even more about this than I do. That's been done already, though, and I think most of us already understand that prospects (especially high school pitching prospects) are very risky, and that the average value of a draft pick is not all that high in terms of major league production after adjusting for this risk, no matter how exciting his peripherals in Low-A are.

Instead, I wanted to come at the question from a different angle with (hopefully) equal salience for building a good, competitive major league team: sure, you've got a boatload of pitching prospects, but where do good starters actually come from?

Approach

In order to have a relative taut, current sample, I used only the 2013-5 seasons, and culled from the list of all starters during those years only those met one or more of the following criteria:

  • Single-season FIP-based WAR (fWAR) of 3.0 or more in any of 2013, 2014, or 2015;
  • Single-season RA9-based WAR (rWAR) of 3.0 or more in any of 2013, 2014, or 2015;
  • Average season fWAR, 2013-5, of 2.0 or more; or
  • Average season rWAR, 2013-5, of 2.0 or more.

What this does is essentially highlight any starter with an above average season in those years, as well as any starter who managed an average performance over that stretch. The end result is 106 pitchers, out of a total of 457 that made any starts in that period, or a bit under a quarter of all eligible pitchers. So, in some ways, we're talking about a high echelon of hurlers, but really, it's pretty much everyone that's "pretty good" and above. After all, a large chunk of the 457 are cup-of-coffee or replacement-level starters, and no one is really interested in those anyway.

The set of 106 pitchers has many of its own warts, driven by the non-handpicked nature of how it was assembled. For example, Braves fans should be familiar with both Eric Stults (3.2 fWAR in 2013) and Bud Norris (3.1 rWAR, 2014) as two guys that have been anything but acceptable or gladdening in a Braves uniform, but are still included in this list of "pretty good starting pitchers." Other questionable names on the list are guys like Ivan Nova (kind of a generic innings-eater who hasn't stayed healthy at this point) and Miguel Gonzalez (who was released by the Orioles in 2016), guys who largely made it onto the list through one good run prevention (rWAR > 3) season and nothing else. Still, about 80 percent of the sample are guys who have accumulated 4+ fWAR in three seasons, and well over half has put up 6+ fWAR in that stretch.

For these pitchers, I was largely interested in how they were acquired, and their pedigrees. In other words, were these pitchers expected to be good before making it in the majors, or did they come out of nowhere? Were they high draft picks? Or, were they mostly stalwart and savvy veterans on the open market? Specifically, I looked at:

Acquisition Status

  • Free agency - was the pitcher ever signed as a free agent in this span of time?
  • Extension - was the pitcher under contract through an extension that kept him signed with a team beyond his team control years (essentially delaying him hitting the free agent market)?
  • Draft / team control - was the pitcher drafted and still under his original team control years?
  • IFA / team control - was the pitcher signed as an International Free Agent and still under his original team control years?
  • Trade - was the pitcher traded in this span of time?
  • Waivers - was the pitcher acquired via waiver claim in this span of time?

Note that these statuses are in no way mutually exclusive, both because the period spans multiple years, but also because many pitchers are traded during their team control years. About half of the pitchers in the sample were "single-status" (that is, only FA signings, or only draft picks under team control, with no change in status or trades of team-controlled players over the period), the rest were traded or otherwise changed status multiple times. (For example, someone like Cole Hamels entered the period of analysis as team-controlled amateur draft pick, but played through an extension, and was also traded.)

Pedigree

  • In what round was the pitcher drafted?
  • Did the player have an overall Top 100 ranking from Baseball America? If so, what was it?
  • Did the player have an overall Top 10 team ranking from Baseball America? If so, what was it?

These data are somewhat more limited, due to a dearth of historical data on prospect expectations. They also don't apply to each pitcher - IFA signings are not drafted, and some players don't get ranked, especially foreign imports like Masahiro Tanaka and Wei-Yin Chen. (Tanaka actually has a weird errata case I've never seen before: he was ranked the #4 overall prospect by BA prior to coming Stateside, but as he wasn't with a team, he did not have a Top 10 team ranking - the only case where this occurred.)

Results (the fun stuff)

First, let's just take a big overview of who we've got in here. This is basically the entire data dump, just oriented by pitcher age. You can mouse over any block to see the pitcher represented.

Note: Since I'm using multi-season stretches here, the age displayed is the pitcher's age in 2014, the middle year of the set. Same goes for any references to age below.

The big takeaway here probably isn't very interesting. Most of the good pitchers in a given stretch are around 24-30, and there are a lot fewer that are much younger or much older, though there is a fairly long tail of older quality guys hanging around. About 80 percent of the good pitchers in this array were around 30 or younger; about a quarter are right in that age 26-27 sweet spot. Takeaways for the Braves? Well, younger pitchers make up more of the productive pitcher population, though we mostly knew this already.

So, now you're thinking, but are the younger pitchers in this group better? And, well, not really. See below.

It's basically a hodge-podge, and Clayton Kershaw rules this particular roost. There's mostly just a giant clump, performance-wise, in the center, and no real pattern. So, most good pitchers are on the younger side, but within the group of good pitchers, age does not really predict quality.

So let's talk about "pedigree." Just about three quarters of these pitchers were acquired through the amateur draft, rather than signed as international free agents. Setting aside the IFAs for now, and, for simplicity, ignoring the documented large gulfs in expected performance across pitchers selected in different parts of the first round, here's what we have as far as the breakdown of these pitchers by draft round:

This is a fairly self-explanatory thing, I think. About 45 percent of all good pitchers, 2013-5, were taken in the first round. (Again, there's some distinction here between compensation picks and "natural" first round picks, but it's obscured here.) In total, one third of all good pitchers were first round picks. That makes first round picks pretty important, especially given that the individual other rounds don't really contribute much at all to the good starter population. The second round is at least somewhat of a contributor, but after that, don't expect much.

You can see a few interesting things if you array value against draft round. The better crop of drafted pitchers don't tend to come after around the fourth round. But at the same time, the sheer number of first round pick pitchers making up the array of good starters makes that particular group include both the weaker and stronger exemplars of the set. In other words, first round picks are both Ian Kennedys and Chris Sales, while later round picks are a little more constrained in that they miss the highest highs of the first round. Which is totally intuitive, really, even if a Mark Buehrle occasionally shows up to be the exception that vaguely validates the rule.

You might be wondering at this point, how about pitchers acquired through international free agency? You can see that below, same charts, but the IFA-signed pitchers (both Latin American and from Asia) are in the leftmost column.

As you can mostly eyeball (and I confirmed by running some statistical tests -- in fact, fWAR between the IFA-signed pitchers in this sample and the pitchers drafted in rounds two through ten are pretty much identical), the IFA signed pitchers who made it to the majors and ended up being good starters kind of look like their counterparts that were drafted after the first round. There's a lot of variation there too, and one of the big takeaways for me is that once a pitcher has ascended to the "pretty good" tier, pedigree considerations don't matter as much.

But, pedigree matter in terms of representing the array of good starters at a given point, at least for 2013-5. In simple terms, if you're looking for a solid pitcher and all you know is the round he was drafted in, having that first round pick status goes a decent way towards giving you some information about how likely he is to be at least serviceable. You can't say the same for pitchers drafted in the other rounds. So, given the rebuilding strategy of hoarding a bunch of first-round pick pitchers, this is at least somewhat of a hopeful finding. Of the Braves array of young arms that are going to be relied upon, only Matt Wisler, Chris Ellis, and John Gant were non-first round picks.

Another way to look at pedigree is by prospect ranking, rather than draft round. The two are generally not dissimilar, but prospect rankings at least incorporate some information on how players perform in the minors. The only historical source of rankings I'm aware of is from Baseball America, and includes their Top 100 MLB and their Top 10 team-specific lists. For these rankings, I used the highest rank a prospect ever achieved, which is not a perfect measure, but one that I think best reflects one-time expectations for each pitcher.

It may or may not surprise you to learn that over 60 percent of average-or-better starting pitchers in the period of analysis were at one point Top 100 prospects from BA. About 44 percent were Top 50 prospects, and 18 (or about 17 percent) were Top 10 prospects at some point. Similarly, over 80 percent of the pitchers we're looking at were n their own team's Top 10 list at some point. Once again, I didn't see much evidence that these rankings are predictive of how good a pitcher is once he's in this group, just that the expectations (and the ability and tools those expectations and rankings are based on) matter.

I really like the following chart set of charts, even if they don't say anything super-interesting. Basically, left-to-right you have first the Top 100 rank, and then if you scroll to the right, where you see "NA," those are the guys that never broke the Top 100 and their team ranks. For guys with both Top 100 and Top 10 team ranks, the team rank is represented at the bottom. The rightmost column with a bunch of circles are the guys never ranked by BA on either the Top 100 or Top 10 team lists. As you can see, there's no real pattern here, or at least not one that really jumps out. If you wanted to talk about average fWAR, you can maybe get a breakdown like:

  • Top 25 MLB ranking: 3.4 average fWAR
  • 25 through 100 MLB ranking: 2.5 average fWAR
  • On team Top 10 list, but not on Top 100 MLB: 2.1 average fWAR
  • Not on either Top 100 MLB, or Top 10 team list: 2.4 average fWAR
  • So, rankings may have some effect, but the set of guys who really come out of nowhere are pretty variable, and on average, have been as successful as all but the highest-ranked MLB pitching prospects. Rankings are good, better rankings are better, and most of the good pitchers are forecasted to be such, both by teams in making draft day decisions, and by the prospect evaluators in assigning rankings. But for the small but sizable bunch of good pitchers riding a dark horse to success, the lack of hype does not presage any kind of diminished effectiveness. Still, if you were going to gamble on Pitcher A versus Pitcher B, the higher-ranked ones do make up a greater share of the overall pool of good pitchers.

    So by this point, I guess what I've learned is that hoarding high-pedigreed pitching prospects is pretty good, because those prospects make up a really substantial chunk of the overall set of decent-to-good starting pitchers out there. This isn't a comment on the riskiness of any individual prospect, nor on the value proposition of this as a strategy versus something else, like hoarding pitching prospects. But, if the Braves did want to field a homegrown, cheap rotation in the future, they're probably going about it the right way.

    As I said earlier, my original interest here was seeing how teams actually build decent pitching staffs. Where do those players come from? Using the charts above, we have a better idea of what kind of pitchers develop into these useful starters, and when they do so. But how do teams go about getting them in their organizations? After all, there are 30 teams, and using a standard five-pitcher rotation, that's 150 pitcher slots per year. Across the three-year sample, I was only able to find about 110 average-or-better pitchers and some of those only had one acceptable season (see Eric Stults). To that end, let's take a look at the acquisition data. Again, due to the multi-season nature of this exercise, as well as the complexities in MLB transactions, it's not as straightforward as a single "how acquired?" value for each pitcher. Instead, we have something that doesn't add to 100%, as follows:

  • 58% of the pitchers in the sample were drafted and under team control for some/all of the sample period.
  • Another 8% were signed as IFAs (and under team control, where relevant) for some/all of the sample period.
  • 4% were acquired through waivers. Not very high, but not zero percent, either.
  • 10% were signed to extensions covering would-be free agent seasons. (Half of these were players who fell into the sample during team control years, and then continued to play for the same team via an extension for one or more additional sample years.)
  • 36% were signed as domestic free agents at some point in the period. On a rough basis, you could figure about a 60/40 breakdown between "homegrown" and "mercenary" quality pitchers.
  • Lastly, and perhaps critically, 46% of these pitchers were traded (and played for their acquiring team prior to free agency).
  • There are more than a few ways to read this information. First, good pitchers are definitely available in free agency, if you have the ability to pony up the necessary funds to acquire their services. If you don't, these types of pitchers are highly traded commodities. I suppose one not particularly interesting thing to say is that good starting pitching is a reasonably well functioning market, in that there are definitely ways to get it if you need it. The avenues all have risk (winner's curse for free agents, prospect risk for prospects), but there's no hyper-dominant trend for homegrown players only. (Only about a fifth of the pitchers in the sample were still in their team control years with their original team for the entire sample.)

    This information does not dovetail very neatly with the strategy the Braves are pursuing. While these data don't say anything about the overall cost-effectiveness or team effectiveness of the pitchers-first prospect acquisition strategy (we'd have a sample size of one, even if any data could say that), they do show that there's no overall impetus to rely on purely homegrown, young arms. Again, doing so may be a really good strategy in the end, but it's not what teams have done to date to field strong staffs. Anyway, rather than reading me blather, just check out this cool image (which does not include waiver-claimed pitchers because the diagram would get overly nasty otherwise with six categories).

    But, here's something else that's interesting. (Sorry, no Tableau this time. If you want a specific array of players by acquisition type in Tableau, let me know, but it was tricky to incorporate the different statuses together.)

    It's not a large sample. There are overlaps within the categories. The margin of error for WAR isn't that precise, and these aren't huge differences. But still, there's something interestingly suggestive there, even if it's not definitive in any way. Pitchers either still under team control frm being drafted, or under extensions buying out free agent seasons, tend to be more productive than their counterparts in free agency. This may be a function of age, akin to what was in the very beginning of this post. And again, the differences are not very large. But maybe there is something to the whole "draft picks over free agents" thing, even beyond the monetary angle.

    That about wraps it up for what I have right now. Feedback on early versions of this really emphasized that I should have included high school vs. college breakdowns for draft picks, and I can definitely just glom that on to the data I already have, but I didn't want to collect more data on these pitchers for this iteration before getting this out there. I can follow up with that at a later point and see if there's anything there.

    Since I know this was long, here's a basic summary of what I learned:

  • Better starters tend to be between the ages of 25-31ish, with a focus around age 27. But there's no easy relationship between age and production for starters: while most good starters tend to be under-30 (but not too young), in the pool of good starters, age isn't particularly predictive of success.
  • A strong majority of good starters in the majors were first round picks, and considerably fewer were later-round picks or international free agents. There's a little bit of a tendency for draft round to influence production among the group of good starters, but not much. There's a wide range of outcomes for even first-round picks, and as you get down in the rounds, the effective production ceiling tends to diminish.
  • Non-drafted players, provided they are able to reach the bucket of "good" starters, have similar ranges of outcomes to high-round picks, and generally look like players picked around rounds two through ten, as far as production.
  • Only about a fifth of all good starters come completely out of nowhere (not appearing on a team's Top 10 prospects list, ever). This is either perfectly reasonable or kind of pitiful, depending on your perspective, as you either find it heartening that Baseball America got it 80% right or 20% wrong. More highly-ranked pitchers tend to be more productive, but the unranked ones have a wide range of outcomes, including some pretty productive ones.
  • Teams build pitching staffs with good players through a bunch of different acquisition strategies, and a basic rule of thumb seems to be 60/40 in terms of team controlled players versus free agency. Trades for these types of players are really prevalent; this might be because the high bust rate of pitching prospects forces diversification. In addition, since teams only get a few first round draft picks a year, and those picks seem to be pretty important, it makes sense that teams would double down by trading for highly-thought of pitching prospects if going the homegrown/cost-effectiveness route.
  • The difference isn't stark, but homegrown/non-free agent pitchers may be marginally more effective than pitchers signed to free agent deals.
  • Thanks for reading. Did you see anything interesting that I missed in any of the stuff above? As usual, I'll try to answer pretty much any question that these things raise in the appropriate level of detail in the comments.