An Accurate Measurement of Scrambling Skill

(This is the first of three planned posts explaining the flaws in commonly used stats for evaluating golfers’s skills at driving, hitting approach shots, and scrambling and laying out a replacement based on publicly available PGA Tour stats.)

The Scrambling stat was developed to measure how often a golfer avoids bogey after missing the green with their approach shot. That requires a golfer to (usually) chip or pitch onto the green and then hole a par putt. Scrambling is simply calculated by dividing successful scrambles (par or less) by total GIR missed. The average for PGA Tour players in 2012 was ~59%, indicating that, when missing the green, they made par a bit more than half the time.

Scrambling is often used to evaluate whether a player has a good “short game”. Luke Donald has ranked 5th, 8th, and 4th in recent years and is generally proclaimed as one of the best short game players on Tour, along with Steve Stricker, Ian Poulter, and Brian Gay (who all have multiple top 10 finishes in the last few years). However, what scrambling really measures is a combination of three skills. First, it measures what it purports to – the ability to hit chips, pitches, sand shots, etc. around the green close to the pin. But it also measures the ability to putt, because scrambling requires a putting stroke to finish up, and hit approach shots, because players hit their chips, pitches, sand shots, etc. from locations that vary in difficulty. A very good putter will have an inflated scrambling ratio because they make a lot of putts after leaving their ball short of the hole that an average putter would miss. A good approach shot player will have an inflated scrambling ratio because when they miss the green, they leave themselves closer to the pin and in better locations (fairway or fringe instead of bunker or rough).

So with those shortcomings, how do you see through the noise and capture only the ability to hit good chips, pitches, sand shots, etc.? I first downloaded the PGA Tour data for scramblings from >30 yards, 20-30 yards, 10-20 yards, and <10 yards. This data represents all scrambling shots taken in tournaments where ShotLink is used (US based tournaments except Majors). I then found each golfer’s GIR in ShotLink measured rounds. Then I calculated how often PGA Tour golfers successfully scramble from each of the aforementioned distance bins (>30 yards – 27%, 20-30 yards – 52%, 10-20 yards – 64%, <10 yards – 85%). I then adjusted each players data to find how often they shot from each of the four distance bins, then multiplied that number by how often the average golfer successfully scrambled from that distance. The result for each golfer is how often the average PGA golfer would be expected to scramble successfully based on where that golfer hit their scrambling shot from. That solves the problem of golfers hitting from varying locations.

To adjust for putting skill, I downloaded each golfer’s Strokes Gained Putting for 2012. This stat measures how well a player putts compared to PGA average based on the length of the putt (ie, players who make more 20 footers than average will be above-average). I threw Strokes Gained Putting into a linear regression with Strokes Gained Putting and the earlier calculated expected scrambling by distance stat as the independent variables and a golfer’s overall Scrambling ratio as the dependent variable. I had 191 golfers in the regression. My R=0.70, which indicates that Putting and location of the shot explains 70% of the Scrambling stat, which is extremely large for a stat that is used to rank golfers ability to hit around the green. Both SGP and expected scrambling by distance were significant at the 0.001 level. The regression produced an equation (y = -0.038+(1.061*Putting)+(.0659*Location). I calculated the Expected Scrambling stat from that equation for each golfer. This measures how often a golfer should get up-and-down given a certain skill at putting and a certain location before the scrambling shot. When adjusting for these factors, Bo Van Pelt, Luke Donald, Brandt Snedeker, and Zach Johnson faced the easiest scrambling situations, being expected to make par or better on 65% of their missed greens.

From their, determining actual skill around the greens was simple. I subtracted a golfer’s Expected Scrambling from their actual Scrambling performance. The result indicated how much more often a player successfully scrambled, corrected for the location of their scrambling shots and their skill putting.

Top 10 and Bottom 10 in Adjusted Scrambling:Several of the golfers in the old Scrambling rankings look good based on this adjusted ranking – Dufner, Poulter, and Rose were in the top 10 before and remain there again. But most of the rest were not highly rated by old Scrambling, highlighted by Nick O’Hern ranking  93rd (roughly average) in Scrambling despite being a poor putter and hitting his shots from the 2nd worst locations of anyone on Tour.

The trailers are more reflective of the old Scrambling rankings, though Bo Van Pelt was ranked 104th in scrambling by the old system, but because of he putted very well last year and hit from the best locations, he comes out 2nd worst in this ranking.

It is worth noting that this analysis ignores the difficulty of courses played. It’s probable that certain courses are more difficult to scramble successfully on, while others are easier. In his seminal Assessing Golfer Performance on the PGA Tour Broadie found that there were differences in course difficulty (~4 strokes between the most & least difficult), but that they were heavily concentrated in the long game (drives and approaches over 100 yards). The difference between the ten most difficult and ten least difficult courses overall was only 0.3 strokes when considering the short game (basically any shots inside 100 yards). His definition of short game means scrambling shots considered above make up roughly 2/3rds of the shots considered. I would expect that differences in scrambling difficulty by courses shouldn’t affect these adjusted Scrambling numbers by more than 2%, though I will revisit the topic of course difficulty in a future post.