An Accurate Measurement of Scrambling Skill

(This is the first of three planned posts explaining the flaws in commonly used stats for evaluating golfers’s skills at driving, hitting approach shots, and scrambling and laying out a replacement based on publicly available PGA Tour stats.)

The Scrambling stat was developed to measure how often a golfer avoids bogey after missing the green with their approach shot. That requires a golfer to (usually) chip or pitch onto the green and then hole a par putt. Scrambling is simply calculated by dividing successful scrambles (par or less) by total GIR missed. The average for PGA Tour players in 2012 was ~59%, indicating that, when missing the green, they made par a bit more than half the time.

Scrambling is often used to evaluate whether a player has a good “short game”. Luke Donald has ranked 5th, 8th, and 4th in recent years and is generally proclaimed as one of the best short game players on Tour, along with Steve Stricker, Ian Poulter, and Brian Gay (who all have multiple top 10 finishes in the last few years). However, what scrambling really measures is a combination of three skills. First, it measures what it purports to – the ability to hit chips, pitches, sand shots, etc. around the green close to the pin. But it also measures the ability to putt, because scrambling requires a putting stroke to finish up, and hit approach shots, because players hit their chips, pitches, sand shots, etc. from locations that vary in difficulty. A very good putter will have an inflated scrambling ratio because they make a lot of putts after leaving their ball short of the hole that an average putter would miss. A good approach shot player will have an inflated scrambling ratio because when they miss the green, they leave themselves closer to the pin and in better locations (fairway or fringe instead of bunker or rough).

So with those shortcomings, how do you see through the noise and capture only the ability to hit good chips, pitches, sand shots, etc.? I first downloaded the PGA Tour data for scramblings from >30 yards, 20-30 yards, 10-20 yards, and <10 yards. This data represents all scrambling shots taken in tournaments where ShotLink is used (US based tournaments except Majors). I then found each golfer’s GIR in ShotLink measured rounds. Then I calculated how often PGA Tour golfers successfully scramble from each of the aforementioned distance bins (>30 yards – 27%, 20-30 yards – 52%, 10-20 yards – 64%, <10 yards – 85%). I then adjusted each players data to find how often they shot from each of the four distance bins, then multiplied that number by how often the average golfer successfully scrambled from that distance. The result for each golfer is how often the average PGA golfer would be expected to scramble successfully based on where that golfer hit their scrambling shot from. That solves the problem of golfers hitting from varying locations.

To adjust for putting skill, I downloaded each golfer’s Strokes Gained Putting for 2012. This stat measures how well a player putts compared to PGA average based on the length of the putt (ie, players who make more 20 footers than average will be above-average). I threw Strokes Gained Putting into a linear regression with Strokes Gained Putting and the earlier calculated expected scrambling by distance stat as the independent variables and a golfer’s overall Scrambling ratio as the dependent variable. I had 191 golfers in the regression. My R=0.70, which indicates that Putting and location of the shot explains 70% of the Scrambling stat, which is extremely large for a stat that is used to rank golfers ability to hit around the green. Both SGP and expected scrambling by distance were significant at the 0.001 level. The regression produced an equation (y = -0.038+(1.061*Putting)+(.0659*Location). I calculated the Expected Scrambling stat from that equation for each golfer. This measures how often a golfer should get up-and-down given a certain skill at putting and a certain location before the scrambling shot. When adjusting for these factors, Bo Van Pelt, Luke Donald, Brandt Snedeker, and Zach Johnson faced the easiest scrambling situations, being expected to make par or better on 65% of their missed greens.

From their, determining actual skill around the greens was simple. I subtracted a golfer’s Expected Scrambling from their actual Scrambling performance. The result indicated how much more often a player successfully scrambled, corrected for the location of their scrambling shots and their skill putting.

Top 10 and Bottom 10 in Adjusted Scrambling:Several of the golfers in the old Scrambling rankings look good based on this adjusted ranking – Dufner, Poulter, and Rose were in the top 10 before and remain there again. But most of the rest were not highly rated by old Scrambling, highlighted by Nick O’Hern ranking  93rd (roughly average) in Scrambling despite being a poor putter and hitting his shots from the 2nd worst locations of anyone on Tour.

The trailers are more reflective of the old Scrambling rankings, though Bo Van Pelt was ranked 104th in scrambling by the old system, but because of he putted very well last year and hit from the best locations, he comes out 2nd worst in this ranking.

It is worth noting that this analysis ignores the difficulty of courses played. It’s probable that certain courses are more difficult to scramble successfully on, while others are easier. In his seminal Assessing Golfer Performance on the PGA Tour Broadie found that there were differences in course difficulty (~4 strokes between the most & least difficult), but that they were heavily concentrated in the long game (drives and approaches over 100 yards). The difference between the ten most difficult and ten least difficult courses overall was only 0.3 strokes when considering the short game (basically any shots inside 100 yards). His definition of short game means scrambling shots considered above make up roughly 2/3rds of the shots considered. I would expect that differences in scrambling difficulty by courses shouldn’t affect these adjusted Scrambling numbers by more than 2%, though I will revisit the topic of course difficulty in a future post.

Predicting College Players in Pro Golf

With the recent end of the NCAA Golf season, top collegiate golfers are terminating their amateur status and entering pro tournaments with the aim of earning enough money to earn status on one of the major tours for 2014. Jordan Spieth was the first to turn pro this season, entering several PGA Tour tournaments early in the year and earning Special Temporary Member status. Just this past week, Alabama’s Justin Thomas and Washington’s Chris Williams turned pro at the Travelers Championship and both made the cut. The main question with these freshly minted pros from my point of view is just how good are they compared to a PGA regular?

Luckily, Golf Week/Jeff Sagarin publishes yearly rankings of college golfers measuring their performance throughout the college season. This season, the top ranked golfer was Michael Kim, the low amateur at the US Open. Sagarin reports the ratings on a scale mirroring a typical golf score, with the most elite golfers earning ratings in the 68.0s while the 25^th best golfer in a season will rate around a 70.0-70.5. How do those ratings translate into the Z-Score Rating System I use?

To find out I constructed a list of golfers that both were ranked in Sagarin’s top 25 for 2010-11 or 2011-12 before turning pro and had at least 8 rounds on the Web.com, European, or PGA Tours in the year after their college season ended (basically June to May). There were 18 such golfers that met both criteria including notables like Harris English, Bud Cauley, and Jordan Spieth. For those 18, I gathered Sagarin ratings for their last two seasons in college (or only one season if they left after freshman year) and their Z-Scores, adjusted for strength of field, for all rounds they played in the year following the end of their last college season. The N for the golfers ranged from 8 rounds to 108 rounds.

I then ran a linear regression with the average Sagarin rating of the final two seasons in college as the independent variable and their Z-Score in the year following as the dependent variable. For the full data-set of 18 golfers, the R² was only 0.09 (graph of data is below) – meaning for our data collegiate performance explained about 30% of performance as a pro.

Now, what can we learn from such a small sample? Perhaps not much, especially since there is likely to be survivor bias in the data. The golfers who perform best in their first few tournaments are more likely to get additional sponsor’s exemptions, gain status on a major tour, and get additional rounds of data. Golfers who struggle early will be forced to play on the NGA Tour or other lower tours that I do not have data for. Perhaps the most important thing is that a golfer would have to play to a rating of 69.3 in college to be projected as a 0.00 as a pro (basically average for the PGA Tour). Only Michael Kim (who is staying an amateur) and Brandon Stone (who just turned pro at this past week’s BMW International and finished T10) were that good in 2012-13.

I will return to this subject when I finish compiling Challenge Tour (European minor league) and NGA Tour and eGolf Tour data for past seasons which will give me a much larger sample to work with and from which to draw conclusions.

Travelers Championship – Final Round

Though Ken Duke and Chris Stroud entered the final round of the Travelers having zero combined PGA Tour victories and trailing co-leaders Bubba Watson, Charley Hoffman, and Graham DeLaet, by Sunday evening, Duke had won his first title, on the second hole of a playoff with Stroud, while Stroud took home his largest ever check. The final round featured two critical moments; the first when solo leader Bubba Watson dropped his drive on the par 3 16th in the water, making triple bogey and torpedoing his chance to win and the second when, having overshot the 18th green and facing a make-or-break situation, Stroud drained his chip to force a playoff with Duke.

To evaluate how Duke and Stroud made it to the playoff Sunday, I copied down each shot taken on the back nine Sunday using the PGA Tour’s ShotLink data. I then employed the Strokes Gained method popularized by Mark Broadie to evaluate the quality of each tee shot, approach, chip, pitch, and putt. This method uses distance from the hole and location of the ball to generate the average number of shots a PGA Tour golfer will take to complete the hole at the end of each shot. If that average is more than one shot less than at the end of the previous shot, the golfer “gained” strokes on the field, while if the average is less than one shot lower than at the end of the previous shot, the golfer “lost” strokes on the field. The strokes gained method does not factor in anything but location of the ball, so severity of the rough or bunkers, speed of the greens, intensity of the wind, etc. are not explicitly accounted for, however, TPC River Highlands played essentially to par this week (70.2 Field Average compared to a Par of 70).

Ken Duke entered the final round at -8, two back of the three leaders and one behind Stroud. He played the front nine in -1 and entered the back nine with a chance to win, as none of those who entered the day ahead of him had placed any more distance between them. Duke made four birdies on the back nine, two set-up by fantastic approach shots and two on long putts.

On the long 462 yard par 4 10th, Duke hammered his drive 316 yards, an above-average drive, and followed with a beautiful approach to 5 feet. That approach took Duke from 2.90 expected strokes to only 1.25 expected strokes, a gain of 0.65 strokes, his 4th most important shot of the back nine. However, that was his only notable above-average “long” shot on the back nine. Most of his remaining tee shots and approaches were quite poor.

Instead, he relied on his short game and putting to survive. On the par 4 12th, Duke hit a poor tee shot and then a poor approach into the rough around 13 yards from the hole. From there, most players take 2.42 strokes to finish. Duke chipped to less than a foot, a shot worth 0.42 strokes gained. Then on the short par 4 15th after he ended up in the rough from around 40 yards off the tee. Duke hit a perfect pitch shot to six feet, setting up a great birdie opportunity which he converted. The average player sitting in the rough from 40 yards takes 2.75 strokes to finish up; Duke’s pitch to six feet was worth 0.35 strokes. On 18, after hitting an extremely nervy tee shot and getting his second to just shy of the green, Duke chipped across the surface to less than two feet, almost guaranteeing a par on the hole. That shot was worth 0.36 strokes gained.

His putting was also very strong though. His birdie putt on 15 from six feet was worth 0.40 strokes, but that was only his third most important putt of the day. Coming off a birdie at 10, Duke hit a quality approach to seventeen feet, removing the threat of bogey, but birdie was doubtful as only around 20% of putts are holed from that distance. However, Duke rolled it in for a gain of 0.80 strokes. His most important putt was on 13 though. Three straight below average shots had left Duke on the green, but at 46 feet birdie was looking extremely unlikely. PGA golfers hole less than 3% of their 40+ foot putts, while three putting almost 15% of the time. On the shorter par 5, Duke was looking at a legitimate possibility of bogeying. Instead, he meandered his 46 foot putt in for birdie, a putt worth an enormous 1.07 strokes gained, his best of the round.

In contrast to Duke, who had eight shots that gained more than a 1/3rd of a stroke – seven of which were putts or short shots (pitches, chips, etc.) – Stroud had only two shots that gained more than 1/3rd of a stroke and two shots that lost him over 0.5 strokes against the field.

Stroud entered the back nine -1 for the day and -10 for the tournament and reeled off three straight pars without any excitement on 10 through 12. On 13, he hit a wonderful short approach from 22 yards that left him with a seven foot putt for birdie. While such a putt isn’t a guarantee, the average player makes it around 60% of the time. Stroud’s miss was worth -0.55 strokes gained. Stroud rebounded on the short par 4 15th by holing a ten footer for birdie, 0.60 strokes gained.

As Duke finished up his round at -12, Stroud teed off on 18 needing a birdie to force a playoff. After an ideal drive left him with only 2.78 expected strokes, Stroud flew his 93 yard approach shot well over the green, a miserable shot that cost him -0.54 strokes and all but eliminated him from contention. From 17 yards, the average PGA player takes 2.32 strokes to finish up. When Stroud rolled in his chip, he gained a whopping 1.32 strokes on the field and forced Duke into a playoff for the Championship.

The two took different paths to the playoff; Duke rode a series of high quality chips, pitches, and putts to birdies and par saves, while Stroud relied on steady play and that one huge moment on 18.