Golf Analytics

How Golfers Win

Tag Archives: data viz

Putting Driven Performance Changes are Illusory

Last week I posted about how repeatable performance on different shot types was from season to season. Tee to green play is more repeatable than putting which is more repeatable than scrambling. That makes sense once you realize that golfers play 2-3 times more tee to green shots than meaningful putts in a round; there’s just more inherent randomness in a season’s worth of putts than in a season’s worth of tee to green shots. Golfers play even fewer scrambling shots resulting in even more randomness in a season’s worth of scrambling.

Last month I also examined how repeatable small samples (4-8 tournaments) of putting performances are, in the context of discussing why I expected Jimmy Walker’s performance to regress to the mean. That micro-study indicated that there was very little correlation between a golfer’s performance in a 4-8 tournament sample of putts and the following 4-8 tournament sample of putts. In the whole, performances in such short samples regress almost entirely to the mean.

Those two lines of inquiry led me to examine whether putting was more random than tee to green performance. I have always believed that improvements/declines that were driven by over-performance in putting were less real than those driven by tee to green over-performance, but I had never actually tested that hypothesis. The key question is whether changes in performance driven by putting are less persistent than those driven by tee to green play. That is when a golfer performs better over the first half of a season, and much of the improvement can be traced back to an improvement in his putting stats, will that golfer continue to perform better in the second half of the season? The evidence says changes in performance driven by putting are more illusory than changes in performance driven by tee to green play.

Design:

I gathered the tournament by tournament overall, tee to green, and putting performances of all PGA Tour golfers in rounds measured by the ShotLink system for 2011-Present. I divided those rounds into roughly half-season chunks (January-May 2011, May-November 2011, January-May 2012, May-November 2012, January-May 2013, May-September 2013, October 2013-Present). Each chunk included around 15-18 tournaments. I considered all golfers who recorded at least 20 rounds in consecutive half-season chunks.

To measure putting performance I used the PGA Tour’s Strokes Gained Putting stat and to measure tee to green performance I used my own overall ratings with putting performance subtracted out. This methodology is consistent with my measurement of tee to green performance in numerous recent work.

Half-Season Correlations by Shot Type:

First, I measured how repeatable putting and tee to green performance was between half-season samples, much like the full-season samples used in this study. I included all golfers with at least 20 rounds in consecutive half-season samples and compared each half-season to the half-season that directly followed, including 2nd halves to 1st halves of following calendar years. This yielded samples of ~800 golfers for both tee to green and putting. Graphs are below.

half tee to green

half putting

Tee to green performance was again more repeatable than putting performance. In the study linked above consecutive full-seasons of tee to green performance were correlated at a R=0.69 level. I found a correlation of R=0.62 between consecutive half-seasons, understandably less given the smaller number of rounds/shots played. The full-season correlation for putting was R=0.55. Half-season putting performances were similarly less correlated than full-seasons at R=0.40. Both these findings are consistent with the understanding that randomness between samples increases when fewer rounds/shots are compared. Most importantly, putting is less repeatable than tee to green play.

Persistence of Changes in Performance by Shot Type:

Next, I measured how persistent changes in performance are when considering putting and tee to green play. That is, when a golfer improves their putting over a half-season sample, how much of that performance is retained in the following half-season? If 100% of the performance is retained, changes in putting performance over a half-season entirely represent a change in true talent. If 0% of the performance is retained, changes in putting performance over a half-season entirely represent randomness. The same for tee to green play. My assumption was that a larger percent of performance would be retained for tee to green play than putting, meaning that half-season samples of putting are more affected by randomness than half-seasons of tee to green play.

To measure the effect, I first established prior expectations of performance for every golfer in my sample. I simply averaged performance in tee to green play and putting for the three years prior to the beginning of each half-season sample. For example, for the May-November 2011 sample, I averaged play between May 2008 and May 2011. This is not an ideal measure of performance, but it provides a consistent baseline for comparisons to be made.

I removed all golfers from the sample who had no prior performances. This reduced my sample to around 750 consecutive half-seasons.

The values I compared were the initial delta (Prior minus 1st Half-season) and the subsequent delta (Prior minus 2nd Half-season). Using this method I can find how persistent a change in performance is between to half-seasons. I did this considering putting and tee to green play. Graphs are below.

persist tee to green

persist putting

Changes in tee to green play were twice as persistent as changes in putting play, meaning golfers who improved their tee to green play retained twice as much of those improvements as golfers who improved a similar amount in putting. Golfers maintained around 60% of their tee to green improvements, but only 30% of their putting improvements. This indicates that putting performances regress more sharply to prior expectations than tee to green performances.

Are Putting Performances More Illusory?

Finally, I gathered the data from above to measure whether changes in performance driven by putting less real than changes in performance driven by tee to green play. I ran a linear regression using the initial delta for overall performance and the initial delta for putting performance as independent variables and the subsequent delta for overall performance as the dependent variable. In short, given a certain overall change in performance and a certain change in putting performance over the first half-season, how much of that overall change in performance is retained over the second half-season?

As the following table shows golfers retain much more of their improvement or decline when that improvement or decline occurred in tee to green shots than if it occurred in putting. The columns show improvements/declines in overall play (considering all shots) and the rows show improvements/declines solely in putting. The table shows that a golfer who improves overall by 0.50 strokes will retain only a quarter of their improvement if all of the improvement was due to putting (0.50), while they will retain over half of their improvement if none of the improvement was due to putting (0.00). The equation used to produce this chart is Subsequent Delta = (0.56 * Initial Overall Delta) – (0.28 * Initial Putting Delta).

delta comparisons

Discussion:

These findings should fundamentally alter how we discuss short-term changes in performance. I’ve already shown repeatedly that performances better than prior expectation will regress to the mean over larger samples. That idea is consistent across sports analytics. However, these findings indicate that the amount of regression depends on which part of a golfer’s game is improving or declining. Golfers who improve on the basis of putting are largely getting lucky and will regress more strongly to the mean than golfers who are improve on the basis of the tee to green game. Those who improve using the tee to green game are showing more robust improvements which should be expected to be more strongly retained.

The golfers who represent either side of this for the 2014 season are Jimmy Walker and Patrick Reed. I’ve discussed both in the past month, alluding to how Walker’s improvements were almost entirely driven by putting and how Reed’s were mostly driven by tee to green play. Based off these findings, Reed is more likely to retain his improvements over the rest of the season, all else being equal, than Walker.

 

All graphs/charts are denominated in strokes better or worse than PGA Tour average. Negative numbers indicate performances better than PGA Tour average.

Advertisements

Regression Rules Everything

This post will be number/graph heavy, but it explains perhaps the most important concept in predicting golf performance – everyone regresses to the mean, no matter their performance. The below are two charts that show this effect in action. The first uses large buckets and compares all players performance in seasons with N > 50 rounds with their performance (regardless of N) in the subsequent season. The following shows similar data, broken down more at a more granular level, which also includes which percentage of seasons meet the criteria. Read the buckets as seasons within 0.05 standard deviations.

initialtosubseqseasons

tableofsubsequentseasons

In the first graph, all golfers better than +0.30 (approximately Web.com Tour average) in year 1 declined in year 2. Those worse (think Challenge Tour average) did not improve or decline, on average. Only those who performed very poorly in year 1 actually improved. For those better than PGA Tour average, the decline was fairly uniform (~0.05 to ~0.10 standard deviations). Remember, these are the aggregation of huge samples; many players improved at all skill levels, but on average regression/decline ruled everything.

In the second graph, the most important lesson is how rare the truly elite seasons are. Only roughly 1/4 of seasons came in below -.15 (which is roughly the talent level of the average PGA Tour card holder). The cut-off for the top 5% of seasons (2010-2012) came in at -0.45. Also, the regression of almost all players is evident; no bucket better than +0.35 improved in the subsequent season.

This data is fairly strong evidence that we should expect decline from most performances, on average. In fact, based on the rarity of rounds and the demonstrated regression, we should be skeptical about predicting any elite performance to be repeated the following season.

Bayesian Prediction of Golfer Performance (Individual Tournament)

I’ve posted several studies attempting to predict golfer performance. This attempted to find the importance of the previous week when predicting the following week. The study was not particularly sophisticated (simple linear regression), but the results indicated that the previous week’s performance should be valued at around 10% of the projection for the golfer the following week (90% would be the two-year performance). This other study attempted to predict golfer performance for an entire season using prior season data. That study found that no matter how many years are used or whether those years are weighted for recency, the resulting correlation is ~70%. Doing better than that for full-season prediction would indicate an additional level of sophistication beyond aggregating prior seasons or weighted data for recency.

This post, however, concerns predicting individual tournament performance using my Bayesian rankings. These rankings are generated each week by combining prior performance and sample performance using the equation ((prior mean/prior variance)+(observed mean/observed variance))/((1/prior variance)+(1/observed variance)). In this way, each golfer’s prediction for a week is updated when new information is encountered. The prior mean for a week is the Bayesian mean generated the prior week. My rankings also slowly regress to a golfer’s two-year performance if they are inactive for a period of weeks. For each week, the prior mean is calculated using the equation  (((Divisor – (Weeks since competed)) / Divisor) * (Prior Mean)) + ((1 – ((Divisor – (Weeks since competed)) / Divisor)) * (Two-year Z-Score)). I use 50 as the Divisor, which weights two-year performance at 2% for 1 week off, 27% for 5 weeks off, and 69% for 10 weeks off.

To measure how predictive these rankings were, I gathered data for all golfers who had accumulated 100 rounds on the PGA, European, Web.com, or Challenge Tour between 1-2010 and 7-2013. My sample was 643 golfers. I then examined performance in all tournaments between the 3-28-2013 and 8-8-2013. My sample was 6246 tournaments played. I then generated Bayesian rankings predicting performance before each of these tournaments played. The mean of my predictions was +0.08, indicating I expected the sample to be slightly worse than PGA average. I then compared each prediction to the golfer’s actual performance.

The table below shows the performance of Bayesian and pure Two-year predictions by including all predictions within +/- 0.05 from the displayed prediction (ie, -0.50 includes all predictions between -0.45 and -0.55). The accompanying graph shows the same information with best-fit lines.

BayesianPredictions

BayesianPredictionsGraph

Obviously, the Bayesian and Two-year predictions perform similarly. To test which is better I tested the mean square error. This shows how closely the prediction matched actual performance. I also included “dumb” predictions which simply predict all rounds will perform to the mean of all predictions (+0.08 for Bayesian, +0.055 for Two-year). The “dumb” predictions are the baseline for judging any predictions. If a prediction can’t beat it, it’s worthless.

The mean square error for the Bayesian predictions was 0.381 and 0.446 for the “dumb” predictions. The mean square error for the Two-year predictions was 0.389 and 0.452 for the “dumb” predictions. So both sets of predictions provide value over the “dumb” predictions, but both perform fairly similarly when compared to the “dumb” predictions (-0.065 for Bayesian and -0.063 for Two-year).

This study indicates two things; first, using Bayesian methods to predict golfer performance doesn’t substantially improve accuracy relative to unweighted aggregation of the last two years of performance, and second, that predicting golfer performance in individual tournaments is very difficult. A mean square error of 0.38 indicates an average miss of 3.5 strokes for golfers playing four rounds and 2.5 strokes for golfers playing two rounds.

Predicting Professional Performance of Collegiate Golfers (Part III)

Last month I posted several studies which measured how well collegiate golfers performed once they reached the professional level, compared to their Sagarin Rating during college. I updated my database with Challenge Tour results from 2011-2013 so this post is an update of those prior studies with slightly larger samples. Later this week I’ll post the results of a study using only the final two years of college performance to see if that predicts professional performance better.

This study uses the sample methodology as the study linked above in Part II. The sample size is 52, average # of college seasons was 3.4, average college performance was 70.8, average professional performance in Z-Score was +0.15.

college golf regression 3

 

The results were less predictive with the larger sample, but still R=0.59 stands as fairly predictive of professional performance. The equation to use is Pro Performance = (0.2113*Avg Sagarin) – 14.796.

Using this predictor my projections for several golfers who recently turned pro follow:
Justin Thomas -0.17
Chris Williams -0.03
T.J. Vogel +0.14
Cody Gribble +0.17
Pedro Figueiredo +0.18
Max Homa +0.21
Kevin Phelan +0.27
Jace Long +0.29
Steven Fox +0.43

Greenbrier Classic – Final Round

If we ever needed a lesson that golf is inherently random and statistics can only do so much to predict what will happen, Sunday’s final round at the Greenbrier certainly provided it. Johnson Wagner entered the final round leading Jimmy Walker by two strokes, with Jonas Blixt sitting three back. By the time Wagner was teeing off on #10 to start the back nine, Walker had not managed to close the gap at all, while Blixt had birdied #9 and #10 to get within a single stroke. From there, Blixt used several fantastic approach shots to set-up birdies, while Wagner made three crucial bogeys to fall out of the lead – handing Blixt his second PGA victory in less than a year. My analysis of the back nine will attempt to quantify strokes gained and lost on the field by each shot Wagner and Blixt made on the back nine, similar to my look at the Travelers.

What was remarkable random about the back-nine was what shots Blixt hit to set-up birdies. Blixt is fairly categorized a player very reliant on his putting for success (he was 2nd in 2012 and 47th this season). He does not hit his approach shots well, sitting nearly at the bottom in GIR, even when we account for his below average performance on tee shots. However on Sunday, Blixt’s two best shots were his approaches on #12 and #16 that set-up birdies – both worth around 3/4ths of a stroke. On the flipside, his normally sterling putting failed him on #11, #13, and #17 setting up two bogeys and depriving him of what would’ve been a decisive birdie.

On the par 5 #12, Blixt sat 108 yards from the hole after his lay-up. From there the average player hits to around 20 feet. Instead, Blixt hit a beauty to five feet, setting up a birdie that tied him with Wagner at -13. Later at the par 4 #16, Blixt was 173 yards in the fairway, a location from which the average pro hits to around 30 feet. Blixt hit an iron to nine feet, producing a make-able birdie putt that he drained to draw ahead of Wagner at -13. His approach here was his 2nd best of the day, behind the approach on #12, and his subsequent putt was his 3rd best shot.

However, along with those great approaches came three awfully poor putts which resulted in a par and two bogeys. On the long par 4 #11, Blixt missed the green, but chipped to 6 feet. PGA players make 2/3rds of those putts; Blixt not only missed but ran it three feet past, leaving a miss-able putt for bogey that he made. He hit another equally poor putt on the long par 4 #13. After laying-up, Blixt hit to 7 feet, but blew his par putt four feet past the hole. He would make the four footer for bogey. Again on the par 5 #17, Blixt was standing over a 7 footer that would’ve put him three clear of Wagner. Seven footers are 55% putts normally, so Blixt’s miss cost him over half of stroke, but at least he didn’t blow it several feet past the hole.

In total, Blixt’s back nine shots were worth: -0.44 strokes (tee shots), +1.85 strokes (approach shots), +0.48 strokes (short game), and -0.35 strokes (putts).

blixtgreenbrier

Johnson Wagner’s back nine was, succinctly, a disaster. He began it with a two stroke lead over Blixt that was quickly shortened to one stroke after Blixt’s birdie ahead of him on #10. At this point, no one else was any better than -11, while Wagner sat at -14. From there, Wagner bogeyed three of the next five holes and watched Blixt draw two strokes ahead of him for the win. For a guy who has been a steadily good putter (29th, 39th, and 41st in Strokes Gained in 2011-13), the flat stick failed him on Sunday. Two of his three worst shots were missed par putts inside 10 feet (on #11 and #13).

It was #15 that really sunk him though. He started #15 even with Blixt at -12. As we saw earlier, Blixt would go onto birdie #16, but Wagner would not have been in terrible position if he had parred #15 and moved on, as 16-18 aren’t difficult holes. His tee shot on the par 3 #15 was poor, ending up in the rough 14 yards from the pin. From there, PGA players bogey about half the time. Wagner blew his next shot 33 feet past the pin, leaving him a nearly guaranteed bogey. Within 15 minutes between Wagner’s bogey on #15 and Blixt’s birdie on #16, the  tournament was all but over.

For the back-nine, Wagner finished -0.23 strokes (tee shots), -0.17 strokes (approach shots), -0.71 strokes (short game), and -1.29 strokes (putts), a thoroughly miserable performance for a guy who is a PGA Tour player because of his ability to putt.

wagnergreenbrier

greenbrier component stats

 

Travelers Championship – Final Round

Though Ken Duke and Chris Stroud entered the final round of the Travelers having zero combined PGA Tour victories and trailing co-leaders Bubba Watson, Charley Hoffman, and Graham DeLaet, by Sunday evening, Duke had won his first title, on the second hole of a playoff with Stroud, while Stroud took home his largest ever check. The final round featured two critical moments; the first when solo leader Bubba Watson dropped his drive on the par 3 16th in the water, making triple bogey and torpedoing his chance to win and the second when, having overshot the 18th green and facing a make-or-break situation, Stroud drained his chip to force a playoff with Duke.

To evaluate how Duke and Stroud made it to the playoff Sunday, I copied down each shot taken on the back nine Sunday using the PGA Tour’s ShotLink data. I then employed the Strokes Gained method popularized by Mark Broadie to evaluate the quality of each tee shot, approach, chip, pitch, and putt. This method uses distance from the hole and location of the ball to generate the average number of shots a PGA Tour golfer will take to complete the hole at the end of each shot. If that average is more than one shot less than at the end of the previous shot, the golfer “gained” strokes on the field, while if the average is less than one shot lower than at the end of the previous shot, the golfer “lost” strokes on the field. The strokes gained method does not factor in anything but location of the ball, so severity of the rough or bunkers, speed of the greens, intensity of the wind, etc. are not explicitly accounted for, however, TPC River Highlands played essentially to par this week (70.2 Field Average compared to a Par of 70).

Ken Duke entered the final round at -8, two back of the three leaders and one behind Stroud. He played the front nine in -1 and entered the back nine with a chance to win, as none of those who entered the day ahead of him had placed any more distance between them. Duke made four birdies on the back nine, two set-up by fantastic approach shots and two on long putts.

On the long 462 yard par 4 10th, Duke hammered his drive 316 yards, an above-average drive, and followed with a beautiful approach to 5 feet. That approach took Duke from 2.90 expected strokes to only 1.25 expected strokes, a gain of 0.65 strokes, his 4th most important shot of the back nine. However, that was his only notable above-average “long” shot on the back nine. Most of his remaining tee shots and approaches were quite poor.

Instead, he relied on his short game and putting to survive. On the par 4 12th, Duke hit a poor tee shot and then a poor approach into the rough around 13 yards from the hole. From there, most players take 2.42 strokes to finish. Duke chipped to less than a foot, a shot worth 0.42 strokes gained. Then on the short par 4 15th after he ended up in the rough from around 40 yards off the tee. Duke hit a perfect pitch shot to six feet, setting up a great birdie opportunity which he converted. The average player sitting in the rough from 40 yards takes 2.75 strokes to finish up; Duke’s pitch to six feet was worth 0.35 strokes. On 18, after hitting an extremely nervy tee shot and getting his second to just shy of the green, Duke chipped across the surface to less than two feet, almost guaranteeing a par on the hole. That shot was worth 0.36 strokes gained.

His putting was also very strong though. His birdie putt on 15 from six feet was worth 0.40 strokes, but that was only his third most important putt of the day. Coming off a birdie at 10, Duke hit a quality approach to seventeen feet, removing the threat of bogey, but birdie was doubtful as only around 20% of putts are holed from that distance. However, Duke rolled it in for a gain of 0.80 strokes. His most important putt was on 13 though. Three straight below average shots had left Duke on the green, but at 46 feet birdie was looking extremely unlikely. PGA golfers hole less than 3% of their 40+ foot putts, while three putting almost 15% of the time. On the shorter par 5, Duke was looking at a legitimate possibility of bogeying. Instead, he meandered his 46 foot putt in for birdie, a putt worth an enormous 1.07 strokes gained, his best of the round.

duketravelers

In contrast to Duke, who had eight shots that gained more than a 1/3rd of a stroke – seven of which were putts or short shots (pitches, chips, etc.) – Stroud had only two shots that gained more than 1/3rd of a stroke and two shots that lost him over 0.5 strokes against the field.

Stroud entered the back nine -1 for the day and -10 for the tournament and reeled off three straight pars without any excitement on 10 through 12. On 13, he hit a wonderful short approach from 22 yards that left him with a seven foot putt for birdie. While such a putt isn’t a guarantee, the average player makes it around 60% of the time. Stroud’s miss was worth -0.55 strokes gained. Stroud rebounded on the short par 4 15th by holing a ten footer for birdie, 0.60 strokes gained.

As Duke finished up his round at -12, Stroud teed off on 18 needing a birdie to force a playoff. After an ideal drive left him with only 2.78 expected strokes, Stroud flew his 93 yard approach shot well over the green, a miserable shot that cost him -0.54 strokes and all but eliminated him from contention. From 17 yards, the average PGA player takes 2.32 strokes to finish up. When Stroud rolled in his chip, he gained a whopping 1.32 strokes on the field and forced Duke into a playoff for the Championship.

stroudtravelers

The two took different paths to the playoff; Duke rode a series of high quality chips, pitches, and putts to birdies and par saves, while Stroud relied on steady play and that one huge moment on 18.