Tag Archives: statistics

A deep dive in to the 2020 NRL premiership

This is my third season preview and I have got some things laughably wrong in the previous attempts (see 2018 and 2019). This year’s will be a slightly different format to previous years but undertaken in the same spirit of considering each team’s strengths, weaknesses and opportunities, as well as assessing the changes made since last year and their potentially positive or negative impact on performance.

However, I plan to have fewer laughably wrong predictions in 2020 simply by making fewer predictions. After all, if you want to see laughably wrong rugby league analysis, you can just pick up a copy of the paper.

Last season in a nutshell

2019 was a weird season and completely different to its equally weird predecessor. In 2018, eight teams finished within a win of each other and then were systematically dismantled by the Roosters and Storm in the finals. In 2019, we had three teams that could clearly play football, another couple that were adequate and a bunch of losers that didn’t want to make the finals. The round 17 golden point field goal shoot-out between the Broncos and Warriors, leading to a draw after multiple botched attempts, encapsulated the lose-at-all-costs mentality that defined positions seven through fifteen on the ladder. In the end, the Roosters emerged victorious in a manner that still infuriates me, with the Raiders running out of points and the Storm running out of steam when it counted.

A relatively quiet off-season – dominated by Latrell Mitchell’s signature, the Tigers’ warchest, Melbourne pollinating the landscape with overpriced talent and what the second Brisbane team should be named – has seen most teams turn up to 2020 in roughly the same shape as they approached 2019. It makes it very difficult to get a grasp on how this year might pan out, without just repeating pretty much what happened in 2019. And, no, neither the Nines nor pre-season trials will provide any insight.

How it all works

I appreciate that it’s difficult to keep up with the Pythago NRL Expanded Universe™ of metrics and ratings. Not only are they generally more complicated than standard stats, I tweak them almost every year based on what I learned during the previous season. I created a short reference guide to what it all means.

2020 team projections are based on round 1 lineups, taken as a mix from NRL.com and League Unlimited. 2020 roster composition is based on the listed signings on League Unlimited (as of 28 February) but 2019 roster information is based only on players who played at least one game.

Jump ahead

Embed from Getty Images

nrl-bne Brisbane Broncos

2020-bne

2020-bne-2

2020-bne-3

2020-bne-4

Brisbane deserved to finish ninth or tenth last season. The Broncos were the second most heavily biased to their forwards, behind the Cowboys and the immutable Taumalolo. The strong and young forward pack means that the Broncos are projected to have the third most production in 2020 but there’s diminishing returns in having powerful forwards if the other parts of the team continue to struggle to execute. The reality is that Brisbane needs less stupidity out of the forwards, more offence out of the backs and an all round improvement in defence.

I assume we will see more of the same from last year because nothing has changed significantly enough to suggest otherwise. Giving the captaincy to Glenn over Boyd doesn’t change the fact that neither should be on the field. If Boyd plays anywhere, that side of the field will shut down in attack and one or two players will have to cover his defensive workload. None of the talk out of the club has really addressed this or any of the many other problems, so I don’t see how they could have fixed them.

As to what question Brodie Croft answers, I don’t know but it isn’t halfback production. Ironically, I think the team would perform better if Milford’s TPR was lower and he didn’t have to waste time carrying so much dead weight, both undercooked rookies and overcooked veterans.

Embed from Getty Images

nrl-cbr Canberra Raiders

2020-cbr-1

2020-cbr-2

2020-cbr-3

2020-cbr-4

Possibly more than any other team, the Raiders have lost the most talent in the off-season. Taylor is down on their prospects but expects Canberra to still perform above average. Elo and Poseidon, carrying through from 2019, expect them to return to premiership contention. The Raiders’ defence wasn’t quite enough to win them the premiership (as a rule of thumb, the Poseidon defence rating should be at least +50) and it would be unlikely to not see some reversion towards mean this year. With luck, it won’t be as disastrous as 2017 and 2018 following 2016.

While Canberra’s defence was good, the attack completely dissipated in the finals. Bringing in an English half is a risk, but so was bringing in English forwards, and it paid handsome dividends. By all accounts, George Williams is the goods and might be the missing piece of the puzzle. Leilua, Rapana and Sezer have all left in the off-season, to be replaced by Curtis Scott, who celebrated by punching some cops. After being mired mid-to-lower-table for so long under the decade-long dual dominance of Sydney and Melbourne, it would be genuinely surprising to see a team turn a corner and transform into perennial challengers.

Embed from Getty Images

nrl-cnt Canterbury Bulldogs

2020-CNT-1

2020-CNT-2

2020-CNT-3

2020-CNT-4

The Bulldogs are behind, way behind.

With Kieran Foran missing most, if not all, of the 2020 season, the Bulldogs either need significant development out of their relatively young squad or to land some signatures. Neither seem likely, especially as the club is likely to still be paying freight on players from the Castle-Hasler era and the current squad do not have the track record to suggest any superstars are emerging (perhaps Renouf To’omaga excepted). The players signed to development contracts do not have particularly impressive stats from the NSW Cup. With last year’s significant outperformance of the fundamentals, reversion to mean would likely mean a wooden spoon.

However, we’re now into our second full season of rebuild at Belmore and the signs have been promising. Late surges of form in 2018 and 2019 when other teams start to switch off towards the end of the season have often been timely, snagging wins that Canterbury have no right to and desperately need. This defiance indicates that Dean Pay can coach (“Dogs of war”, etc, etc) and jag the seven or eight wins required to avoid the spoon. I’m comparatively bullish on the Bulldogs but they need to resolve their cap issues to get some talent on board if they want to really progress.

Embed from Getty Images

nrl-cro Cronulla Sharks

2020-Cro-1

2020-Cro-2

2020-Cro-3

2020-Cro-4

With their home games moved to Kogarah, we may finally get an answer to the age-old question: what do the Cronulla Sharks actually do to justify their place in the NRL? 

The Sharks’ 12-12 record and seventh place belied how well they played last season. Let down significantly by their goal kicking, the Sharks lost a record five games despite scoring more tries. While that’s a NSWRL/NRL record, I doubt that’s ever happened at any other time in football. The odds of it are simply astronomical. Tack on a couple of extra wins to last year’s total to appropriately set your expectations.

Cronulla should have the talent to comfortably make the finals in 2020. We probably won’t see anything much more interesting than that out of them unless a couple of the top clubs stumble.

With Paul Gallen retired, the team will have to adjust their production bias away from the forwards. I still have question marks on Bronson Xerri but his production last year was impressive and Braden Hamlin-Uele should probably be starting.

Embed from Getty Images

nrl-gct Gold Coast Titans

2020-gct-1

2020-gct-2

2020-gct-3

2020-gct-4

Perhaps the most significant thing to happen to the Titans last season was being surpassed by Newcastle, to be left at the bottom of the league in class Elo ratings. It might be recalled that the Knights were the worst NRL team of all time in 2016 and since then, the Knights have gotten better and the Titans so much worse.

Last season, you would have only taken a handful of players from the Titans to your own club given the opportunity: Arrow, Fotuaika, Brimson (who has a surprisingly low TPR) and maybe Tyrone Roberts if you were feeling generous. The Titans managed to hang on to them, except Arrow who will be departing for Souths next year. The rest of the roster under Garth Brennan was a joke, hence the 4-20 record, so hopes are pinned on the incoming Justin Holbrook, having left the best Super League team for the worst NRL team. Indeed, last season the Titans were ranked lower than half of the Super League.

With the number of experienced veterans and the talent pool on their door step, the Titans really should be better than they are. They are not and the sims reflect it. Fans will hope the new coach can get more out of the squad. Appointing Kevin Proctor captain is not the most auspicious start to turning around the club’s culture. Sick 9s jersey though.

Embed from Getty Images

nrl-man Manly Sea Eagles

2020-man-1

2020-man-2

2020-man-3

2020-man-4

The Taylors bear out how low expectations were for Manly in 2019, only for those expectations to be obliterated. The Sea Eagles were one of the few teams outside the big two that could win regularly. I went out on a limb pre-season and suggested Manly would make the finals. While that was pure luck on my part, they managed to do it. It turns out Des Hasler can still coach, even after taking some shine off his reputation while at the Bulldogs.

Backing up without the element of surprise and the reversion to mean will be challenging. Reversion to mean is a harsh mistress and often a huge outperformance is punished with an equally severe reaction in the opposite direction in the following season. The law of averages demands its tribute. For now at least, Manly’s prospects for 2020 appear to be good and based on sound fundamentals.

It hasn’t been discussed nearly enough how costly Manase Fainu missing some (most? all?) of the upcoming season will be. He was one of the big unknowns that stepped up last year and with Api Koroisau now at Penrith, Manly are bereft of options at hooker. It is too early to discuss Cade Cust as a long-term successor to Daly Cherry-Evans but he had an impressive debut season.

Embed from Getty Images

nrl-mel Melbourne Storm

2020-mel-1

2020-mel-2

2020-mel-3

2020-mel-4

The Storm and Craig Bellamy, as they often are, were the biggest outperformers of their projections in the league. Melbourne finished the season with a 20-4 record, a record only bettered* by the Storm’s 21-3 2007 season. Unlike 2017, where it seemed inevitable that the Storm would win the premiership after winning 20 games, they never seemed to get much credit for what was still a very impressive season in 2019.

Melbourne just have the knack of taking extremely talented young men, putting them on the football field and winning games. Positions don’t seem important, neither do the names. It will likely continue forever because there is plenty of talent pushing through in reserve grade. Even the departure of several reasonable quality players doesn’t seem to have made a dent in their prospects.

So yeah, they’re pretty good. If I’m lucky, I may live long enough to see the next Broncos win over the Storm, an event about as frequent as Halley’s Comet.

Embed from Getty Images

nrl-new Newcastle Knights

2020-new-1

2020-new-2

2020-new-3

2020-new-4

The Knights will be glad to see the back of the 2010s, where they were the worst team in the NRL and nearly went broke. The good news is that the Knights might legitimately make the finals this year.

The Knights massively outperformed in 2018, which then led to many talking heads predicting serious success in 2019. Success wasn’t forthcoming because the fundamentals weren’t there. Instead, we had a heady mix of nostalgia, over-excitement and Blue bias that completely crippled the predominantly Sydney-based media’s capacity to objectively analyse (I have the same problem in the opposite direction but at least I’m aware of it).

Mitchell Pearce had a career season in 2019, at least until I wrote about it, but otherwise the team struggled to meet expectations. I’m more of a numbers guy than a culture guy, but even I could see that the team was often not trying. Results from round 16 through 21 last year bear that out. Their thrashing at the hands of the Titans in round 5 was more typical of the season than the six wins that followed.

The finishing touches to the “rebuild” have now been applied, not least Adam O’Brien replacing Nathan Brown as head coach, to bring the Knights back in contention for finals places. Newcastle are still a way off challenging for the premiership.

Embed from Getty Images

nrl-nzw New Zealand Warriors

2020-nzw-1

2020-nzw-2

2020-nzw-3

2020-nzw-4

People think the Warriors are bad. They haven’t been. New Zealand’s club embodies average-ness with every fibre and loves to squander an opportunity. The thing about the median is that it’s not last place, so I’m always wary of any prediction that gives the spoon to the Warriors.

The loss of Shaun Johnson was not well compensated and the team is now overly reliant on Roger Tuivasa-Sheck and the back line to generate production. The forward pack has not been impressive as a whole. The lack of star power – currently projected to be zero players – is concerning, although not damning. Kodi Nikorima is, at best, a below average halfback and Chanel Harris-Tavita is apparently too young to start but he’s far better bet (.098 in 2019 compared to the .085-ish range Nikorima has played in the last three years). The Warriors will chase eighth place with the Broncos, Tigers and Knights until they get tired and slump down the ladder.

More worryingly, the Warriors are on the precipice of falling full-time into the ‘bad’ category and once that happens, I don’t know how the club will pull itself out. The Auckland Rugby League should be a conveyor belt of talent and the Warriors should be at least Broncos-calibre, if not the Storm. Until that gets worked out, New Zealand will probably bounce along the bottom of the ladder.

Embed from Getty Images

nrl-nqc North Queensland Cowboys

2020-nqc-1

2020-nqc-2

2020-nqc-3

2020-nqc-4

A good showing at the 9s tournament in Perth has clouded judgement about what the Cowboys are capable of. Consider their stacked halves options of Michael Morgan, Jake Clifford and Scott Drinkwater. Drinkwater is only a thought there because Valentine Holmes is obviously the fullback. The ever-reliable Vaa’i Taumalolo will put the team on his back and Kyle Feldt will finish in the corner.

It sounds good in principle but most of these pieces have been available for the last three years and, other than limping to the grand final in 2017 and avoiding the spoon in 2018 and 2019, those three years have had little to celebrate. After all, we’re projecting a team with some well-known players to only be twelfth best. Without Taumalolo, a certified freak and statistical anomaly, that number would be a lot closer to the bottom.

Paul Green seems intent on stifling the creativity of his playmakers and/or was overly reliant on Johnathan Thurston to make plays. Either way, he has to adjust to the new Thurston-less world where scoring six to twelve points is not going to be enough. Despite delivering the premiership in 2015, a bad 2020 might be the end of the road for Green.

Embed from Getty Images

nrl-par Parramatta Eels

2020-par-1

2020-par-2

2020-par-3

2020-par-4

I think this is it for the Eels. They are due for their once-a-decade (give or take) tilt at the premiership.

The Taylors are not too crash hot on the Eels. There are holes in key TPR ratings: Reed Mahoney at hooker, Dylan Brown nominally at five-eighth and, to a lesser extent, Clint Gutherson at fullback. The forward pack is slightly above average but none are exceptional. Reagan Campbell-Gillard might be one of those high-TPR, low-impact players, like Aaron Woods. On the other hand, Parramatta are capable of outperforming their projections which, for their top players at least, seem conservative. Last season’s hiccups only came when meeting the Storm, a hurdle that has felled better teams in the past.

The Eels are one of the better set up football clubs in Sydney. They have a good new stadium in the heart of their community, not too far from their leagues club. They’ve had a reasonable amount of on-field success the last few years if we ignore the total and inexplicable collapse that was 2018 (which might explain the conservative projections). It will be worth keeping an eye out to see if the club an build on this and win two premierships this season to complete their five year plan.

If not, 2021 will probably be a tear down, followed by a firesale clearance, and then a rebuild.

Embed from Getty Images

nrl-pen Penrith Panthers

2020-pen-1

2020-pen-2

2020-pen-3

2020-pen-4

The numbers suggest a tough year ahead for the Panthers, with not much to look forward to. The projected team is only two Taylors per game better than the Bulldogs. TPR lists only four guys worth a damn, roughly the same as the Titans. The sims have ten wins and eleventh place on the ladder picked out for Penrith, a re-run of 2019.

My gut says Penrith could do a lot this year. The grand final might be a step too far but it wouldn’t surprise me to see them scrapping for a second week final nor would it surprise me if we wrote them off as finals contenders shortly after Origin. The risk is there is plenty of potential but not a lot of proven execution, as last year’s rookies become this year’s sophomores and the pack that was bulldozing the league a few years ago slowly being whittled away.

It might not matter if this year is a write-off for the Panthers if they can channel the experience into development, making this squad better in future campaigns. Ivan Cleary and a Gould-less Panthers will have to take better care of the next generation than they have done in the past.

Embed from Getty Images

nrl-ssr South Sydney Rabbitohs

2020-SSR-1

2020-SSR-2

2020-SSR-3

2020-SSR-4

I have a quiet confidence in Souths’ premiership aspirations but questions remain unanswered.

Souths’ spine is projected to be a full five Taylors per game better than Melbourne’s, which is next best, so it is little surprise that the Rabbitohs are the mostly heavily biased to their playmakers. Damien Cook is the keystone of the spine and has been the league’s most productive player by TPR two years running. The .200 barrier hasn’t been broken since Robbie Farah did it back-to-back in 2013 and 2014, years that the Tigers won a combined seventeen games. After two years of wrecking the league, have coaches finally watched enough tape of Damien Cook to put a lid on him? More pressingly, will Damien Cook turn up this postseason?

Latrell Mitchell’s mooted move to fullback returns him to a position he hasn’t officially played since his 2016 season for the Roosters. He put up an average TPR of .087 then. Mitchell is projected to carry through his (famously quite lazy) productivity at centre and bring .120 of production to fullback. I am loathe to make individual manual tweaks to my systems, so that seems like a bad assumption that is worth adjusting for. 30 pips of production at fullback is worth about 10 Taylors, enough to move Souths from fourth best squad to outside the top eight. Questions: will Latrell at fullback work? Will Latrell put his full back into working?

If they fail, it is not clear if the rest of the team will be able to pick up enough slack to keep the Bunnies in the premiership hunt. Adam Reynolds and Cody Walker form a potent pair. Cameron Murray looks ready to go up another level. But is the forward pack good enough without numerous Burgii? Edene Gebbie looked a little lost at the 9s, so who else is waiting in the wings if needed?

Is Wayne cooked?

Embed from Getty Images

nrl-sgi St George Illawarra Dragons

2020-sgi-1

2020-sgi-2

2020-sgi-3

2020-sgi-4

I didn’t want to make any specific predictions but wooden spoon, anyone?

It would be the first for Illawarra since 1989 and the first for St George since 1938. The reality is that Paul McGregor’s head is already on the chopping block. Since taking the reins, the Dragon’s class rating has dropped nearly 100 points, an untenable position and one no major league coach of the last two decades has been able to drag their team out before their time was up. No improvements to the roster, no improvements to coaching… wait, didn’t the Dragons sign Shane Flanagan as an “assistant”? That will be an interesting play and may well push the Dragons up the ladder.

The squad itself isn’t magic but should be better than last place. New signing Isaac Luke has always been a productive player but he will presumably be second fiddle to Cameron McInnes when he returns from injury, reducing the potential volume of work Luke could be doing. Indeed, St George Illawarra are extremely reliant on their spine to perform. While Hunt, Norman and McInnes have been productive, I don’t think they’ve been especially effective. The Dragons are also still searching for a fullback. Lomax may or may not be it.

If Flanagan really is the de facto, if not de jure, head coach, then he should be able to coax that performance out of the roster. If McGregor is still in charge, then a 5-0 start will turn into a 7-17 season and the cycle will begin anew.

Embed from Getty Images

nrl-esr Sydney Roosters

2020-eSR-1

2020-eSR-2

2020-eSR-3

2020-eSR-4

I’m quite comfortable assuming that the Roosters won’t go three in a row. They’re still good though, probably even still good enough for a minor premiership. A projected 15+ wins and the second best squad on paper is not going to have trouble reaching a preliminary final. The Storm are the only team superior on paper and they share the equal best class Elo rating.

When we talk about the trinity of rugby league – hungah, pashun and desiyah – do the Roosters still espouse these values? Cooper Cronk’s retirement and nominal replacement with near-rookie Kyle Flanagan is the kind of loss of edge that turns premiership winners into runners-up, as the Storm have amply demonstrated.

After all, it’s not just about production. Yelling at other players to get them organised is a rare and extremely valuable commodity. Luke Keary may have it but it will be the first time in his career that the 28 year old will be the elder of the halves pairing. But to put this supposed weakness into context, the Roosters will absolutely be a top four team come September.

Embed from Getty Images

nrl-wst Wests Tigers

2020-wst-1

2020-wst-2

2020-wst-3

2020-wst-4

The Tigers continue to defy my predictions of a wooden spoon to instead finish ninth. Last year, they really should have been eighth and the 12-12 record the year before should have seen them in the top eight. Basically, bad luck has kept them from breaking the NRL’s longest finals drought.

Still, you make your own luck. The Tigers were the biggest movers in the off-season and showed unusual astuteness in their acquisitions: Leilua times two, Adam Doueihi, Walters and maybe Harry Grant (.266 TPR in 2019’s QCup) will land.

The projections and the sims lock in a knife-edge battle for the Tigers to take that final step from ninth to eighth. Exactly 50% chance of making the finals, exactly 12.0 wins projected and an average finishing position of 8.6. I’m not ready to make them a lock but this is the best chance Wests have had in a long time.

All they had to do was spend their money wisely. Now they just need to lock down a home ground.

Primer – TPR

For the third season in a row, I’m changing the player rating system. We mourn the passing of Statscore (not really) and PPG (again, not really) as we slowly converge on to a system that I can take for granted and don’t have to refine any further.

The core of the system hasn’t changed. The proposition is that there are important and unimportant statistics and that counting the important ones provides information about players and teams and can be predictive.

PPG was useful, and development and application through 2019 demonstrated that:

The last one should be taught in universities as a perfect example of ringing the bell at the top. Sheer narrative power subsequently forced Pearce back to mean and Brown onto the compost heap.

The mechanics of PPG have been preserved through TPR. My biggest issue is that when I wrote about production (that is, the accumulation of useful statistics), I didn’t have any units to work with. I originally didn’t think this would be a problem but it would make some things clearer if I did have units. So I took a leaf from the sciences and landed on naming it after the man that could do it all, David “Coal Train” Taylor.

Embed from Getty Images

“PPG”, which was Production – and not Points – Per Game, doesn’t make much sense now, so that’s been punted and replaced with TPR, or Taylor Player Rating. There has been a substantial change in the way I’d calculated WARG in the primer at the start of 2019 and the way I calculated it in Rugby league’s replacement player at the end. The latter method is now canonical but the name is going to stick.

In brief, TPR and WARG are derived through the following six steps:

  1. Run linear regressions to confirm which statistics correlate with winning percentage. The stats get distributed in to buckets and we review the success of teams achieving those statistics. One crucial change was to exclude any buckets from the regression with fewer than ten games in it. We end up with tries, running metres, kick return metres, post-contact metres, line breaks, line break assists, try assists, tackle busts, hit ups, dummy half run metres, missed tackles (negative), kick metres, forced drop outs, errors (negative) and, in Queensland only, penalties (negative) as having significant correlations out of the data provided by the NRL.
  2. Take the slope of the trendline calculated in the regression and weight it by its correlation (higher the correlation, the higher the weighting). Through this weighting, we develop a series of equivalences between stats. The below is shows the quantities required of each stat to be equivalent to one try in 2020:
    equivalences
  3. Players who accumulate these statistics are said to be generating production, which is now measured in Taylors, and is the product of the weighting/slope multiplied by the quantity of stats accumulated multiplied by 1000. However, due to the limitations of the statistics, some positions on the field generate significantly more Taylors than others.
    Average Taylors per game by position (1)
  4. To combat this, the production generated each game is then compared to the average production generated at that position (averaging previous 5 seasons of data in NRL, 3 seasons for State Cup). We make the same adjustments for time on field as in PPG and then divide by 10 for aesthetic purposes. The resulting number is the Taylor Player Rating, or TPR.
  5. We derive a formula for estimating win probability based on production for each competition and then substitute in a winning percentage of .083 (or two wins in twenty-four games, per the previous definition of a replacement-level team) and estimate the amount of production created by a team of fringe players against the competition average. This gives us a TPR that we can set replacement level at. The Taylors created over and above replacement level is added to the notional replacement level team’s production and the increase in winning probability is attributed to that player as a Win Above Reserve Grade, or WARG. Replacement level in TPR for the NRL is .057, Queensland is .072 and NSW is .070. The career WARG leaders are currently:
    career warg
  6. Finally, we go back and check that it all makes sense by confirming that TPR has some predictive power (~61% successful tipping rate, head-to-head) and there’s a correlation with team performance (~0.60 r-squared for team season production against team winning percentage).

For a more in-depth explanation, you can refer back to the original PPG primer. The differences between last year’s system and this year’s are slight and, for most intents and purposes, PPG and TPR are equivalent. Some of the changes are small in impact but important.

The most obvious change is the addition of NSW Cup data to the Queensland Cup and NRL datasets. This was driven by my interest in assessing the farm systems of each NRL club and you can’t make a decent fist of that if you’re missing twelve feeder clubs from the picture. It will also allow me to better test talent identification in the lower levels if I have more talents to identify and to better set expectations of players as they move between competitions.

For the most recent seasons, TPR only uses past data to calculate its variables, whereas PPG used all of the data available and created a false sense of success. A system that uses 2018 data to create after-the-fact predictions for the 2018 season isn’t going to give you an accurate view of how it will perform in 2019.

Finally, projecting player performance into the future is a pretty powerful concept, even if the tools for doing so are limited. I went back and re-derived all of the reversion-to-mean formulas used in The Art of Projection. It turns out that the constants for the projection formula don’t change much between seasons, so this is fixed across the datasets for now. It also turns out adjustments for age and experience are different and largely useless under the TPR system, such is the ephemera of statistical analysis.

One application for projections is that I’ll be able to run season simulations using the winning probability formula and team production that will be able to measure the impact of including or excluding a player on the outcome of a team’s season. It may not be super-accurate (the projections have large average errors) but it will be interesting. I also like the idea that out- or under-performance of projections as an assessment of coaching.

Finally, to reiterate things that I think are important caveats: TPR is a value-over-average rate statistic, while WARG is a volume statistic. No, statistics don’t tell the whole story and even these ones don’t measure effectiveness. Yes, any player rating system is going to have a certain level of arbitrariness to it because the system designer has to make decisions about what they consider important and unimportant. I’m fully aware of these things and wrote 1500 words accordingly at the end of the PPG primer.

A thing I’m trying to do this season is publish all of my rating systems on Google Sheets so anyone can have a look. You can see match-by-match ratings for NRL and the two State Cups if that’s your jam.

The Challenge Cup in 2020

Despite being an avowed radical when it comes to rugby league culture – best encapsulated as “burn it all down and guillotine those that disagree” – I can see why the Challenge Cup has appeal. Running a format similar to that of the much more famous FA Cup, there’s something between the destructive chaos wrought by knock-out football and the ability for any club to participate no matter the background that has massive appeal.

Embed from Getty Images

A mix of apathy, ignorance, time slots and broadcast restrictions mean that I’ve never taken much interest in the Challenge Cup. That is, up until the Catalan Dragons won it in 2018. I discovered that the Challenge Cup had a long history of international club participation. Alongside Catalans, clubs from outside the United Kingdom, including Toronto Wolfpack, Red Star Belgrade, Longhorns of Ireland, Pia, Paris St Germain, Lezignan, Carcassonne, Toulouse, Villeneuve and Saint-Gaudens from France and Dinamo Moscow and Lokomotiv Moscow, have had a crack at the tournament. In this sense, the Cup provides an excellent platform for teams to chance their hand at a higher level than they may get at home, without the commitment of joining the RFL pyramid. But 2018 was the first time a team from outside had won the competition.

The Cup has faced declining interest in recent years and not just because a French team made the final. It remains to be seen how, with the likes of knock-out finals in the Super League and Magic Weekend becoming the sport’s pilgrimage, the Wembley final maintains relevance in the 21st century. As hosting rights become a viable revenue stream for rugby league, the RFL will need to find a way to make the finals, at least, saleable and that will likely involve leaving London and following the cash.

Challenge Cup Final Attendances

The other challenge is finding space for it in an increasingly crammed calendar. With Super League now contested over an unnecessary twenty-nine rounds plus finals and with the slowly growing re-emphasis on international football, the half dozen weekends required to contest the Cup are valuable real estate, especially given the opportunity cost and the wear and tear on players. In my vision for a future hyper-mega-league that the sport should be working towards, even with a shortened club season, I’m not sure I’d make room for the Cup as it currently stands. On the other hand, if Origin doesn’t work in Europe, then maybe this is their equivalent.

Current travails aside, the competition for 2020 is already underway. The second round completed, the number of non-league teams is being whittled down in anticipation of the League 1 clubs joining the fray in round 3. In the fourth round, the Championship clubs will join, followed by the bottom four of Super League in the fifth and the balance in the sixth round with finals to commence thereafter. In this, we take a look ahead at how this year’s Cup might unfold.

Rating the Challengers

You should be familiar enough with how we approach Elo ratings by now but just in case, we use two systems side-by-side. The first is a form rating, which measures short term performance by calculations based on the points margin of each game, and the second is a class rating, which measures long term performance based on small rating changes for wins and losses. You can review the history of the RFL class ratings in the post from earlier this week.

I used the class rating system as a basis for simulating the 2020 Cup 50,000 times. Historically, the class rating has been a comparatively poor predictor of match results but this system has the advantage of rating teams across leagues, saving us the trouble of trying to estimate differences between the leagues and applying corrective measures, as I did for the GRLFC Rankings. Obviously, this is an important feature when the fates can have clubs from different leagues mixing and matching.

Here’s how each team stacks up going into 2020:

2020 RFL CLASS

Worth nothing that the ratings of new teams, like Toronto and Toulouse, are a little underdone. They haven’t had as much time as the other clubs to beef up their rating, even though they’ve accumulated points about as quickly as humanly possible.

It’s probably going the way you’d expect

In 2020, we’re really only considering three obvious challengers:

2020 Challenge Cup

After a poor couple of seasons, the class rating doesn’t see Leeds as a member of the big four anymore, leaving St Helens, Wigan and defending champions Warrington as the favourites to hoist the trophy at Wembley.

Outside of that, two things surprise me about the results of this exercise. The first is how many teams are exactly zero chance of winning the Cup. Sixteen of the thirty-seven teams registered no simulated wins. Of these, half a dozen made it as far as the final in a handful of cases but failed to win any of them. That’s got to be deflating. Worse still are clubs like Skolars and Coventry, who never made the semis, or West Wales, who didn’t clear the sixth round. The simple fact is if you need consecutive low probability events to go your way, eventually they pile up into an impossibility. As an exercise, multiply 30% by 30% five or six times and you’ll see what the Raiders and Bears are up against.

On the contrary, and the second surprising thing, is how many clubs can still realistically challenge. The entire Super League has a shot, largely by virtue of playing fewer games with far better squads, but even the likes of the London Broncos and Leigh Centurions could be featuring in the latter stages of the tournament, if not fighting for the title. The aforementioned big three total up to a 61% chance with the implied flipside being that there’s a 39% chance of any other team winning. If Wests Tigers can win a NRL premiership, then anything is possible.

One of the myths underpinning the Challenge Cup, and indeed similar tournaments around the world, is that anyone can win it. Reality doesn’t bear that out – the last team from outside the top flight to win the FA Cup was in 1980 and I’m not sure if or when that has happened in the Challenge Cup – but it’s still a part of the competition’s narrative. Based on the simulations, you can see that, by far, the most likely pathway is for each division to be squeezed out one-by-one as the rounds progress. Any team may be able to participate but that doesn’t necessarily mean they have the capability to compete.

League Composition of Challenge Cup

Context from past cups

The chaos of knock-out football is one thing, but all of this is probabilistic. Ince Rose Bridge could very well win the Challenge Cup but it is extremely unlikely. I don’t think that’s controversial. I always say anything can happen and even the word “impossible” does a lot of heavy lifting in this context, but some events are very difficult to see occurring.

Here we have no hard and fast predictions – making them isn’t really my style in any case – but we can see from running the same exercise for the last two Cups how “good” the simulations are at making predictions. After all, the probabilities above are only as good as the mechanism for deriving them.

This time two years, the graph looked a lot like this:

2018 Challenge Cup

And the Catalan Dragons (2.5%) won. Last year:

2019 Challenge Cup

And the Warrington Wolves (12.1%) won. This should give fans some hope that we aren’t here to witness St Helens waltz their way to The Double but might seem to undermine the quality of the forecasting.

Looking closer, what we find across the two tournaments is that, for each team, the team exits at the round estimated to be the most likely about 40% of the time. About the same number again are only off by one round, either exiting one round earlier or surviving one round longer than their most likely outcome.

I think that’s pretty reasonable. The 2019 Wigan Warriors were the only team who had winning the Cup as their most likely outcome. It remains to be seen whether the 2020 St Helens side, who are similarly favoured, will manage to go one better than 2019 or if the likes of Huddersfield or Castleford will spring a surprise.

Tonga are still a long shot for the 2021 World Cup

After a dominating win over Great Britain, followed by a tough and exciting win over Australia, some extremely exuberant pundits decided that Tonga were the best nation in the world and worthy of Tier 1 status.

Putting aside the fact that a poor island nation of 100,000 is not capable of generating enough native talent to compete with Australia or England and so not at all suitable for Tier 1 status, I find it hard to believe that two wins is enough to reach the top of the rugby league pile. Then the IRL updated their rankings and put New Zealand at number one. It all got too much for me.

Fortunately, the draw for the 2021 World Cup restored some sanity, despite the slightly incongruous setting of Buckingham Palace and newsworthy presence of the Duke of Sussex. With 641 days to kickoff, I got a bit excited and looked at next year’s tournament.

Embed from Getty Images

Read more

Qui va gagner Elite 1?

Fortunately for almost all of my readers, I have reached the limit of the French taught to me in one semester of university in just the title. I’m still not sure if I should have used the definite article, so this post will be in English but it will be about French rugby league. This is fortuitous, as the Elite 1 Championship kicks off with a Magic Weekend in Carcassonne on very early Sunday morning.

I think rugby league nerds, like me, find the idyllic notion of French rugby league to be very appealling. France is cool and exotic, particularly to Australians, in a way that rugby league generally is not. I think adopting a substantial portion of the French rugby league vocabulary would give the sport a much needed touch of class: talonneur for hooker, pilier for prop, demi for half and so on.

The unfortunate reality is French rugby league has the same cultural notions as in the Anglosphere but this is masked by an impenetrable veil of français and a lack of money, quality and prestige, which leads to minimal coverage.

Embed from Getty Images

France was the fifth nation to join the rugby league fraternity in the mid-1930s. Clubs are located predominantly in the Occitanie, a region home to roughly five million people that has very approximate parallels to Spain’s Catalonia. Armed with a per capita GDP that is only bettered by Australia out of rugby league’s economies, France has been given all the advantages to succeed and failed to capitalise.

We can blame the Vichy government as much as we want but France’s golden age on the international stage occurred in the early 1950s, well after liberation in August 1944. Instead of developing into a force that should now be on par with at least New Zealand, France has slid inexorably backwards due to a lack of interest, a lack of investment and the rise of professional rugby union. It took forty years to simply win the right to refer to the game as rugby.

France finished 0-3 at the last World Cup and put up a 1-2 performance at the 9s in October. These dismal results have been made worse still by the current Chanticleers tour, where captain Jason Batieri walked away, the national side absent fourteen pros was dismantled by the Junior Kangaroos and then racist comments allegedly made by the chair of the Fédération Française de Rugby à XIII (FFR) were leaked. A lawsuit is in the offing. The next world ranking update would probably see France slide down from sixth but Samoa are somehow just as impotent on the field.

This should be a wake up call. Many of us hope that France will, at some point, turn it all around. There is absolutely no basis for this belief. At least, Theo Fages won this year’s Super League grand final in the halves for St Helens and there are plenty of Frenchmen plying their trade for the Catalan Dragons. Few, if any, have made the leap across to the NRL.

Le championnat

Despite this, I think we all benefit from learning about the wider world of rugby league. I didn’t know anything about the Elite 1, other than it existed and I own a Palau jersey, until a few weeks ago, so I’m going to share what I’ve learnt from Wikipedia and the excellent French rugby league resource, Treize Mondial.

The top domestic premiership is the Elite 1 championship, which comprises nineteen rounds of the regular season and three weeks of barrages (finals). The season starts in November, with the top six progressing to finals in June. Elite 1 sits atop the French rugby league pyramid, with optional promotion and relegation to Elite 2 and the National Divisions split into conferences underneath. Clubs compete in the Lord Derby Cup, the French counterpart to England’s Challenge Cup. Elite 1 is semi-pro but is considered below the RFL’s League 1.

A local TV deal has been struck for the upcoming season. Some games will be available for streaming live on viaOccitanie at what are, quite frankly, ungodly hours to be awake in Australia to watch fourth division European rugby league.

There are ten clubs competing in Elite 1 for the 2019-20 season, with eight located in Occitania, Villenueve in Nouvelle-Aquitaine and Avignon on the border but officially in the region of Provence-Alpes-Côte d’Azur.

occitanie clubs.PNG

Only Toulouse can claim to be a city by global standards. While I am usually critical of rugby league’s tendency to embed itself in suburbs and small towns at the expense of focussing on larger metropolitan areas, there are some fairly substantial differences between Pyreneean France and western Sydney.

albi.png

el1-alb ALBI RUGBY LEAGUE

Les Tigres

Somehow rugby league has managed to find itself, even in France, in a coal mining town. Albi was one of the first places that coal extraction was attempted on an industrial scale in the nineteenth century. In 2010, the cité épiscopale was UNESCO Heritage listed, including the 13th century Sainte-Cécile cathedral and the Berbie Palace on the banks of the Tarn. The irony being that Albigensian, the demonym for the people of Albi, is most closely associated with heresy. Catharism touched many French rugby league towns during the Middle Ages.

As much as coal mining is a recurring theme, every rugby league competition seems to have a club nicknamed the Tigers. Les Tigres have five national championships to their name but it has been over four decades since the last in 1977. Their one and only cup came in 1974. Since then, in 2008, the original Albi club went bankrupt and dissolved, ending 74 years of history. The new Albi club was reinstated in the second division, rising to the first for the 2015-16 season, where they have finished mid-table each season.

avignon.png

el1-avi SO AVIGNON XIII

Les Bisons

Did you know that there was for a time two Popes? It happens more often than you would think but in this specific case, the Western Schism in the fourteenth century saw a Pope base himself out of the city of Avignon. This is possibly the only rugby league city on the face of the earth that can boast having been home to a Pope.

However, we are more interested in their treizistes. The Avignonais side play in blue and white as the Bisons. Avignon have had some success, including five cups, the most recent in 2013, and their first and only Elite 1 title in 2018. Avignon advanced through the semi-finals, despite losing 23-16, after St Esteve Catalan were disqualified for fielding ineligible players.

carcassonne.png

el1-car CARCASSONNE XIII

Les Canaris

Carcassonne is the only place out of the ten listed that I’ve actually visited and the only one with a popular board game named after it. Carcassonne boasts a bunch of cool things: a walled medieval city (check out the torture museum), probably the nicest jersey in the Elite 1, and it was the home of Puig Aubert, star of the 1950s touring French national team that beat Australia in a Test series. His statue adorns the Stade Albert Domec, the city’s rugby ground.

The Canaries, known that because of the striking yellow colour of their kit and logo, have fifteen Coupes de France, the most of any club, and eleven national titles, equal most with pre-merger Catalans. Carcassonne are the reigning Lord Derby Cup holders, beating St Esteve Catalan 22-6 in front of a crowd of 4,000 in Perpignan.

lezignan.png

el1-lez FC LEZIGNAN XIII

Les Sangliers (Wild boars), Les Moulins

Lezignan-Corbieres is a fairly typical French town of around 11,000. It has a cathedral, wine making facilities, municipal baths and convenient access to the motorway. It also boasts a good rugby league team. Les Sangliers have the fourth most national titles, with seven total including four in a row from 2008 to 2011. This is the longest streak of any club, a feat shared only with the 1982 to 1985 XIII Catalans. Six cups also sit in the trophy case, with the most recent silverware won in 2015.

limoux.png

el1-lim XIII LIMOUXIN

Les Grizzlies, Les Blanquetiers

Limoux is probably more famous for its local sparking white wine than its football team. Blanquette de Limoux is an appellation d’origine contrôlée, meaning that, like champagne, true Blanquette de Limoux can only be made from certain grapes, grown in a particular area and processed in a specified way. The wine is the centrepiece of the Carnival of Limoux, which bills itself as France’s longest festival.

Speaking of those fifteen, Limoux have been one of the more recently successful teams, winning back to back national titles in 2016 and 2017, adding to their sole previous championship from 1968. In the cup, Les Blanquetiers have been more Poulidoresque, winning two in 1996 and 2008, but also losing ten grand finals in their sixty-eight year history, the third most of any team.

palau.png

el1-pal PALAU XIII

Les Broncos

Palau (pronounced “pah-lo”) is not to be confused with the Pacific island nation (pronounced “pah-lau”) or any number of small French towns of the same name. The Broncos are from Palau-del-Vidre at the foot of the Pyrenees, population 3,226, approximately twenty kilometres south of Perpignan. Fun fact: Palau hosts an international glass festival.

Palau have only recently been promoted to Elite 1, taking the step up in 2013 after dominating Elite 2 for the previous five seasons. The team has never won the national championship or the Lord Derby Cup. The 2018-19 season was the Broncos’ best to date, finishing seventh.

perpignan.png

el1-sec ST ESTEVE XIII CATALAN

Les Baby Dracs

XIII Catalan were founded in 1934 in Perpignan. In 1965, a mere six kilometres away, local rivals Saint-Estève were founded on the other side of La Têt. Catalan won eleven national championships and Saint-Estève six. In 2000, the clubs merged into a Perpignanais super-club, then called Union Treiziste Catalan. UTC were granted a licence to join Super League for the 2006 season, ahead of Toulouse and Villeneuve. They’ve remained there ever since, winning France’s first Challenge Cup in 2018.

In France, UTC continued in Elite 1, winning the 2005 and 2019 championships and the Lord Derby in 2016. The club was renamed St Esteve XIII Catalan and plays out of the Stade Municipal in Saint-Estève. The Baby Dragons serve as a feeder club for the Super League club.

st gaudens.png

el1-gau RC ST GAUDENS XIII

Les Ours (Bears)

The rugby league club was founded in 1958 but the town traces its history back to the Roman era. Saint-Gaudens is named for Gaudens, a fifth century martyr who was decapitated by Visigoths and who I cannot find any other details about. The Route d’Occitanie, a professional cycling race, visits Saint-Gaudens often and has been won by the top names in the sport.

Saint-Gaudens XIII have four French national titles – 1970, 1974, 1991 and 2004 – and three cups – 1973, 1991 and 1992. Member troubles hit Saint-Gaudens in 2011 and the team was forced to sit out the 2011-12 season. The club returned the following season in Elite 2 before rejoining Elite 1 in 2016. Les Ours have yet to qualify for the finals since their return.

toulouse.png

el1-to2 TOULOUSE OLYMPIQUE ELITE

Les Broncos

Toulouse is the largest city represented in Elite 1, with a metro population of 1.3 million. The senior team are currently in their second attempt to climb the RFL pyramid. The first attempt saw them spend three seasons in the Championship from 2009 to 2011. Following their fifth and sixth national titles in back-to-back years and a cup/championship double in 2014, Toulouse rejoined League One in 2016. Olympique immediately secured promotion to the Championship, where they have remained since the 2017 season.

In 2016, to keep a Toulousain presence in the top tier of French rugby league, the then Toulouse Jules-Julien Broncos were taken over and promoted from Elite 2 to serve as the reserve team. The consequence of this is that the junior Toulouse side is not very good and will require some time before they are able to challenge for Elite 1 honours.

Depending on which source you look at, some still refer to the team as the Broncos, as a nod to the previous incarnation, but the club seems to prefer Toulouse Olympique Elite.

villenueve.png

el1-vil VILLENEUVE XIII RL

Les Léopards

The Leopards hail from Villenueve-sur-Lot in the Nouvelle-Aquitaine region, just a stone’s throw over the border from Occitania. This is not to be confused with the other ten thousand French towns named Villenueve. The bridge across the river gave the town some prominence during the Middle Ages, as one of the few crossings of the Lot. In modern times, I personally don’t believe it but French Wikipedia says Villenueve-sur-Lot was a hot spot for jazz.

Villeneuve are one of the more historically successful French clubs, having won the national premiership and the Lord Derby Cup nine times each. Villeneuve is the third on the all-time list for both competitions, behind Catalans and Carcassonne, and both were last won in 2003. This was the end of an exceptional five year run for the club, including three championship/cup doubles. A bankruptcy in 2005 followed, with a failed bid for a Super League licence in 2006, which eventually went to Catalans.

Ratings

Statistics are pretty hard to come by but I did manage to dig out results for 2016-17 season onwards and tries scored for the 2017-18 and 2018-19 seasons*. From that, I was able to construct form (short term) and class (long term) Elo ratings and Poseidon ratings.

The ratings as at the end of the 2018-19 season are:

m23-ratings-2019.PNG

While I can’t do some of the high level analysis that I do for the NRL, with class ratings we can at least set the Disappointment Line. This is calculated by determining how many games a team of that class rating will win against league average opposition. If a team wins more regular season games than the line set, their season is officially pas décevant.

Note that the Elite 1 uses a bonus point system. Three points are awarded for a win and one point for a loss of twelve or less. Palau finished last season with just five wins but amassed eleven bonus points out of fourteen losses. On that basis, I’ve also set a line in terms of competition points (three times the line), which might prove a bit on the easy side to beat.

m23-disappointment-2019.PNG

With the points difference from last season, we can measure each team’s fortuitousness. When teams outperform their Pythagorean expectation, they typically (but not always) revert towards the mean in the following season and vice versa. The greater the difference the actual and Pythagorean wins, the stronger the reversion typically is.

m23-fortune-2019.PNG

La saison à venir

Even with these simple ratings, we can do simulations. I’m in the process of rebuilding all of my datasets to be more organised and convenient, so I’m going to use the Elite 1 season as a test for some changes.

Like the Stocky, these sims are Monte Carlo simulations. The sims are “cold”, where ratings do not change within the simulation, whereas previously the sims were “hot”. This greatly reduces the amount of computational power required and there’s philosophical reasons for preferring cold over hot.

To test the ability of the sims, I used the ratings from the end of the 2017-18 season to see how well they predicted the known outcomes of the 2018-19 season.

2018-19-m23-sims.PNG

These simulations are never going to be able to see what we don’t know, especially with the limited information at our disposal for Elite 1. There’s always surprise packets in every season. The important thing is to get the other teams that aren’t surprise packets roughly correct.

Considering that we were working off the 2017-18 ratings as inputs to the 2018-19 season, with no considerations for team changes or any other background noise, that level of accuracy is not too bad.

The mean absolute error (MAE) was 2.1 wins for both form and Poseidon sims, with a successful prediction of the eventual champion, St Esteve Catalan. The sims were too down on Carcassonne and Villenueve, while overly expectant of defending champions Avignon. A weighted average of Poseidon and form (ratio 2:1) delivered the lowest MAE at 1.9. Introducing the results of the class sim only increases the error.

Trialling this in one very specific instance is obviously problematic (I can only assume this what the French almost definitely don’t call le bias de petit montant) and I won’t be extrapolating this particular method without testing on multiple competitions across multiple seasons but it’s a start.

Looking ahead to the upcoming season, plugging in the 2018-19 ratings to the 2019-20 draw gives us the following:

2019-20-m23-sims.PNG

The 2019-20 season is looking far more open than the 2018-19 season did, even if the simulations belie how close Les Canaris came to a double. Whereas the Baby Dracs were outright favourites in premiership percentages, if not regular season wins, this year we have at least three front-runners with two more close behind. Treize Mondial have named Carcassonne their pre-season favourites for the title (assuming the mayonnaise comes together), and I’m inclined to agree, although St Esteve Catalan and Limoux look to be in the mix.

Lezignan and Albi look good for the top six. Palau, who are primed to take a step forward, and Villenueve will likely scrap out for the final place in the barrages. Unless there’s a surprise resurgence, we’re expecting Avignon, St Gaudens and Olympique to continue to struggle. Their ratings at the end of the last season do not hold much promise. Avignon, having signed Jack Payne from Mounties, probably have the most upside potential.

I don’t plan to keep this up to date with every round but will likely check in on progress and update ratings once a month or so. It will be an interesting follow.

* More results would be better if anyone has them. French Wikipedia doesn’t list the results in order, which is not helpful, and the FFR website is a mess. Contact me if you have something you think might be useful.

The Art of Projection

Over the last month, we’ve been looking at rating players using a metric called Production Per Game, or PPG. We’ve used it to find players at the higher end, justifying million dollar salaries, and at the lower end, identifying fringe first graders.

The tricky thing about rating players is determining what information from the past can be used to project the player’s performance into the future. I hope it’s obvious why this might be interesting.

Within a player’s career, there is a noticeable amount of variation from season to season. On average, players get two pips (one pip is .001 of a PPG rating) worse, although the actual range is lies between improving by 96 pips or losing 86 somewhere (standard deviation of 24 pips) from season to season.

Embed from Getty Images

Read more

Rugby league’s replacement player

Last week we looked at valuing players, specifically Tom Trbojevic, Daly Cherry-Evans and Addin Fonua-Blake, by their contributions to Manly’s winning ways. There were two problems though:

  1. How do we deal with someone like Marty Taupau? He’s played all of the games this year so we can’t find an understudy to compare him to.
  2. What if your understudy is pretty good? Cherry-Evans looks like he is comparatively less valuable than his colleagues because Kane Elgey is about 20% more productive at halfback than Brendan Elliot or Toafofoa Sipley were at their respective positions.

We should establish a fixed benchmark to compare players against.

Baseball uses the concept of a “replacement level“. I touched on it briefly last week but the rugby league equivalent would be a top level reserve grader who can be acquired for the league’s minimum salary ($105,000 in 2019). By definition, the replacement level player provides the right value for money for that salary. If a player provides less value, they shouldn’t be in first grade because they can’t justify their pay packet. The trick is to find players who provide more value and then pay them accordingly to win games for your club.

Embed from Getty Images

Read more

« Older Entries