Tag Archives: sports

A Shallow Dive into the 2021 Brisbane Broncos

The Broncos didn’t finish last. They didn’t even finish second last. The Broncos wound up in fourteenth, 7-17 with a -249 points difference. Their points difference was only third worst in the league. The attack was also third worst, somehow behind the seventh placed Knights, and the defence an astonishing fourth worst. 2020 was so bad that all of these are actually positive things. As a bonus, the retiring has-been didn’t use the final game of the season as an opportunity for a gender reveal.

The Victory Lap

From the pre-season deep dive:

The Broncos aren’t going to get the spoon again. Sorry, it’s just not how football works. They probably will not make the finals but there is literally only a single direction that the sport’s biggest franchise can take coming off a 3-17 season that included a 59-0 flogging, somehow breaking the previous year’s record flogging.

*tweet of Brodie Croft playing halfback pre-season*

Never mind. It looks like the Kevolution might take a little longer than initially anticipated.

While I didn’t have particularly high hopes for the Broncos, I thought they might improve somewhat on last year… So be it, although if Walters can’t get it sorted, the squad will have to be scattered to the four winds for their own good and Brisbane will have to start again with a bunch has-beens while the farm system replenishes… The alternative is teaching the younger players to play eighty minutes of football and winning some – it doesn’t even have to be a lot! – games.

Brodie Croft’s spectre has finally been vanquished from Red Hill, off to Salford per my prophetic meme.

The rest more or less came to pass. The Broncos were travelling poorly, with a roster that was worse than 2020’s and results that were better, and started offloading almost whoever they could get rid of. Matt Lodge, gone. Tevita Pangai, gone. Tom Dearden, gone. Reece Walsh, gone. That last one was probably not the smartest decision but we can blame the old front office, who are almost all gone as well, for that.

Then they started getting better and in a prime example of nature healing, debuting children again. And they won some more games – it wasn’t even that many! – and finished above the Cowboys, which is all we ever really wanted.

What happened

Let’s check in and see how our new coach is doing.

It’s an interesting strategy to crash the team even faster than your most loathed predecessor, especially if one wants to retain one’s job.

The rule of thumb is that once a coach loses 50 points, irrespective of their starting point, they end up losing their job sooner or later. It is extremely rare to be given the opportunity to turn it around after that kind of performance, although there are exceptions (see Catalans below). Walters’ sits at -31, having bottomed out at -38. While there’s currently no real pressure, a bad start to next season will almost certainly seal his fate. He will have to work extremely hard to break even and even that might not be enough to get a contract extension. A career season is what’s required.

Now let’s check in with how he went during his only other head coaching appointment.

I see. One could also use this chart to compare Walters’ performance to that of, say, Trent Robinson. I’m sure it will be fine.

There’s always next year

Adam Reynolds. Kurt Capewell. Jordan Pereira. If at least two of those names don’t fill the Broncos’ rivals’ hearts with dread, then I don’t know football.

The Broncos appear to have made the right moves to re-balance the roster but then that’s been ostensibly the case the last couple of off-seasons. The promise of high performance and, even laughably, a premiership window dissolved into the mirages they always were under the twin tyrannies of incompetent coaching and incompetent administration. I’ll believe that the Broncos have improved when I see them improve. Rationally, it should be the case but irrationally, I’ve sat through 58-0 and 59-0 and a combined 10-34 record over those seasons and, like dental surgery, I’d honestly prefer not to have to do that again.

Ben Ikin and Dave Donaghy should be good for it and Kevin Walters might not be. It’ll take time as every NRL rebuild project over the last decade has generally shown. Finals might be on the cards if everything goes well but I suspect just getting out of the basement will be achievement enough. Best case scenario is that football will go to pot again and a 10-14 record will be enough to qualify for the post-season with the Broncos on the right side of it this time. Worst case scenario is probably more of the same as this year which will be bearable but might only speed up my disengagement with the sport. There are so many other sports teams out there.

2021 Super League WIP Report

We’re about a third of the way through the Super League season (maybe? Depending on covid and weather, I suppose) and it’s time to review how each team and player are performing. We’ll be looking at this using the following analytical tools:

  • FormElo ratings that reflect short term performance.
  • Production – The accumulation of valuable work on field, as measured by statistics that correlate with winning. Production is measured in Rismans.
  • Disappointment Line – The minimum number of wins for the season to not be considered disappointing by fans, as calculated by the pre-season class (long term performance) Elo rating.
  • 1st order winsPythagorean expectation calculated by points for and against.
  • 2nd order wins – Pythagorean expectation calculated by SCWP.

There’s more detail at How It All Works.

I won’t be providing much analysis for individual teams or players, instead preferring to let you do the work for your favourite team or player with the context about how the tools work provided.

WIP graphs

Comparing win percentage of the Disappointment Line, actual wins (0th), Pythagorean wins (1st) and 2nd order wins.

Note that the zero to one scale for winning percentage can make significant differences look minor; a .100 change in winning percentage is worth up to 2.5 wins at season’s end.

Some of the games have no or incomplete stats, so the second order stats might not move in line with the first order or winning percentage. Some teams haven’t played all of their games between round 1 and 11, in which case the lines do not change through this round. Statistics do not include Castleford’s forefit to St Helens.

Significant divergences between actual and Pythagorean wins are usually indicative of mean regression in future. That is, teams underperforming will improve their actual win percentage and vice versa so that the two numbers tend to converge. However, there’s always one or two that manage to avoid this convergence. We call them lucky/unlucky, depending on the direction that they miss and the ladder is currently a mess.

The outlook for each team:

  • Actual wins understating year to date performance / potentially tending better – St Helens, Leeds, Huddersfield, Leigh
  • Actual wins overstating year to date performance / potentially tending worse – Catalans, Wigan
  • Actual wins reflective of year to date performance – Warrington, Hull FC, Hull KR, Castleford, Wakefield Trinity, Salford

For teams with less than .500 percentage, 2nd order wins is always better than actual wins and vice versa. The trick here is to not focus on this specific value per se but to look at it as a prediction for next year’s performance. The next season’s performance for most teams will fall within one standard deviation (plus/minus .130) of their 2nd order win percentage for this season.

Form

Form Elo ratings at the end of 2020 and the end of round 11

1st Order Wins

Winning percentage estimated by Pythagorean expectation of actual points for and against

2nd Order Wins

Winning percentage estimated by Pythagorean expectation of SCWP for and against

Player Leaderboards

Rismans

Rismans are the unit of measure for the amount of valuable work done (production), equivalent to Taylors in Australia. Due to the state of the dataset, not all games and appearances will have been captured.

The career leader (2017 – now) for total Rismans is Jermaine McGillvary with 4402. The single season record holder is Jackson Hastings in 2019 with 1768. 2020 was topped by Bevan French with 1114.

Rismans per game

As above but averaged per game (minimum 5 games)

The career leader (2017 – now) for Rismans per game is Peta Hiku with 74.1 (minimum 10 games). The single season record holder is (minimum 5 games) Craig Hall in 2018 with 76.7. 2020 was topped by Ash Handley with 68.0.

BNE2.2: The Gemba Report

Previously, in our series on Brisbane expansion:

*****

The short version is that twelve NRL clubs have chipped in to get a study (that the ARLC probably should have commissioned themselves) to look at the impacts of expansion on the competition. They have engaged Gemba Sports Group to do this and The Australian has actually published the report in full (not paywalled).

That the existing NRL clubs, who may have to face another competitor and who stand to have their ARLC voting power mildly diluted and potentially lose future funding, commissioned the report to make a specific point needs to be kept in mind. That I want to make a different point probably also needs to be considered.

This has not generated much interest, largely because the content has been paywalled at The Australian and the articles are largely made up of quotes from the heads of Manly, Penrith and the Gold Coast talking their own book.

It is common knowledge that the player talent is not there. If there is enough talent for another team, you wouldn’t have the Bulldogs trying to get (Matt) Burton or the Wests Tigers wanting Dane Laurie. It doesn’t pass the pub test.

Brian Fletcher, Panthers CEO, 26 March 2021

On that basis, a club should be worth $50m. So what I am saying is … if a club wants to come and have a seat at the table, they have to be stumping up.

Scott Penn, Manly owner, 1 June 2021

But if expansion is done quickly at the cost to the game and the existing clubs, then we need to pause and think it through.

Steve Mitchell, Titans CEO, 23 June 2021

The basic breakdown of the report is that there’s four “sprints” (?):

  1. Impact on Fans
  2. Impact on Revenues
  3. Impact on Operations and Football
  4. Investing in NRLW and Participation

The executive summary covers the most important information over about the first 40 pages or so and then goes into more depth in the remaining 160 pages of the report.

My inclination is not to document every item of interest. You can read the report for that. My goal is to look at the arguments being made, not as the words of Gemba Group but as those of the incumbent NRL clubs, to see if they hold water.

Sprint One: Fans

This sprint boils down to the following key points –

  • The Queensland market has had declining interest in NRL over the last few years
  • The Queensland market is pretty well saturated, as measured by self-declared interest in the NRL and self-declared support of existing NRL teams
  • Therefore, for a new club to survive, they will need to convert the small group of uncommitted NRL fans (approx. 22,000 in south-east Queensland) and cannibalise the existing fanbases to get to the 150,000 or so the smallest NRL clubs have to sustain them

Fundamentally, that’s a more or less sound argument and not one that I dispute, other than to note that the declining interest in NRL in Queensland over the last few years can be easily correlated with the performance of the three Queensland clubs, having peaked in 2015 with the all-Queensland grand final.

There is, however, some issues with the numbers. This slide shows what Gemba estimates to be the fanbase of each team.

pp13

They’ve got some of the clubs right, although I have no idea about the accuracy of what they’re measuring, but there’s some obvious exceptions that don’t really square with the other big source of fan data: TV ratings. The idea that the Roosters are the third most popular club in the NRL is laughable, as evidenced by any other data point you care to name. The Storm are big – and bigger than most would credit them – but not Broncos big. The Rabbitohs are not less popular than the Raiders.

Some of the discrepancy can be explained by using point in time data in 2020, rather than over a longer time frame and factoring in other sources of information to synthesise a more wholistic view of the size of fanbases.

More of the discrepancy can be explained by the difference between what people say they do and what people actually do. The point of a new team is to get eyeballs on TV and, to a lesser extent, attend games and buy merchandise. Having people identify as fanatics is great but does not necessarily align with what the NRL are looking for. It might also mean their methodology has misestimated the number of casual or uncommitted fans, which changes the business case if some supposed fanatics are actually more malleable in their allegiances.

Half of NRL Fanatics that reside in Greater Brisbane currently support the Brisbane Broncos

The risk of fan cannibalisation is not limited to Queensland Clubs, with there being more NRL Fanatics in candidate locations that support interstate teams than the Cowboys or Titans

pp16

Cannibalising the existing Broncos fanbase is inevitable and perhaps even desirable. After all, the fans of the Cowboys came from somewhere and the same applies to the Titans. Indeed, the Broncos fanbase is made up of people who either were or would have been self-declared fanatics of the BRL clubs. People can change.

Gemba notes that they have not undertaken the bespoke research – code for the clubs didn’t stump up the fees – required to draw any conclusions here.

The Broncos have been awful of late and, as evidenced by the increasing ratings and attendances of the Lions now and in the early ’00s, the people of Brisbane prefer to follow a winning team. If the new team comes in and is successful, they will find fans (if they aren’t successful in the medium term, they will be another Titans). If the fans come from the Broncos, the Broncos have fans to spare. Having one team in the league loom commercially over the rest has its drawbacks, specifically forcing the country to watch a terrible Broncos team get pasted on free-to-air most weeks, lest the commercial base of the sport collapse.

The most important point this section could make is actually irrelevant to expansion but incredibly important to the future of rugby league (so will be ignored), which is the slowly dwindling interest as other global behemoths invade the market, especially among younger fans. These globalisation risks are something I highlighted earlier in the year.

Sprint Two: Revenue

Gemba have suggested that a seventeenth team would add twelve matches a season and modelled revenues improvements based on that. The total audience would grow but its likely, in their view, that average ratings per match would remain stagnant.

They seem to be working from the assumption that their estimate of the fanbase size is correct but the ratings don’t correlate well.

pp71

I think they have this backwards. The ratings are the goal and the number of fanatics is largely immaterial except as a means to generate ratings. In fact, I think ratings give you a better estimate of the fanbase size than whatever data they’ve used. Gemba probably would have been better spending the money on buying an Oztam subscription than commissioning a survey.

Their assumption is that the new club will have a small fanbase is probably valid and consequently will generate minimal ratings (similar to that of Penrith or Cronulla), excepting for the newly created Queensland derbies and additional matches, and this underpins their subsequent analysis. Given that averages remain about the same and the increase to total audience is driven by the additional matches, the implication is that another team is not required to play more games to generate that revenue.

This is not a view I subscribe to. I think it’s likely that a new Brisbane team will have reasonable-to-good ratings because people in Brisbane like watching football and want skin in the game, even if it’s not their preferred team. Most weeks, the new Brisbane team will play in lieu of a below average team and raise the average ratings for each round and that, even without extra games, average and total ratings will rise in the short to medium term.

I’d be surprised if broadcasters or the players’ association agreed to a longer season. While there is scope to amend the value of the broadcast deal based on expansion (but who knows whether or how this will be triggered), I think any uplift in average and/or total ratings would only accrue to the broadcasters under the current deal. I talked about this in BNE2.1.

Overall, Gemba forecasts small benefits for the league as a whole. They do this in a logical way with some evidence but I think they’ve understated the upside. You can make up your own mind.

Sprint Three: Operations

In summary –

  • NRL costs are going up, which is largely because the new team will be entitled to the central distribution plus some additional costs to move the morass of Sydney clubs to south-east Queensland more frequently (estimated by the report at $345,000).
  • Based on the AFL experience, promoting existing clubs into established markets results in league-average contributions to the new clubs. Starting new clubs from scratch in new markets requires significantly more investment.
  • It costs about $23.6 million per year to run an average NRL club, so a new club needs to find about $10 million a year over their distribution to break even.
  • A new mouth to feed means less sponsorship revenue for everyone else.
  • With a new club increasing the total salary pool from $147 million to $156 million, salary inflation will result in putting existing clubs under pressure to manage their lists.

Cap management is a challenge that clubs currently face, but may be compounded in the short term with the addition of a new team

pp139

It would be a shame if clubs had to do their jobs better, a wholly unprecedented situation not found anywhere else in society.

After that, it starts to get maddening.

If you’re not familiar with corporate risk analysis, it’s largely about identifying qualitative concerns and less about quantitative analysis and that’s how this sprint largely runs.

There will be dilution of talent as a result of more players entering the league, but there are concerns that the quality of talent will not be up to NRL standard.

pp26

There’s no evidence for this, simply the assertion which is backed by a logic but its a logic that suits the needs of the existing clubs. Where benefits are noted for balance, there are considerably fewer bullet points when compared to the risks.

In reality, if there’s a shortage of talent, it’s in the club offices, not on the football field. It is particularly galling to hear these sentiments echoed by the Panthers, while they were undefeated in first and second grade football at the time of the report’s writing, forcing Matt Burton to either play in reggies or as a first grade centre, because they are just so overladen with talent. That he can go elsewhere for an opportunity to play first grade in the halves is considered a failing of the current system, which is baffling.

There is a concern amongst interviewed clubs about the quality of the players that would be entering the league to fill the additional spots created by a new team. However, there is a general sentiment that there are enough players to fill an additional team of 34 players.

pp143

The corollary is the fate of the 2019 Sunshine Coast Falcons and how other NRL teams just allowed Melbourne to keep their most talented members, which included at various points Harry Grant, Justin Olam, Nicho Hynes, Tui Kamikamica, Tino Fa’asuamaleaui and Ryan Panehuyzen, without a fight. Instead, Melbourne’s rivals preferred to wait until their market value had risen beyond cheap before showing any interest.

Feedback from current NRL clubs suggests there is currently a lack of incentives and structures in place for clubs to prioritise the development of junior talent. If a new club does not take a development mindset towards building junior talent, there is a risk that current Queensland based clubs will be negatively impacted as they may lose junior talent for little compensation

pp161

For mine, it’s clear that the talent is there but the scouting and development structures are not, which is ironic considering the report states that clubs are concerned that there’s no reward for developing talent. It pays huge dividends because the rest of the league won’t scout outside their postcode. This self-serving analysis is not a useful contribution to the discourse.

It may become increasingly more challenging for a new club to establish talent pathways, as existing teams continue to grow their presence in Queensland from a talent identification perspective (e.g. Melbourne Storm’s affiliation with the Sunshine Coast Falcons and Brisbane Easts, and the NZ Warriors’ affiliation with Redcliffe Dolphins)

pp34

The report notes that clubs who rely on Queensland talent will have to spend more to maintain their pipeline. There are three unaffiliated clubs in the Queensland Cup, two of which will sign up with any NRL club that will have them. Of the three clubs that might come into the NRL, they all exist in some form in QCup, meaning the Storm might have to shift partners from Easts (or rely solely on Sunshine Coast), the Warriors might re-enter the NSW Cup with their own side (which might happen anyway) in lieu of using Redcliffe and the Jets would obviously link up with their currently unaffiliated namesakes. There are still substantial opportunities for more clubs to utilise Queensland in their pathways.

The report rightly identifies that there are unanswered questions about how the new team will operate. Will they be given cap dispensations? How will the draw be structured? What’s going to happen to distributions? That’s on the NRL to address. But if the result of expansion is player salaries going up and additional playing and coaching jobs being created, this is presented in the executive summary as a bad thing (the body is more balanced). It is, of course, bad if your business is running a football club but it is less so if you are ordinary fan who simply wants to see the people responsible for playing the game rewarded for their efforts.

Outside consultants are unlikely to publicly publish a report which states that their clients could and should actually be a lot better at their jobs. What’s irritating is that this has been relayed, verbatim, with little to no analysis or interpretation or context, by the media. Fortunately, it’s been paywalled so no one else seems to have picked up on these extremely lazy talking points.

Sprint Four: Women/grassroots

The final sprint talks about the current risks and opportunities for the women’s game and grassroots. This seems like a non-sequitir, given the report is notionally about adding another NRLM team in Brisbane, but there is an opportunity cost. The money that’s invested in a new team is money that can’t be invested in the NRLW and women’s game.

To a level that’s true. Of course, the unstated assumption is that the existing clubs’ central distribution is to be left untouched, even though a large number of clubs do not generate sufficient commercial return to justify such a generous grant and their existence is largely subsidised by the Warriors, Broncos and other big clubs. Perhaps we should cut the men’s club distribution by $1 million a year and reinvest that in the women’s game? We could then have our cake and eat it. That this would come at the expense of a number of precariously balanced legacy clubs wouldn’t bother me in the slightest.

One strange thing to note is that the report states that the NRLW has the fewest passionate fans but also shows that it rates better than Super Rugby, A-League, AFLW and NBL. Gemba seems to have again confused their fan metrics with what actually matters. The NRLW has the least ability to drive subscriptions, because it is bundled with the NRL rights. What is not stated is the commercial impact if NRLW gets its own broadcast deal. The competition could become self-funding and realise its potential. When you think about the NRLW in terms of the failure of the current administration to capitalise on its potential, then this negates the opportunity cost and makes this entire line of argument redundant in the context of expanding the NRLM.

pp 182
pp 194

Personally, I find it gross that a number of NRL clubs have shown little to no interest in the women’s game, not even bothering to pretend to want a NRLW franchise, but will use it as a distraction to further their case that they should not be subject to any further competition in the men’s league.

The real opportunity cost is using this time and money to put another team in Brisbane, presumably at the behest and benefit of Nine, when the report notes that the demand for a team in Western Australia or South Australia is about the same as it is for a second Brisbane team. Of course, that would cost the NRL a lot more money.

Conclusion

Consultancy, irrespective of the discipline, is fundamentally about giving the client what they want within the legal and ethical bounds of the profession. Most reports of this nature are used by C-suite executives to bludgeon their counterparts on the opposite side of the board room table, rather than being works of science. Many simply go unread, their heftiness being the primary goal.

Gemba’s brief was to develop a report that highlights the heretofore unexplored impacts of the second Brisbane team, with an emphasis on commercial and quantative analysis, and they’ve hit it out of the park. It’s clear that sprint one and two is their own work. As I get older, I realise in these grey-scaled scenarios, there’s not a right or wrong, so much as a more defensible or a less defensible position and while I could quibble here or there, their arguments are sound. I disagree with their conclusions but that doesn’t make me right and if I had done a similar report, not only would it look terrible but it would have had a very different goal in mind.

Where it falls down is where Gemba have been guided by their clients, the football clubs. Sprints three is about repeating the club’s concerns, dressed up in the language of corporate risk management. The implication is that the clubs see what’s good for them as being what’s good for rugby league.

Nothing could be further from the truth. The sport would greatly benefit from not only removing these lunatics from control of the ARLC but also shedding several or more from the top tier of the sport and redirecting their central funding to support expansion and the women’s game and grassroots or country football. On that basis, it’s difficult to swallow the conclusions reached in this part but it’s created the most talking points in the media, so in that sense it’s mission accomplished.

Still, it’s better to have some information out there, than not all, as long as we understand the context in which it was created, which is that most of the existing clubs do not want any further competition. There are a lot of interesting bits of information that I have not covered.

The ARLC pursued something similar under the Greenberg administration and this was subsequently buried in a dark hole by V’Landys. It seems unlikely that we will ever see that report.

Nor will we likely ever know on what basis the ARLC makes their decision about the successful expansion option. The information released by the clubs themselves is fairly scant – Easts held a press conference that could have been an email to talk finance and there’s reports the Jets are short on bank guarantees – and I assume the contents of the bids will remain commercial-in-confidence.

Irrespective, I have no doubt that a second Brisbane team will be announced in about a month. I guess we’ll see what happens and if its a disaster, at least the Panthers and Titans will be able to say I told you so.

2021 Queensland Cup WIP Report

We’re over half way into the Intrust Super Cup / Queensland Cup and it’s time to review how each team and player are performing. We’ll be looking at this using the following analytical tools:

  • FormElo ratings that reflect short term performance.
  • Production – The accumulation of valuable work on field, as measured by statistics that correlate with winning. Production is measured in Taylors.
  • Disappointment Line – The minimum number of wins for the season to not be considered disappointing by fans, as calculated by the pre-season class (long term performance) Elo rating.
  • 1st order winsPythagorean expectation calculated by points for and against.
  • 2nd order wins – Pythagorean expectation calculated by SCWP.

There’s more detail at How It All Works.

I won’t be providing much analysis for individual teams or players, instead preferring to let you do the work for your favourite team or player with the context about how the tools work provided.

WIP graphs

Comparing win percentage of the Disappointment Line, actual wins (0th), Pythagorean wins (1st) and 2nd order wins.

Note that the zero to one scale for winning percentage can make significant differences look minor; a .100 change in winning percentage is worth 1.9 wins at season’s end.

There’s no projections this year as we didn’t have any information from last season to work with.

Significant divergences between actual and Pythagorean wins are usually indicative of mean regression in future. That is, teams underperforming will improve their actual win percentage and vice versa so that the two numbers tend to converge. However, there’s always one or two that manage to avoid this convergence. We call them lucky/unlucky, depending on the direction that they miss and the ladder is currently a mess.

The outlook for each team:

  • Actual wins understating year to date performance / potentially tending better – Blackhawks, Tigers, Jets, Cutters, Capras
  • Actual wins overstating year to date performance / potentially tending worse – Wynnum, Devils, Tweed, Redcliffe, Pride
  • Actual wins reflective of year to date performance – Falcons, Bears, Magpies, Hunters

For teams with less than .500 percentage, 2nd order wins is always better than actual wins and vice versa. The trick here is to not focus on this specific value per se but to look at it as a prediction for next year’s performance. The next season’s performance for most teams will fall within one standard deviation (plus/minus .130) of their 2nd order win percentage for this season.

Form

Form Elo ratings at the end of 2020 and the end of round 11

1st Order Wins

Winning percentage estimated by Pythagorean expectation of actual points for and against

2nd Order Wins

Winning percentage estimated by Pythagorean expectation of SCWP for and against

Player Leaderboards

Wins Above Reserve Grade

WARG is a volume stat that compares the total amount of valuable work done (production), when compared to a replacement level player (or fringe first grader in the vernacular) at that position, irrespective of the time on field. A replacement level player has 0 WARG. It’s interesting to correlate who is the top of this leaderboard and who has been promoted to the NRL this season.

2019 was topped by Harry Grant with 2.6 WARG. The career leader (2016 – now) is Jonathon Reuben with 7.8 WARG. The single season record holder is Scott Drinkwater in 2018 with 3.0 WARG.

Taylor Player Rating

TPR is a rate stat that compares the amount of valuable work done (production) per game, factoring in time on field, to the average player at that position. An average player has a rating of .100. Minimum 5 games need to be played to qualify for TPR.

The 2019 regular season was topped by Harry Grant with single season record TPR of .266. The career (2016 – now, regular season only, minimum 15 games) leaders is also Harry Grant with a TPR of .195.

2021 WARG by position

WARG as generated when the player is listed in that position so far this season.

2021 NRL WIP Report

We’re just a bit over half way into the 2021 NRL season and it’s time to review how each team and player are performing. We’ll be looking at this using the following analytical tools:

  • FormElo ratings that reflect short term performance.
  • Production – The accumulation of valuable work on field, as measured by statistics that correlate with winning. Production is measured in Taylors.
  • Disappointment Line – The minimum number of wins for the season to not be considered disappointing by fans, as calculated by the pre-season class (long term performance) Elo rating.
  • 1st order winsPythagorean expectation calculated by points for and against.
  • 2nd order wins – Pythagorean expectation calculated by SCWP.

There’s more detail at How It All Works.

Unlike the season preview deep dive, I won’t be providing much analysis for individual teams or players, instead preferring to let you do the work for your favourite team or player with the context about how the tools work provided.

WIP graphs

Comparing win percentage of the Disappointment Line, pre-season projected wins, actual wins (0th), Pythagorean wins (1st) and 2nd order wins.

To be perfectly honest, I’m not 100% sure what these are meant to look like, having been the first time I’ve presented this information in this format. For example, I’m surprised that the pre-season projections and Disappointment Line are often pretty much the same because I’d never thought to check previously. Note that the zero to one scale for winning percentage can make significant differences look minor; a .100 change in winning percentage is worth 2.4 wins at season’s end.

The projections are naturally more conservative (less likely to predict outliers) to cover the potential spread of results. Think of, for example, Penrith as having 15.6 projected wins plus/minus 3 wins. While some projections are right, it tends to overshoot really bad teams and undershoot really good teams.

Significant divergences between actual and Pythagorean wins are usually indicative of mean regression in future. That is, teams underperforming will improve their actual win percentage and vice versa so that the two numbers tend to converge. However, there’s always one or two that manage to avoid this convergence. We call them lucky/unlucky, depending on the direction that they miss.

The current status of each team:

  • Way out – Cowboys, Rabbitohs
  • Actual wins understating year to date performance / potentially tending better – Titans, Panthers, Dragons,
  • Actual wins overstating year to date performance / potentially tending worse – Knights
  • Actual wins reflective of year to date performance – Broncos, Raiders, Bulldogs, Sharks, Sea Eagles, Storm, Warriors, Eels, Rosoters, Tigers

For teams with less than .500 percentage, 2nd order wins is always better than actual wins and vice versa. The trick here is to not focus on this specific value per se but to look at it as a prediction for next year’s performance. The next season’s performance for most teams will fall within one standard deviation (plus/minus .130) of their 2nd order win percentage for this season.

Form

Form Elo ratings at the end of 2020 and the end of round 14

Production

Pre-season projected and average year-to-date Taylors per game

1st Order Wins

Winning percentage estimated by Pythagorean expectation of actual points for and against

2nd Order Wins

Winning percentage estimated by Pythagorean expectation of SCWP for and against

Player Leaderboards

Wins Above Reserve Grade

WARG is a volume stat that compares the total amount of valuable work done (production), when compared to a replacement level player (or fringe first grader in the vernacular) at that position, irrespective of the time on field. A replacement level player has 0 WARG.

2020 was topped by Daly Cherry-Evans with 2.1 WARG. The career leader (2013 – now) is James Tedesco with 11.3 WARG. The single season record holder is Jarryd Hayne in 2014 with 2.3 WARG.

Taylor Player Rating

TPR is a rate stat that compares the amount of valuable work done (production) per game, factoring in time on field, to the average player at that position. An average player has a rating of .100. Minimum 5 games need to be played to qualify for TPR.

The 2020 regular season was topped by Cameron Smith with a TPR of .229. The career (2013 – now, regular season only, minimum 10 games) leaders are Harry Grant and Robbie Farah with a TPR of .164. The single regular season record holder is Robbie Farah with a TPR of .244 in 2013.

2021 WARG by position

WARG as generated when the player is listed in that position so far this season.

2021 TPR by position

TPR as generated when the player is listed in that position so far this season (minimum five games).

Ranking every rugby league team in the world

If you’re not interested in how the rankings work and just want to see the outputs, click here.

It began as a simple exercise to try and rate Super League players, much in the way that I rate NRL and Queensland Cup players. It turns out that the Super League website makes that an impossible task because it is a garbage fire for stats. Moving on from the wasted effort, I thought I might still do team ratings for the RFL system, mostly out of my increased interest with the Toronto Wolfpack’s promotion into Super League.

Then I thought about the Kaiviti Silktails of Fiji entering into the New South Wales system and wondered if I should take a look at the leagues there, despite my dubiousness about whether anyone in NSW cared about lower grade football when they could follow the Dragons, the Tigers or the Knights in so-called first grade.

From there I spiralled into a mishmash of US college football tradition, websites in Serbian and copying and pasting. When I came to, I had a neatly formatted spreadsheet covering a decade of world club rugby league.

Embed from Getty Images

Ranking the world

Invariably, creating any sort of evaluation system requires judgements by the evaluator about who to include or exclude and what the evaluation system considers to be “good”. I’ll explain my position and you can decide whether or not you like it.

Scoring the teams uses an average of four similar rating systems that look at performance over different time intervals.

We’ve long had form and class Elo ratings for the NRL and Queensland Cup. Form is about the short term performance of clubs, and can represent anywhere from four to eight weeks of results depending on the draw and league, while class is about long term performance, and can represent the average of years of performance. Form is a better predictor of match results, class is a better predictor of fan disappointment.

I created similar systems for another ten leagues in NSW, PNG, France (see also my Elite 1 season preview), the UK and the USA. They work along the same lines as the NRL and Queensland Cup editions. The average rating within an Elo system is approximately 1500 and the disparity in ratings can be used to estimate match outcome probabilities.

Both sets of Elo ratings are adjusted by a classification system I borrowed from baseball. To acknowledge the fact that a 1700 team in the BRL is not likely to be as good as a 1300 team in Super League, we adjust the team ratings so we can attempt to compare apples to apples –

  • Majors: NRL (ratings adjusted by +500) & Super League (+380)
  • Triple-A (AAA): QCup, NSW Cup and RFL Championship (all +85)
  • Double-A (AA): Ron Massey, RFL League 1, FFR Elite 1 (all -300)
  • High-A (A+): Brisbane RL, FFR Elite 2 (all -700)
  • Low-A (A-): USARL (-1000)

In Elo terms, a difference of 120 points between teams, like between an average NRL and an average Super League team, makes the NRL team 2:1 favourites. A 415 point gap gives the less favoured team a 8.4% chance of winning (equivalent to the replacement level), 800 points 1%, 1200 points 0.1% and 1600 points 0.01%. Consider the improbability of the Jacksonville Axemen beating the Melbourne Storm and you get an idea of where I’m coming from.

Between short term form and long term class, we’re missing a medium term component that represents roughly a single year of performance. I originally was going to create Poseidon ratings for the leagues, so I took a simpler approach and used points scored per game and points conceded per game over a regular season in lieu.

I then made my simplification much more complicated by doing a linear regression of winning percentage across all leagues compared to points scored per game and a second regression against points conceded per game. This gives a formula that converts the components of for and against into winning percentage, which is in turn converted to an equivalent Elo rating, which is then adjusted per the above. It also allows me to compare points scored per game – as a measure of competitiveness or quality or both? – across different leagues.

Competitiveness or quality.png

This specifically is just trivia but from an overall analytics perspective, the risk is if only the top league is analysed and analysts assume that the same principles apply to all leagues, incorrect conclusions will be drawn about the sport.

The ranking is decided by which team has the highest average score across the four rating components, which are given equal weighting. I call it the Global Rugby League Football Club Rankings, or GRLFC for short.

While it’s possible for teams to game a single system, it would be nigh on impossible to game all components, so I feel relatively comfortable that the highest ranked team is the “best”.

That said, form ratings and the for-and-against components only work on regular season results. Class ratings are the only component that takes into account playoff (and Challenge Cup, where applicable) performance. You may think finals footy deserve more weighting but I would put it to you that “the grand final winner is always the best team” and “any rugby league team can win on their day” are two mutually exclusive thoughts and I prefer to believe the latter. If you want to further mull it over, consider that Newtown finished seventh on the ladder in the twelve team NSW Cup in 2019 and then went on to win the Cup and then the State Championship.

Each club (as represented by their combination of name, colours and logo) is only represented once in each year’s rankings, by the version of that club in the highest league. For example, Wentworthville have been in the NSW Cup and the lower tier Ron Massey Cup. To date, Wenty have been represented in the rankings by their state cup team. However, as the Magpies will be replaced in the NSW Cup by Parra reserve grade in 2020, and while this doesn’t change much in reality, they will be henceforth represented in the rankings by their Ron Massey team. This is mostly because it makes the rankings a little more interesting, not having been clogged up by a half dozen clones of the NSWRL clubs.

I would like to have included the Auckland Rugby League’s Fox Memorial comp as a double-A league but it seems to be impossible to find scores. I also would have liked to add more low-A comps, like those in Serbia or Netherlands or maybe even Nigeria or Kenya, but scores for these comps are even more difficult to find or have incomplete results or don’t really play enough games. As a result, we may never know whether the Otahuhu Leopards are better than the Villeneuve Léopards.

I drove myself mad enough to trying to get the results that I did. I don’t feel the need to delve further into district comps in Australia but, who knows, I may well change my mind on that. It would be nice to go further back on some comps, particularly in France and PNG, but we have what we have. A big thanks to rugbyleagueproject.org, leagueunlimited.com and treizemondial.fr for hosting what they do have, because we can’t possibly rely on federations to have curated their own records and history.

A full season of results is required for a club to be ranked. This is only a problem for French clubs, with both Elite 1 and 2 running through their winter and the date the ranking is nominally calculated is December 31. A French club’s first part season is given a provisional place in the rankings, converting to a ranking the year after, based on the previous twelve months’ worth of results.

The rankings can be seen for 2009 through 2019 here. Your current top seeds in each competition are –

  • NRL (Major): nrl-mel Melbourne Storm (1)
  • Super League (Major): esl-shl St Helens (5)
  • Championship (AAA): esl-tor Toronto Wolfpack (29)
  • Queensland Cup (AAA): qcup-scf Sunshine Coast Falcons (30)
  • NSW Cup (AAA): nsw-nwt Newtown Jets (40)
  • Ron Massey (AA): nsw-mry St Marys (63)
  • League 1 (AA): rfl-old Oldham Roughyeds (64)
  • PNG NRLC (AA): png-lae Lae Tigers (66)
  • Elite 1 (AA): el1-alb Albi Tigers (69)
  • Elite 2 (A+): el2-vgh Villegailhenc-Aragon (101)
  • BRL (A+): qld-wsp West Brisbane Panthers (105)
  • USARL (A-): usa-jax Jacksonville Axemen (109)

Women’s Rankings

In an ideal world, we’d have a women’s ranking to complement the men’s. But the NRLW has only completed 14 games, which is not a sufficient sample although we may see that double in 2020. The QRLW will only commence this year and it remains to be seen what the NSWRL is going to do with their women’s premiership, whether this becomes the equivalent of a Ron Massey Cup to a new NSWRLW/women’s NSW Cup or if, as is usually the case, the Sydney comp will be promoted to be the state comp.

In the more enlightened Europe, the women’s Super League has completed its first season, comprising 14 rounds, and the Elite Feminine has just commenced its second season, the previous being 12 rounds. The bones are there for a women’s club ranking, but it will take time for Australia to catch up a little and make the rankings more balanced. With any luck, I should be able to deliver the first rankings at the end of this year.

The World Club Challenge

International club football is a rare thing, indeed. The ridiculously lopsided 1997 World Club Challenge (Australian clubs scored 2506 points to the Europeans’ 957) largely put paid to the idea that there could be a competition on an equal footing between the two major leagues of football. Other than a short lived World Club Series, which was overly reliant on the charity of big Australian clubs, all that remains of the concept is the World Club Challenge match-up between the winners of the Super League and the NRL.

First held irregularly since 1976 and annually since 2000, the match suffers from the disparity in the quality of the leagues – obviously driven by money – and a lack of interest – largely driven by a lack of promotion and lack of commitment from most Australian clubs. The advantage has ebbed and flowed, generally in favour of the Australian sides but in the late 2000s, the English fought back before being pummelled back into submission more recently.

World Club Challenge For and Against.png

Incidentally, I arrived at a 120 point discount between the NRL and Super League based on Super League clubs’ for and against in the WCC over the last twenty years. The application of Pythagorean expectation and then converting that (approx. 33% win percentage for SL) into Elo rating points.

Still, I believe that the WCC should be one of the centrepieces of the season, not unlike an abbreviated World Series or Super Bowl. A match day programme could be filled out by play-offs from the champions of the men’s, women’s and secondary men’s comps – perhaps with the winners of the NRL State Championship and the winner of a play-off of the premiers of the RFL Championship and Elite 1 – in the Pacific and the Atlantic. Such an event could be saleable to broadcasters, sponsors and hosts.

Of course, if successful, the WCC would then undermine the respective competitions’ grand final days, so there’s an obvious conflict of interest. The conflict is difficult to resolve when the stakeholders are more interested in maintaining their own position than making money or securing a commercial future. While cash may be a corrupting influence, the game will not survive as a professional sport without it.

Given the absence of international club fixtures, you could fairly wonder what the applications of this ranking system might be, other than to have a rough guess at whether the Gold Coast Titans are better or worse than the Sunshine Coast Falcons (the answer is: slightly better). My feel is that the final score is a rough proxy for a singular globalised Elo rating system. Consequently, it may not be very good but I looked back to the last ten WCCs.

GRLFC vs WCC.png

It was successful in predicting the higher ranked team winning eight of the ten matches but not particularly predictive in terms of the gap between the teams (the trendline above shows basically zero correlation) nor in the scale of favouritism (favourites won 80% of the time compared to 65.9% predicted probability). Still, it’s only a sample size of ten games where the Super League sides have been beaten pretty comprehensively.

In the meantime, this gives the English something to work towards.

2019 Pacific Tests Tips, State of Origin Previews and Other Rep Weekend Stuff

For rugby league nerds, this weekend is the true rugby league Christmas. We have a phenomenal mix of men’s and women’s games, featuring top line players representing teams from all over the southern hemisphere. There’s a World Cup qualifier in Sydney between the Cook Islands and South Africa. There’s two State of Origin matches, one for each cisgender. We have two women’s internationals, featuring Samoa, New Zealand, Papua New Guinea and Fiji, as well as the NRL’s Pacific Invitational with the same teams, except that famously Pacific nation, Lebanon, plays in lieu of New Zealand. The Kiwis will instead be facing down arch rivals Mate Ma’a Tonga to avenge their 2017 World Cup embarrassment in the opening fixture of the new Oceania Cup.

(Also worth a look this weekend in boring old club football: London Broncos vs Hull Kingston Rovers in a relegation battle. Kickoff 4.45AM AEST on Friday)

It’s a feast of rugby league and significantly more tantalising than what the NRL has served up lately. Here’s a bit of insight into each.

Embed from Getty Images

Read more

Rating the 2019 State of Origin teams

It’s that time of the year again. The weather has just turned cold and the NRL season has built just enough momentum to be interesting and has now been brought to a screeching halt. It’s State of Origin time, the world’s only all-star game that the players actually care about. Naturally, the first question anyone needs to consider is: which team looks stronger on paper?

When it comes to assessing representative games, we don’t have access to the usual team rating tools and, even if we did, the gaps between matches and changes to the teams are so significant that Elo ratings aren’t particularly useful. This year, we can evaluate the Origin teams using Production Per Game (PPG), which is a player rating tool.

Embed from Getty Images

Read more

The young forwards are not to blame for the Broncos’ demise

To not put too fine a point on it, the Broncos have been shockingly bad in 2019. Riding the hype train in this year, they were touted as premiership contenders (disclosure: including by me).

The Broncos have won just two games. One was against a Cowboys side that is facing similar struggles and another against a Sharks team bereft of its star power. The other six games have been losses, ranging from a late field goal from Corey Norman sealing the win for the Dragons, to thirty-two point demolitions at the hands of Easts and then again from Souths.

The finger pointing has begun. The Broncos’ extremely youthful pack has come in for criticism, both for a lack of go-forward and a lack of consistency.

The statistics tell a different story.

Embed from Getty Images

Read more

Are the Warriors the most inconsistent team in the NRL?

Despite being set an 18 point line, the Warriors managed to run the Storm close on ANZAC Day in Melbourne. Were it not for a dubious decision or two in the closing phase of the game, the Aucklanders might have left AAMI Park with the win. That led to people, including me, wondering out loud via social media where that level of play had come from, especially compared to some of the Warriors’ earlier displays this season.

Some people suggested the Warriors in Melbourne was a Thing. We only need to remember last year’s 50-10 drubbing to dispel that but also recall that the Warriors have a 7-13-1 record in Melbourne, with an average margin of thirteen points in the Storm’s favour.

So what gives?

Embed from Getty Images

Read more

« Older Entries