I don't have a understanding of how rating systems work and maybe it was naive to make such a comment above however most probably don't know either and I just based my logic off 1v1 ladder which is also Elo? Bit confusing tbh.

Would you mind linking a source for someone noob to understand how the Elo rating system works and maybe put it on the MDL of some sorts cause education of how the rating system works would be good? Instead of me and maybe others assuming it has similarities like 1v1 ladder for expiration assuming they are roughly the same... because that's the only rating system they are exposed to.. sorta?

Edited 7/31/2017 04:35:56

There is a link under FAQs > Ranking.
https://metinmediamath.wordpress.com/2013/11/27/how-to-calculate-the-elo-rating-including-example/

EDIT : You can also use this site to calculate the effect of an individual game. Just remember to use k=32.
http://www.3dkingdoms.com/chess/elo.htm

Edited 8/1/2017 04:42:58

If all that matters is that your current rating is 1554, and that rating was computed from past events, then doesn't this mean that the past actually does matter?

Of course, your results determine your Elo rating. But the point I was trying to make was that how you get to a rating is irrelevant to your future. i.e, 10-0 can have the same effect as 20-10 if it meant that both those records yield the same rating(depending on your opponents).

This is not what you see on regular WL ladders(BayesElo). Losses bog you down there. So people usually make runs and wait for losses to expire to achieve "higher ratings/rank". Platinum was sort of hinting at that, which is something he doesn't need to worry about on this ladder because it doesn't work that way here.

Great change, MotD. I really like, that you talked to an expert like MathWolf and improved the rating system.

Hey I lost 8 ranks with this new system! Replace it immediately back with the old one that rewarded snail players like me!
/end{trolling}

I'm very happy MotD contacted me to discuss this! Overall, I think this is an improvement (obviously), although small bugs and kinks may need to be ironed out over time.

About "expiration" in general: the technical argument goes that games either "expire immediately" or "expire at infinity". Once a game is counted into the rating, it will never be used again. This makes sense as the rating is updated with the game when it happens. All systems using expiring games update the rating using games that finished several months in the past, which, if you think about it, doesn't make all that much sense.
This is also what MotD means above when he says that "how you get to a rating is irrelevant to your future." Once the game is counted and you have the new rating, the game won't be used again. How the rating was obtained is irrelevant when the next game comes along (which isn't the case in BayesELO).

The actual impact of a game on the rating will slowly decrease over time. (i.e. what would the rating be without this game?) Every new game against an opponent of similar skill has theoretically a larger impact than every previous game because it applies a change on the updated ranking. This can be practically tested immediately if you'd flip the order of a win and a loss against oppenents rated exactly the same: win, then lose gives a lower rating that lose, then win. (Sidenote: ELO actually rewards delaying wins rather than delaying losses, which is a great feature I think!)
This is an advantage over TrueSkill (used in RT ladder), where the impact of early games can be large and linger for quite a while, not even taking matchmaking into consideration.

Experts generally agree that the three main drawbacks (outside some technical issues) of ELO are
(1) rewarding inactivity
(2) the (subjective) choice of a K-factor
(3) the relative importance matchmaking

For MDL, the main problem was (1), which we tried to fix with the activity bonus. As currently implemented, this bonus is similar to a time-varying Poisson process (count data, here "counting" recent games). Additionally, convergence towards 1500 seems generally appropriate for long absence as there is no guarantee the rating is still correct.

I think (2) is less relevant for MDL. The subjective choice here simply means how much fluctuation MotD (or the players) want. A little more fluctuation is good and adds excitement for these kinds of competition, so a high K seems suitable. If the idea would be to have a rigorous ranking where you need many games to move substantially, a lower K would be needed.

From what I experienced myself, the matchmaking at MDL is automatic and quite good, which makes gaming the system (choosing one's opponents) more difficult and (3) irrelevant.

Great uppdate motd, and great job Mat wolf aswell, and also a very good comment by you above! Im exited for the future of MDL! This is a well needed update and its amazing!

I like the general idea. I really do. I'm not sure about the math aspect of it though.

Finishing a game grants 4 activity points. The points are capped at 80. From equation (1-0.02)^d=1-(4/80) you get d=2,5. That means that to get full rating bonus from activity, you need to finish a game every 2.5 days on average.

As a player who plays just a few games at the time (I usually limit myself to 6 MD games total), the idea that I'll always be behind in points compared to players of same skill level who play 9 simultaneous games is a little bit off-putting. I can already tell you I'm not the only one. (I know I'm not even participating in MDL now, but I was going to join again after I leave 2v2 ladder)

Did you consider delaying that counter? Let's say you start losing activity points a week after your last game finishes. That + 2,5 would give you over 9 days to finish a game on average. You could change the point loss to 2,5% on every day after the first week to get the same result of 0.5 after 34 days.

Just my suggestions. Though I don't believe you and Math Wolf did not consider it already. So may I ask why did you decide on such a move? I agree with not promoting non-activity. But are you sure that blatantly giving more points to players who play faster is a good idea?

PS: great job with the ladder overall. We all appreciate the effort you put into this.

@Rento: I'm a snail myself, so we did carefully consider these numbers. I knew it would penalise me as well. :-)

The impact should be reasonable. To use myself as an example: I play few games at the time and slow, barely enough to remain ranked actually. Yet, at this moment, I have 25 activity points. A criterion that I proposed to MotD was that anyone putting it at least some minimum effort needs to get half the activity points (40). A total of 30 games in 5 months (you needed 20 to remain ranked) gives you (slightly more than) that. The difference between a very active player and someone who achieves this is less than 40 points, which seems to make a difference of only a few ranks.

I personally think this is correct and fair because the rating of someone who plays more is more accurate.
Someone who wins his last 10 games, but didn't play all that much during that period (and may be delaying losses), does not deserve the same activity bonus as someone who goes 20-5 during the same period. Yet, without activity bonus, they might get the same rating. For the first player, there is a much larger risk that this rating is inflated. Note that TrueSkill, used on RT, also penalises for (lower) activity and more severely than this system. From a mathematical and theoretical point of view, I don't see it as an activity bonus, but as an accuracy bonus. My first proposal to MotD was about using a variance measure to achieve this same effect, but that was not as practical.

Either how, as far as I understood, these numbers are not final yet and may be adapted based on how much activity should translate to a full activity bonus. I'm sure MotD doesn't want it to be a reason for people not to join, but at the same time it should remain an incentive to play more than just two games at a time.

I'm personally not a fan of delaying the decay. This would give an incentive to stall losses with exactly a week in between, which we definitely want to avoid!

EDIT: some more details and example added in second real paragraph.

Edited 7/31/2017 15:45:44

Well here's where we disagree: 40 or 50 points difference is a big deal if you ask me. I mean, I was never in top of that ladder, but from what I've heard you get literally a few points for every win up there. (Can I ask for confirmation MotD?)

To put things in perspective:
- finishing a game every 2,5 day gives you full 80 points
- every 3 days - 70 points (-10 points already! just half a day difference in playing speed)
- every 5 days - 40 points (40 points behind)
- every 7 days - 30 points (50 points behind!)

Does that really seem fine to you guys?

I personally think this is correct and fair because the rating of someone who plays more is more accurate.
Someone who wins his last 10 games, but didn't play all that much during that period (and may be delaying losses), does not deserve the same activity bonus as someone who goes 20-5 during the same period

I agree partly, however sometimes a person is being penalised for not finishing games because their opponents are playing ridiculously slow. Is it fair to penalise the other person who would play fast given the chance?

Currently I have 5 games ongoing. (1 just finished because I asked the other person to play faster) but 4 out of those 5 games are being played ridiculously slow by my opponents. The one that just finished today was over two weeks and the other 4 are about to be 2 weeks. That means I have not finished a single game in two weeks, not from any fault of my own, yet I am penalised for those extra points I could have had. I did talk to Motd about it and he explained it doesn't have that much impact and it will even out due to getting faster opponents another time and I'm fine with that. However I just wanted to comment on your statement where you said it was fair.

Is there any way the "Best Rating Achieved" can be recalculated factoring in this new rating system? Ex. I had 1750+ on the old rating system, but with this new one, my current rating (below my former max in relation to the old rating system) is automatically above the old max rating I had on the old rating system. I'm not sure whether the data has been lost, but I think the conversion of old -> new should also apply to the former max rating achieved.

Well here's where we disagree: 40 or 50 points difference is a big deal if you ask me. I mean, I was never in top of that ladder, but from what I've heard you get literally a few points for every win up there. (Can I ask for confirmation MotD?)

That's not completely true anymore. This had to do with the expiration, which pulled the rating back down again and again whenever a previous win expired. A win while ranked at the top should now actually make a difference. Note that the top rated players all have a much higher rating now because of this (as it should be!)

To put things in perspective:
- finishing a game every 2,5 day gives you full 80 points
- every 3 days - 70 points (-10 points already! just half a day difference in playing speed)
- every 5 days - 40 points (40 points behind)
- every 7 days - 30 points (50 points behind!)

Does that really seem fine to you guys?

It's of course up to MotD to decide if he wants to change the numbers (I'm in contact with him and open for discussion about it at all times). Personally, I'm happy to talk about them here and discuss the pros and cons of them.

That said, I personally agree on these numbers and they look fine to me (as a slow player!).
Because:
Every 2.5 days means almost 3 times more games than every 7 days! That's 3 times as much chance of an upset loss, because more games also has a higher chance of getting an opponent that is somewhat further away in rating. A difference of 50 points to offset that seems fair to me with a K-factor of 32. After all, a game against an equally rated opponent nets you 16 points while a big upset loss can cost you 30. Lower amounts of games do lead to reduced precision and associated behaviour to game the system.

Also every 2.5 days versus every 3 days is 20% more games as well. It's not half a day difference in play speed necessarily, it can also be 6 games instead of 5, which is a tangible difference, I'd say.

Again, they are not final and as far as I know it's open for discussion what should be enough to warrant a maximum bonus. Giving 5 points per game for example, would already mean you need 20% less games to achieve the maximum (and would give everyone except those at the cap a 25% higher bonus). A total of 8 points per game (or alternatively, 4 when started, 4 when finished), would mean 30 games in 5 months is enough to reach the max.

I personally think this is correct and fair because the rating of someone who plays more is more accurate.
Someone who wins his last 10 games, but didn't play all that much during that period (and may be delaying losses), does not deserve the same activity bonus as someone who goes 20-5 during the same period

I agree partly, however sometimes a person is being penalised for not finishing games because their opponents are playing ridiculously slow. Is it fair to penalise the other person who would play fast given the chance?

Currently I have 5 games ongoing. (1 just finished because I asked the other person to play faster) but 4 out of those 5 games are being played ridiculously slow by my opponents. The one that just finished today was over two weeks and the other 4 are about to be 2 weeks. That means I have not finished a single game in two weeks, not from any fault of my own, yet I am penalised for those extra points I could have had. I did talk to Motd about it and he explained it doesn't have that much impact and it will even out due to getting faster opponents another time and I'm fine with that. However I just wanted to comment on your statement where you said it was fair.

That's obviously a fair point. There are three (four) things I can say about this:
* I agree with MotD that over time it will offset each other, but this will indeed lead to small changes during periods when many games finish close after one another and periods where few expire. I'm not aware of any reasonable solution to this.
* After reading about the update, comparing the numbers and thinking everything through again, we discussed a small (future) back-end change that should make sure that this has no impact on very active players who are/were at the cap. If you accumulated points above the cap, some of those are retained in the back-end, so that you can keep maximum bonus for a few days after finishing the last game (not all to avoid the most extremely active players to remain at +80 a month after they leave!)
* Splitting the bonus in 2 points when starting a game and 2 when finishing could offset this and allow active players to add games in such situations? I'm not sure if this would be easy to program however.
* You are lucky I am not one of your opponents. ;-)

EDIT: minor text fixes and included reply to Styxie as well.

Edited 7/31/2017 16:19:03

* You are lucky I am not one of your opponents. ;-)

Haha yup. I'm not complaining as such, I understand some players play slower than others especially when busy and I don't really have a problem with people playing slowly. Just wanted to point out that while slow players are being penalised which may be deserved, this impacts their opponents too which may or may not be deserved.

* After reading about the update, comparing the numbers and thinking everything through again, we discussed a small (future) back-end change that should make sure that this has no impact on very active players who are/were at the cap. If you accumulated points above the cap, some of those are retained in the back-end, so that you can keep maximum bonus for a few days after finishing the last game (not all to avoid the most extremely active players to remain at +80 a month after they leave!)

If it does fix the issue, then that's great and appreciate you and motd taking it into consideration. Thanks for all the effort and discussion.

"Haha yup. I'm not complaining as such, I understand some players play slower than others especially when busy and I don't really have a problem with people playing slowly. Just wanted to point out that while slow players are being penalised which may be deserved, this impacts their opponents too which may or may not be deserved."


Well this is so easy to fix. whenever you end up in such situation, and that cant be to often u end up in 5/5 opponents being real slowpokes, but when that happen its just to incresse tha gameamount to 6 or 7, get 2 new games and then reduce it again. Playing 5 games agains fast players takes more effort thab 7 games against slow players so it shouldnt be a problem?

I don't see it as an activity bonus, but as an accuracy bonus

I think this is the best way to think about it. Also, if you look at rating differences on the ladder, especially at the very top, the difference between places is often (though certainly not always) more than 40 points anyway, so you might not lose out in placement based on the accuracy/activity bonus anyway. I think it's a positive thing, and I speak as a person who has somehow been playing even slower on this ladder than Math Wolf! *gasp*

Is there any way the "Best Rating Achieved" can be recalculated factoring in this new rating system?

I agree with this also, though probably don't do it until you finalize the fine-tuning of the rating system. As is, the historical graphs are pretty meaningless, so it would be nice to have it back dated.

Edited 7/31/2017 17:40:28

I don't think the activity bonus will work as an incentive to play more games (and thus inflate the MDL appeal, which I guess is the underlying goal here). As for me, players are already playing the maximum their personal time allow.
Hoever, this bonus will penalyze players whom time for MDL is more limited than other players. As such, this could actually play against MDL appeal.
If anything, 4 points per finished game could be reduced, to 2 or 1, for example. The incentive to play more would still be there (and 20 or 40 points per period can already make a difference versus a close ranked player), and "slow players" (by that I mean those at 2 games at once) would not be penalyzed that much.

Edited 7/31/2017 17:46:25

@Rento,

Well here's where we disagree: 40 or 50 points difference is a big deal if you ask me. I mean, I was never in top of that ladder, but from what I've heard you get literally a few points for every win up there. (Can I ask for confirmation MotD?)

To put things in perspective:
- finishing a game every 2,5 day gives you full 80 points
- every 3 days - 70 points (-10 points already! just half a day difference in playing speed)
- every 5 days - 40 points (40 points behind)
- every 7 days - 30 points (50 points behind!)

Does that really seem fine to you guys?

I can confirm that you do get a lot of points for winning games(which is great!). However, in the old system all of these points would be drained due to the games expiring(which is why it was hard to rise). We have solved that problem, so 40-50 points isn't that big of a deal even for someone at the very top(I could get +40 right now with 3-4 wins against the top 10).

When you say that this is unfair to those with lower activity, you're making an assumption that the rating of the inactive player is as accurate as the active one. Every time you play more games your rating is more accurate as you're putting more points on the line(which will be taken away if you lose). Someone who has played a hundred games to get a 1700 Elo rating has done much more than someone who got to 1700 with just 10 games(but a pure Elo rating system doesn't reflect that). My goal isn't to force everyone to play hundreds of games. Most people should be able to get a reasonable bonus from activity even if they play at their own pace.

Math Wolf's detailed reasoning is something I definitely agree with. However, like he said, we can definitely tweak the system if it is significantly skewed towards active players. Let us observe the behavior over the next week and revisit this conversation.

Edited 7/31/2017 19:01:03

@AG/Beren

Is there any way the "Best Rating Achieved" can be recalculated factoring in this new rating system? Ex. I had 1750+ on the old rating system, but with this new one, my current rating (below my former max in relation to the old rating system) is automatically above the old max rating I had on the old rating system. I'm not sure whether the data has been lost, but I think the conversion of old -> new should also apply to the former max rating achieved.

I agree with this also, though probably don't do it until you finalize the fine-tuning of the rating system. As is, the historical graphs are pretty meaningless, so it would be nice to have it back dated.

The best rating achieved could possibly be recalculated. But the history charts are going to be much harder to calculate. I do not store information pertaining to players joining/leaving the ladder. So I would need a way to reconstruct that using the existing charts. I'm also not sure if it is the right thing to do. If the competition ranked players for the last 9 months using a certain metric, the historical charts should reflect that in my opinion(Even the trophies were handed out using that metric). I'll give it some more thought and get back to you on that.

Edited 7/31/2017 18:31:05

@Mike

I don't think the activity bonus will work as an incentive to play more games (and thus inflate the MDL appeal, which I guess is the underlying goal here).

I couldn't disagree more. The intent of this change is not to "inflate MDL appeal". MDL is as appealing as it needs to be. This change is meant to improve the accuracy of the rating system with regards to estimating the skill level of a player. Someone who plays more has a more accurate rating.

As for me, players are already playing the maximum their personal time allow.
Hoever, this bonus will penalyze players whom time for MDL is more limited than other players. As such, this could actually play against MDL appeal.
If anything, 4 points per finished game could be reduced, to 2 or 1, for example. The incentive to play more would still be there (and 20 or 40 points per period can already make a difference versus a close ranked player), and "slow players" (by that I mean those at 2 games at once) would not be penalyzed that much.

It's unfortunate if this change reduces the appeal of MDL in any way. But I don't think it should influence the decisions made for the good of the ladder(which this change is, in my opinion).

Edited 7/31/2017 18:49:15