The polling period has ended - thanks to everyone who voted. Time for results!
Lots of people are definitely passionate about this issue, and a lot of good points and ideas have been brought up in this thread.
I was a bit surprised by the results:
- 10% of voters chose to switch to Traditional ELO
- 43% of voters chose to stick with the current system
- 37% of voters would like to pursue some other rating system
- 10% of voters chose the "I don't care" option
The first conclusion I can draw from this is clear: Traditional ELO is out.
Let's look at the rating system options:
- WHR: After reading the introduction on this paper, it certainly sounds better than the current Bayesian system. However, no implementations are available, which makes this one of the most time-consuming systems to adopt. If someone released an implementation it would make this doable, but for the moment it's not really feasible without a large investment that I don't have time for.
- TrueSkill: [Implementations](
http://www.moserware.com/2010/03/computing-your-skill.html) exist, so TrueSkill is more feasible. More investigation needs to be done in order to determine if it's really the best fit and how much work it would be to integrate.
- Bayesian ELO (current system): While certainly not perfect, it has two things going for it:
- The implementation cost is zero, since it's already implemented.
- More WarLight players prefer it than any other system. Potentially even by a wide margin, since the 37% that chose they wanted to pursue some other rating system may be fragmented between multiple different rating systems.
It's important to understand that the most limiting factor of what can be done with WarLight is development time. I have to carefully choose what I spend time implementing, as there is easily over a thousand different things I'd like to be adding to WarLight right now. What these poll results are telling me is that changing the rating system shouldn't be at the top of that list right now.
I don't want to change for the sake of change. Yes, the current system has warts, but every system will likely have warts. It can take a lot of investigation to determine which warts are preferrable.
I know some people won't be pleased with these results. This doesn't mean that the rating system will never change, but it does mean that I'm not going to consider it my highest priority at the moment.
If anyone has development skills and is super passionate about getting a different rating system implemented, I'd welcome the help in investigating a new one - send me an e-mail!
