Edited my post with further discussion, but my point is that but for lack of participation, the MDL would be easily our best option for figuring out where players stand in terms of strategic ability.
I think we should use it as the end-all-be-all instead of the 1v1 Ladder like Wally is attempting to do.
Using a single-template ladder like the 1v1 Ladder is flawed due to the limitations of general Warzone ability. One thing I think we've mostly avoided here is that Warzone settings can get really diverse, even if you stick to what's generally considered strategic. And you can be good at one thing without being good at everything else; the fewer templates you use, the less confidently you can say that your results reflect actual general strategic ability (this is just intuitively true, although it's also supported by data).
On the Multi-Day Ladder, as players get better they also get (generally) less consistent across templates:
(note that r^2 is going to be small here even if there's a strong correlation purely because there's a lot of noise to deal with due to sample size issues- instead, it's better to just look at this visually and focus on the clearly visible linear growth pattern you see as you look left to right at most of the datapoints)
At the same time, though, it's not accurate to say that these great players are mostly doing well on a handful of templates and that's what's driving them up. While it's true that their performance across templates gets less consistent, this is really happening more at the edges. The middle 50% of templates for a player- templates where they perform better than they do on their worst 25% and worse than on their top 25%- has a much more consistent IQR (interquartile range- the difference in skill between their 75th percentile template and their 25th percentile template)- actually it's almost constant if you take out some players near the bottom (who'll have 0 inconsistency and IQR because they haven't played much). So as players get better, they get better by about the same amount across their average templates; they might be able to do some specific optimizations for some templates and lag behind a bit on some others, but there's still an empirically demonstrable "general skill" that better players have more of.
This means that a single-template ladder can be rather flawed because it's relatively easy to just adapt to that one template without having a corresponding improvement in general Warzone ability (at least on canonically strategic templates). But if you have a ladder like the MDL with considerable variation in templates, then you can get much more reliable results.
Edited 2/26/2018 18:38:28