To me 'strategic' and 'high rating deviation' are one and the same thing and the latter is objectively measurable.
That's imho the best proxy but not precisely equivalent.
I'm having trouble articulating an adversarial case as a Warlight template, but in the general case you can imagine a "sticky lottery"- e.g., a game of "Who is the Tallest?" (
https://www.youtube.com/watch?v=fG8g9A7pBqg) or a lottery that just looks at your profile ID to decide if you should win. That would lead to high rating deviation but not measure something we'd consider skill.
More practically, high standard deviation is a function of both the game and the players, since it also picks up on variance in how well the players understand the template. INSS templates probably have lower standard deviation when players have yet to figure them out, and might have lower standard deviation again later in their lifetimes when players learn them better so you have less of a bimodal skill distribution between "people who at least know the tricks" and "people who are clueless." You could also imagine- as another contrived example- a lottery vs. a lottery that has one trick. I.e., the former evenly splits win probability between players while the latter evenly splits win probability between players who did one thing correctly (say, you need to play a card)- how much more strategic is the latter? Would you be able to capture that difference meaningfully by looking at rating deviation? What would rating deviation actually capture here? What does that imply about what it captures in, say, the MTL?
You
could construct a definition of "strategic" w.r.t. rating deviation that avoids the pitfalls above (something that looks at how much the template itself creates rating deviation among players who are near-peers in skill; i.e., how good it is at capturing purely 'skill' rather than the noise of 'luck,' even when the skill difference is small), but it gets in the way of your final point:
objectively measurable.
Other than rating deviation, I think there's other measurable tools you can/should use to tell if a template is strategic, each with its own drawbacks:
- how well a well-fit Elo (or similar) model is able to explain/predict the results (you can use both measures of fit & p-values)
- how correlated performance on that template is with other templates (i.e., "does this capture 'general Warlight ability' or some other skill?" - like IQ)
- basic stats, like how common upsets are (and their magnitude)
I tried this stuff a
long time ago and admittedly clumsily (sorry Rufus!) for the MTL templates (
https://bit.ly/mdl-analysis); from experience, I think it's rather difficult to get sufficient, consistent, and high-enough quality data to get meaningful answers that don't pick up on all sorts of confounding factors.
As a final thought, I think "strategic" is the wrong framing here to build on; I'd recommend framing the analysis here instead in terms of "skill" and "luck" and starting from there. You can probably crib a lot off of other fields that try to understand variations in ability & how they're captured by tasks, like psychometrics & collective intelligence. Imho once you have a definition of "skill" (as a lazy example: suppose everyone on Earth has an otherwise unobservable organ that determines their underlying ability to play Warlight; skill is the relative size of this organ, and a template is more strategic if it's able to more accurately approximate it quickly & consistently), this stuff gets a lot easier and practical to measure.
Otherwise it tends to get religious.
Edited 6/30/2024 15:29:50
