Ooh, I just thought of another potential solution to this!
Okay:
- For each official bot, a dedicated player account is created, similar to the 'AIServer Bot' Account, but with a different name for each, indicating which type and version the bot is.
- The purpose of these accounts is *only* to play in the 1v1 ladder (against humans and other official bots only) to assess the '1v1 template skill' of each official bot.
- After qualifying for a rating, whatever the current rating is for that bot becomes its 'official current rating'.
- This is a simple, slightly crude, but much-better-than-nothing method to assess the relative skill values of each bot.
- A setting is added to templates (ideally within the enabled/disabled, not-before-turn-X, versions-of-bots-allowed-checklist framework) to limit the skill level of allowed bots.
- Settings could be similar to how the game pre-requisite settings work: Unrestricted bot skill level (default for custom games); 'No bots allowed that are higher than rating R' (e.g. R = 1000 (perhaps for auto-games), R = 1200, R = 1500); perhaps even 'No bots allowed *below* rating R' because Why not? Could be interesting.
This would even accommodate AIs being improved over time, as either a) a new, separate account could be created for each major version of a bot, with a separate rating, and the option to choose it separately from other versions of the same bot, or b) if we just keep one account per bot-type then the 'official current rating' of the bot will automatically adjust as it plays more ladder games.
Finally, and here's the kicker, when playing in the 1v1 ladder, you're not allowed to use a bot that's ranked higher than you are. (In fact, perhaps there's a 'discount', like 20%, so you can't use a bot that has a rating higher than 80% of your own rating. This would prevent people from using bots to 'hold' their ratings in the ladder without actually playing any further games themselves; once their rating drops to below 125% of the bot's rating, they no longer qualify to use that bot, and are forced to use a weaker one.)
Edited 4/14/2016 16:58:53