Since most lists about maps are based on quantity, I have developed a metric that summarizes quality and success for map makers, which includes quantity as well.
The exact formula (beware math) is below.
The final results are as follows:

1	RA			43.558
2	Incaman			42.998
3	Срећко			40.829
4	Fizzer			38.125
5	[中国阳朔]Chaos		32.633
6	Major Risk		30.994
7	Issander		30.741
8	The King [Belgian G...	30.638
9	Roblexa Caritelisa 948	28.971
10	{rp} pedrito		28.401
11	Ottaman			26.580
12	Troll			25.847
13	Caia Veloso		25.441
14	Qi			22.770
15	Saruman			22.537
16	Grundie			22.021
17	Apollo			21.693
18	Kheimon			20.261
19	Muppet			19.953
20	Vampires – Kayn		19.824
21	Moros			18.292
22	Dameon			17.976
23	[中国阳朔]TexasJohn	17.500
24	King Philip		17.339
25	Nate			16.301
26	C.K – 胖鹌鹑		16.216
27	eliod			15.999
28	DynamiteT		15.622
29	Dogrosebush		15.140
30	Phil			14.722
31	Smok			14.275
32	Koger			14.211
33	Arc Light		14.179
34	Ranek			13.831
35	NinjaNic		13.702
36	Huruey • apex		13.622
37	Urfang			13.621
38	sue			13.537
39	muligan			12.864
40	Megli			12.498
41	Guderian		11.611
42	Smorgborg		11.365
43	Gunz			11.315
44	Worldbeing		10.214
45	Traintown		10.079
46	Opum's Razor		9.817
47	Lord Syvedyas		9.705
48	Yeezus Theory		9.680
49	kendou			9.283
50	Jon Snow		9.233

59	Riyamitie		7.452
61	[NL] Willem van Oranje	7.297
66	Metaltubbie		6.561
67	Frankdeslimste • apex	6.553
69	skunk940		6.311
70	Clement Attlee		6.280
72	Nathan			5.882
137	Kroma			1.966
148	ChrisCMU		1.628
163	Bananashake		1.125
168	Sir Nick		0.992

*229 mapmakers were ranked. Extrapolating, I'm guessing there are at least ~400 mapmakers in total.

For the formula:

• Only maps with a rating higher than 3.5 were considered.
  
  
• Define the base score "B" of each map as its rating substracted by 3.5. This generates scores between 0 and 1. (Only maps with ratings higher than 3.5 were considers, no maps have ratings higher than 4.5).
  Why 3.5? Because I only want to include maps with decent quality. Maps with ratings over 3.5 must have gotten at least half of the time a vote of 4 or regular votes of 5 stars. I found 4 too high as there are still many decent, often played, maps with ratings between 3.5 and 4. Also, at a first glance, the average and median ratings of maps both seem to be close to 3.5.
  
  
• Use V as notation for the number of votes for a map. The vote-corrected score S = B * V/(V+100). Thus, the score S is always higher than 0, and always lower than B. When a map has a large number of votes, S is close to B. When this number is low, S is closer to 0.
  Why do I do this? There are two reasons: (1) more votes means a more reliable score. I want to avoid inflation from maps who just received some 10-15 votes of all the mapmaker's friends or alts. In general, the rating of (highly ranked) maps often tends to decrease when more votes are cast. Technically, I am adding 100 votes of "3.5", fictional users who pull the rating towards the generally expected mean. This could also be interpreted as a prior distribution. (2) more votes means more plays and more success. I don't just want to measure quality. You can have the best map on WL, but if nobody plays it, the success is limited. By downweighting with 100 fictional votes, I guarantee that maps who are rated often get most of their base score B in their corrected score S, while maps that are hardly played, only received a fraction of the base score.
  Example: 11 votes: S=B/10, 25 votes: S=B/5. 50 votes: S=B/3, 100 votes: S=B/2, 200 votes: S=2B/3, 900 votes: S=90% of B, over 2000 votes (most played maps): S > 95% of B (96.4% for the most played map).
  
  
• The score of each map maker is the weighted sum of his best 10 maps with ratings over 3.5. (Only 7 mapmakers actually had 10 or more maps with a rating over 3.5, for all others, only the available maps were counted). The weights were as follows: 25, 18, 14, 10, 8, 7, 6, 5, 4, 3. So the total score T=25*S1+18*S2+14*S3+... These weights do sum to 100, so technically, the maximum score that could be reached would be 150 (if a mapmaker had 10 maps, all ranked 5 with a huge amount of votes). In practice, I expected no mapmaker to score more than 50.
  Why those weights? I thought it was important to give all mapmakers a fair chance, even those with a relatively small amount of maps. Thus, a few maps should carry most of the weight. Specifically, I wanted the first map to count as a quarter of the total score as this is the "masterwork". The first 3 maps cover more than half, with the first four maps covering over 2/3rd of the total score and the first half exactly three quarter (which implies that maps 2-5 count for half, 6-10 for a quarter and the first for a quarter).
  
  
• Note that the scores of the maps are sorted by vote-corrected score S and not by base rating B! As an example: Opum's Razor has a map with a rating of 3,9716 and 141 votes, so B=0.4716, V=141, S=0.276. Another map has a rating of 4.0000 and 48 votes, so B=0.5, V=48, S=0.162. Thus, the first map is given weight 25, the second (although with a higher rating), is given weight 18. The other way around, he would have been penalised for having his lower ranked map played more and his total score T would have been 9.021, which would have dropped him to 51st place.
  
  
• Summary of the formula: http://www.texpaste.com/n/zvr6xxne
  
  
• Scores were collected on November 11-November 13 and thus may show small innaccuracies. All scores above 4 were collected on the same day.
  

If anyone who was not ranked wants to know their rating and/or rank, feel free to ask.

Edited 11/15/2014 22:07:06

Discussion of the results:

• RA wins it, but just barely. The difference between RA and Incaman is obvious. While Incaman has 3 of the highest rates maps on Warlight, including the single highest rated map (Rise of Rome: 4,4448, S=0.901), RA beats him by the quantity of his quality: 7 maps rated 4 or higher with one extra rated 3.8896. No other mapmaker has more than 4 maps rated higher than 4. (although Apollo and Roblexa Caritelisa 948 both have 4 maps with rating higher than 4 and 1 with a rating of exactly 4 at the moment the data were gathered.)
  
• Срећко combines top quality (Europe Big, Modern Europe both rated over 4.3) with quantity (9 maps rated over 3.5). After his 3 highest rated maps, most aren't played very often, so he still has potential.
  
• Fizzer basically surfs on the success of all the Earth adaptations. Most of those are both highly and often rated. Medium Earth leads the way, but is overall only the 5th most rated map! (behind Earth, Europe, Rise of Rome and Imperium Romanum)
  
• [中国阳朔]Chaos is maybe the most consistent quality mapmaker around. I lost count of the number of maps in the 3.7-4.1 region with ~100 ratings that he has.
  
• Major Risk gets most of the success of the European Borders map, followed by 3 other highly ranked, but somewhat less played maps.
  
• Issander's highest rated map is hardly played, but his Huge World makes up for that. Many maps with high ratings, but some have very few ratings.
  
• The King, similarly to Issander combines the result of 2 highly rated maps (almost equal S score) with many other good quality maps.
  
• Roblexa Caritelisa 948 has a large amount of good quality maps, some of which are not often rated (including the highest rated map).
  
• Both {rp} pedrito and Ottaman have 2 highly rated, often played maps.
  
• Based on only the Europe map, Troll would still be ranked 15th.
  
• Caia Veloso, Kheimon, Vampires - Kayn, Dameon, [中国阳朔]TexasJohn and some others get most, if not all of their success from one highly rated and often played map.
  
• Qi, Saruman, Apollo, Muppet and Moros use decent quantity on the other hand. Apollo has over 10 qualified maps, but none of them is rated more than 100 times.
  
• Grundie equals Earth and small Earth. His maps are rated a combined 4400 times!
  

Edited 11/14/2014 10:03:24

I have a few projects in the making that will hopefully bump my very low score up.

Go, Statistician clan!

This is all looks like complex work, thanks! Now we finally know who is actually best.

Слава Срећки!
Слава Ра-у!

And, also we know how self-inflated some people really are (skunk940...).

Edited 11/13/2014 23:07:58

{Wolf whistles}

That's a big list! I'm probably going to have to take a look at RA's maps after this.

... But I would really enjoy the formula more if it was presented just a little more elegantly (a very good example for this is Euler's identity). But if you don't, it's fine!

... It's just that I'm a sucker for nice-looking formulas. :p

Edited 11/14/2014 00:00:15

Nice work!

Thanks Math Wolf, a thorough analysis and a great read.

@ Phulesdorp: http://www.texpaste.com/n/zvr6xxne
@ Yeezus Theory: I personally feel every point above 0 is a contribution to WL as it means you made at least one map rated higher than 3.5. So I would never call your score low. I think it's not that easy to get into the top 50 of this ranking.

This moment in time my maps are overrated, the biggest amount of ratings is only 91. I've yet to prove myself. I still have a couple of maps to try and make widely playable.

My point, the formula is good. But doesn't take the amount played into consideration. Though I can't say I keep track of how many games are played on my maps. But I can say only about 10-20 maps are played frequently.

Edited 11/14/2014 00:37:30

Nice. :)

@Apollo, it looks like he's using number of ratings as a proxy for number of plays, since we don't have that data at this point. Hopefully Fizzer will release that information, at which point we could incorporate that into a new formula for this.

Twenty fifth. hmm guess I better keep working.

Good maths

@ Beren Erchamion (& Apollo): That's mostly correct indeed. I do need the number of votes itself as those define the accuracy and precision of the rating (reason 1). But for the success (reason 2), it would be better to have the actual number of plays. Ideally, that part of the formula would be a product of terms including the number of votes and number of plays rather than just the votes.

As far as I know, there is no data on the number of plays of a map and guesstimates are very difficult to obtain, I think. So I feel that for average to highly rated maps, the number of votes is as good a proxy as any at this moment. (For lower rated maps, I think it would be a bad estimate as people would play once, vote and never play again.)

Additionally, multiplayer games get saved and stored and thus can be counted (technically), but single-player games not. (They do not have an ID!) So it may actually be impossible to retrospectively count the total number of plays of a map. It should be relatively easily (server side, crawler) to count the number of multiplayer games of a map.

Well, Skunk is only 0.031 points ahead of me despite the fact he's made more maps than anyone and I've only ever made 5.

Wow, it's incredibly cool to see myself at place 21 of such a ranking, let alone one made using solid maths. Thank you!

I'd like to know where I rank. I think I only have 4 maps done, but all are well made IMO.

So... from what I'm understanding, it's practically a Simpson's paradox to all "unpopular" maps? That's slightly discouraging for me and my mapmaking skills. :(

And will this list ever get expanded upon in the future (since I don't know what a crawler is, I'm taking it as a good sign)?

Interesting :D but your whole value system is a bit flawed.

Edited 11/14/2014 20:55:07

Thats right. next time, take care that incaman is the number one!^^

I find this fascinating. Out of the top 50, I probably didn't even know 1/2 of those people made a map at all. It means either the rating system is flawed or they have great maps I need to check out.