Posted on 05 February 2016 - 09:49 AM

I really like to analyze the elo system, for a couple of reasons, but lately I've come across this problem of elo inflation and deflation. If you don't know what I mean by that, I'll give you the short version: a normal ladder is one that got the average # of players joined of all the ladders, e.g. there's 3 hypothetical ladders, named A B and C. A has 4 players joined, B has 6, and C has 8. The average # of players joined is 6, and since B has 6 players joined, it's a normal ladder - neither inflated nor deflated. C has an above average # of players joined, making it an inflated ladder, and A has a below average # of players joined, making it a deflated ladder. Switching paragraphs for dramatic effect | )

In the real world, obviously no ladder is exactly normal, simply because there are 15 different ladders, making it unrealistic to expect the average to even be a whole number, let alone expect one of the ladders to exist staticly on that whole number. But beyond that, it's important to note that the # of players in every ladder is constantly expanding, which is what makes this idea of static elo exchanges (like, how much elo gets sent to the winner of a given match and taken from the loser of that match) pretty inconsistent with the elo system as a whole.

The reasoning behind this is that elo is supposed to be exponentially harder to get as you climb the board. In theory, it's supposed to be much harder to go from 2300 to 2400 than from 2200 to 2300, and, although it is, it isn't all that much harder. That's because, again in theory, the ease at which you gain elo is based on the elo pool (or, the total amount of elo that exists in any 1 ladder, which is directly based on the # of players that have joined that ladder. In fact it's actually 1400xtotal # of players in a given ladder that will give you the elo pool of that ladder.) and its size. As the elo pool grows, the spread of the elo distribution increases, making it easier to reach higher (and btw lower) elos. This not only makes elos across ladders virtually incomparable, but it makes elos across seasons incomparable, which is much, much worse. 2600 for danteh now is pretty crazy, but in a couple seasons it won't be such a feat. And a couple seasons ago it would have been next to impossible.

But that's not the only problem. The fact that the individual player's grind depends on the size of the elo pool means that somebody grinding at the end of the season has an easier time than somebody grinding at the start of the season. If I wanted to achieve the highest elo I could, I would wait until the end of the season, when the pool is as inflated as possible, and I'd have a much easier time. People with the same 2200 elo would be worse than 2200s at the beginning of the season.

No more backup reasoning, I'm going to class. Elo transfers should be dynamic and should change as an elo pool inflates.

Posted on 05 February 2016 - 10:40 AM

Tl;dr?

Last edited on 05 February 2016 - 10:59 AM by tikkalawl

@MasterGberry

Posted on 05 February 2016 - 11:02 AM

Or, you could be on West like me where it's
Ladder A has 33,
Ladder B has 0,
Ladder C has 0,
Ladder D has 0,
Ladder E has 0,
Ladder F has 0,
Ladder G has 0,
Ladder H has 0,
ect.

Posted on 05 February 2016 - 12:56 PM

last paragraph

Posted on 05 February 2016 - 05:07 PM

The theory is correct but wrong at the same time.

In an ideal matchmaking system a 2500 would be paired against 2200's.

Yes there is an inflation factor. There is nothing wrong with it. If you are rank 1 you are rank 1.

Other rating systems might be investigated in the future with season 13 changes.

Posted on 05 February 2016 - 05:41 PM

i fail to see the "problem"

Posted on 05 February 2016 - 05:49 PM

Last edited on 06 February 2016 - 01:12 AM by Hivlik

But global elo takes the average of elos that are based on a different scale. Build UHC is more important than Horse, because it's easier to get a higher elo at an equal skill level, which is not how it is supposed to work.
edit: inflation matters very little within the bounds of a single ladder, but across multiple ladders the inflation factor has big implications

Posted on 06 February 2016 - 10:09 PM

Jesus iCqntRead

Posted on 07 February 2016 - 01:26 AM

First off, I love that you love Elo analysis Hivlik. A man after my own heart.

I wanted to see if the data backed up your argument by graphing "average top ten player elo" against "pages in ladder standings". UNFORTUNATELY, it seems gberry has removed access to all pages past 20, so there goes that! I didn't like this whole "big data" trend our world was heading towards anyway!

Nevertheless, If I had to guess, I'd say that you are correct. Larger elo pool = larger slice to the best players. This does make it hard to compare top players between ladders and between seasons. As @MasterGberry said, if the only thing that mattered was ladder comparison, then yes, one could just look at the rank #.

This doesn't work for global elo, and it is true that achieving a top rank in a very popular ladder is much more rewarding in global elo than the same feat in a smaller ladder. In fact, I don't really like that global ladder uses the same scale as Elo. It isn't Elo.

Some possible solutions to this problem, from easiest to hardest:

• Delete global elo!
• Leave it how it is and suck it up. Players would realize that mastering the right ladders is as important as being good in the first place. (Bad scenario).
• Use a different measure for global elo, like median elo instead of mean. I kinda like this because it allows you to completely suck in a ladder or two, and kick ass in another ladder or two, without distorting your global elo.
• Use the average or median rank # (not rank Elo) to calculate global "rank". This isn't perfect, but would probably reduce inflation significantly.
• Weight the influence of each ladder in the global elo. A formula might be (adjustedElo = ((myElo - 1400)/(playersInLadder)*K)+1400 K is just some constant that makes the elo look realistic for all ladders.

Just some ideas! Interesting question Hivlik!

Posted on 08 February 2016 - 03:27 PM

@qazzy1122

As far as I can see, the current system isn't all that bad yet. Nobody but you and I really analyzes elo to my knowledge, and nobody really knows what elo inflation is. As much as elo is becoming more and more competitive, it's nowhere near the scale at which players actually really get into analyzing it unless they don't have much else better to do, which is pretty much why I'm here. We could keep the current system until people start to manipulate it to shoot up the global board, and I don't see that happening for quite a while, but once it does, there's a change that needs to be made. The problem, though, is that it's hard to find a change that is a) just as easily understandable to the average dummy as global elo but is also b) less influenced by elo inflation. I have also toyed with the idea of a formula to remove the inflation factor when regarding elo, but in reality, it's just far too complex for everybody to understand. I don't like medians, either, because I think outlier elos should still be weighed just as heavily as they are now, just without the notion of the inflation factor. I'd say your last scenario, the one with the formula, is the best way to go about it, because a consistent system is more important than a simple one, but again, it's not perfect.

Posted on 09 February 2016 - 11:30 PM

@Hivlik

Yea it really isn't a problem right now. I'd guess that the top ten global elo wise are most likely the actual top ten pvpers, the top 25 global elo are actually the top 25 give or take, etc.. If it actually becomes a problem it will be worth addressing but right now it's just fun data!

Posted on 09 February 2016 - 11:39 PM

What other cool elo things are there

Last edited on 09 February 2016 - 11:42 PM by qazzy1122

There are tons of fun things to do regarding elo distribution W/L distribution, and other factors. Just gotta get an idea and go investigate!

Posted on 09 February 2016 - 11:44 PM

oh right, an old idea i had was that w/l statistic means absolutely nothing, what's in fact far more important is total # of games played that shows something

Posted on 10 February 2016 - 01:38 AM

I did a (far more basic) analysis of UHC statistics and how immensely topheavy the wins are if you look at leaderboard distribution. I'm very interested in things like this and will likely take a look at it and crunch some numbers when I have the time. It's very interesting to me how Build is used to carry overall elos, and how the "goal" of having all elos be equal to grind for global is impractical and disadvantageous with the current elo dynamic.

Sidenote: I got my highest non-build elo ever on the very last day of season 8. Playing late in the season is a huge advantage.

Posted on 10 February 2016 - 07:59 AM

@Unwise UHC stats are much different. You could win a uhc simply because there's nobody else in the game, but for elo, every time you win a match, somebody else loses that match. Every time you gain an elo, somebody else loses that elo. It lets us know that the average elo is 1400 and the average w/l ratio is 1

Posted on 11 February 2016 - 12:34 AM

@MasterGberry we should have a Badlion Analytics blog somewhere

Posted on 11 February 2016 - 12:44 AM

I wasn't really comparing the two, just expressing an interest in the overall analysis of Badlion leaderboards. Elo is very different from how wins work.