Tier Talk
What are card tiers? What is the right way to assign cards to a tier? And how do we ensure the right reward always ends up in the right persons’ hands?
In all the years I’ve been on Sorare there have always been issues with the rewards. Complaints used to be more frequent (and hostile), it’s not as hot of an issue these days, but the underlying problem is definitely still there.
[Side note, the introduction of essence hid much of the problem because they only really reward Stars, Tier 1's, and Tier 2's now. The higher volume of rewards in Tier 3’s, 4’s, and 5’s are obfuscated away via Standard essence crafts, so you rarely even realize or care if it was improperly assigned, you just notice whether you selected the best card available.]
Even when card tiers were at the forefront of debate, I don’t recall seeing too many proposals on HOW to fix them, it was usually just empty complaints because most of the time it felt like a common sense issue.
The problem is, it is not feasible to manually sort thousands of players into tiers in real time, every single week. So even though it might seem like common sense if a person were to do it, you need to program the logic to do it systematically. And what logic to use can also be subjective.
Let’s break it down.
What are card tiers?
Groups of players with a similar level of desirability. [Note the word choice]
Why do we bother putting them in tiers at all?
That way we can ensure users that finish higher in the leaderboards get rewarded with the more desirable cards.
Why not just give the 1st place finisher the most desirable card, and the 2nd place finisher the second most desirable card and so forth?
Technically you could. This was a debate in the past, but I’d argue it is less optimal that way.
From Sorare’s perspective, instead of just needing broad strokes of player ranking accuracy, you open yourself up to everybody nitpicking individual rankings, all the time. The complaints would be endless because most individual rankings are subjective.
From a User’s perspective, you do get some satisfaction finishing higher than someone else if you know the card is better, but you also sacrifice the joy that comes from the random opening of the card, and if the card below you is actually the one you want you might find yourself dissatisfied with finishing higher. You should always be happy to finish as high as possible.
So what is the right way to assign cards to a tier?
Well even though in the end we just need broad strokes of player categories for tiers, in reality we still have to rank / sort the players behind the scenes and then decide where the imaginary tier cutoffs are.
So how do we rank players first?
Remember how I intentionally chose the word ‘desirability’ before? I chose an ambiguous word because desire is subjective.
Some people think the cards that improve a team/gallery the most [ie. a high scorer] should be ranked higher, others think a card that they can sell for the most should be ranked higher. Often times those things overlap, but never exactly, and there are always exceptions (ex. injured stars).
No matter how you rank them, some people will always disagree, but it should be easily understood and consistently applied.
[The main suggestion I’ve seen in the past is just using the (limited) card prices as a proxy for all the other player variables — position, supply, stats, etc.]
For me, ranking on price and then saying some arbitrary price percentile like top 20% are Tier 1, next 20% are Tier 2, next 20% are Tier 3, etc. doesn’t feel as defensible logically and goes against the nature of the game. We are here to play a game first, and that means winning cards that help us in the game, and only if the card doesn’t help us are we really looking to sell it.
[Also imagine explaining those tiers to someone new: ‘Why is it a Tier 3 instead of Tier 4? Oh it just means these cards are more expensive than these other cards. See this one is above the 60th percentile card cost which is $3.22 right now so it gets classified into the 3rd tier.’]
Lame.
[Also I have no idea what logic they use currently and I just made up those numbers as an example.]
What I think is a better representation of the game and easier to understand / defend, is going off of the quality of the player.
Unfortunately it is not quite as simple as just using their average scores (as I’m sure Sorare realized long ago).
Using my beloved Sounders as a case study… sorting them on L15
We quickly learn scores can’t be the sole criteria, as avg scores include the games regardless of whether the player started, subbed or did not play.
The top player ranked is a player that has barely played (due to injury). We have to weigh in if they regularly play. Do we care whether they start or just sub on? Absolutely. Using % played can be misleading because what if someone is a regular 60th minute sub on? I would rank someone that only plays 80-90% of games but as a starter higher vs someone that plays 100% of games but mainly as a sub.
Get Bhab’s stories in your inbox
Join Medium for free to get updates from this writer.
Sorting them first on # of Starts (with a secondary sort on L15 score)
It’s not perfect, but incorporating who regularly starts gets us a bit closer.
What about if we combine scoring and starting?
(Not an actual proposal but just to test some theory I multiplied L15 score by how many starts and then sorted so we could weigh them together.)
Again, still not a perfect idea, but closer. I can see some challenges / exceptions still. Like how to evaluate Andrew Thomas vs Stefan Frei who have basically been 50:50 over this time span. Or how to handle good players that have missed stretches of games with injuries.
I could keep going but the core of what I’m getting at is the rankings should primarily reflect the real world, and reflect Sorare (price) secondarily.
For a common sense base structure I would propose something like:
Essentially sorting first by Starters then Scores then Prices. Regular starters become Tier 2’s that get upgraded to Tier 1 or Star based on Scoring and Pricing… bench players become Tier 4’s that get upgraded to Tier 3 based on Scoring… and players that don’t make the 18 are Tier 5’s.
Sorare is already set up to group similar players — just look at the Playing Status filter on the Scouting page. (FYI @ Sorare — looking at Sounders players as an example, whatever current logic you have for this ain’t it).
Again, this would be my base model I would architect from and continue to fine tune until it was more robust.
How do we ensure the right reward always ends up in the right persons hands?
No system is ever perfect, and especially with something as dynamic as this there are bound to be errors and exceptions to the norm.
In any process you have the standard ‘happy’ path for how things go when it works according to plan, and then you have the exception handling paths for how things work when it does not go according to plan. But each time things don’t go according to plan, you can still learn from and incorporate amendments to the process so regardless of the scenario there is a standardized response to it the next time.
Reminder: The real risk to the ecosystem occurs when the true tier (desirability) of a card awarded is lower than what was expected to be given out. If a card gets misclassified in a way that the user is given a better card than expected, while it is still technically a flaw in the system you don’t have the same negative repercussions.
Scenario 1: A player transfers teams and will be going from a bench warmer to regular starter.
Decision: Do you let the player naturally work their way up the tiers via starter % / scoring? Or do you intervene and create a subprocess to manage tiering during transfers?
Risk: Low risk since if you don’t change their tier someone being rewarded a Tier 3/4 might be ecstatic they are given a player now performing at a Tier 1/2 level.
Scenario 2: A good player gets injured and will be out for a while.
Decision: Do you let the player naturally fall down the tiers via lower starter %? Or do you intervene and create a subprocess to manage injured players?
Risk: High risk since if you don’t do anything someone being awarded a Tier 1/2 might be angry they are given a player as useful as a Tier 5.
See how the emphasis should be on preventing cards from being assigned higher than reality? That is the major pain point. If you over compensate to one side that is the side to get right.
Regardless of the decisions though the core tiering process should stay the same, but it is important to establish a rule and process to handle all similar exceptions in the future. Everyone is happier when they know what to expect.
To Recap:
Define what card tiers actually mean — based on a common sense reality of player quality.
Establish a clear and objective process on how tiers get consistently assigned.
Communicate how (and when) outliers are manually handled.
Thanks for coming to my Tier Talk
Appendix
“Stars” are especially important to get right, as that is the peak reward to showcase and be excited about, and while top X players by price within the Tier 1 pool is one option, it also could warrant a white glove service being curated by Sorare weekly (similar to how which Unique cards get auctioned is curated). Or something more challenging but engaging would be to create a mechanism for it to be curated by the community (every month?).
Just a thought.
Also, which tier a player is in should be constant across all scarcities. If you don’t want to award / don’t have a Tier 1/2 to award in Unique during a midweek, don’t pawn off a shit player that’s Tier 4 in Limited and claim as a Tier 1 in Unique and insult our intelligence. The card is what it is, just say you are awarding a Tier 4 if that’s what you’re really giving out. Honesty and transparency go a long way.
Alright that’s it I’m out. Go Sounders.
