The Instant Feedback System, as the name implies, is the system that analyzes player reports and chat logs and issues punishments when due; all of this happens within 15 minutes from the report that triggers the review [417, 574].
The punishments being issued vary from Chat Restrictions, to bans according to the afflicted account’s behaviour history.
The system initiates analysis on a reported player, and checks the in-game data (such as statistics, chats, item builds) . The reports, however, are not used indiscriminately, but are first filtered to ignore the false (i.e. not justified) reports, which lead the reporter to lose some of their “report value” [179, 227]. Correct reports that do not lead to punishments are “stored” and can be used for the next analysis on the same player account [137, 227].
Behavioural standards that influence report validity are not dictated by Riot, but by the players themselves: this is accomplished through data gathering from old Tribunal, reports, honours and social media [80, 389]. For example, “gg ez” has become a reportable option fairly early in the system’s lifespan specifically thanks to this unique method [80, 144]. These data points also help Riot refine the Instant Feedback System, in order to catch and punish players that otherwise would stay unpunished for too long . Speed of the response is the key here: feedback being delivered practically instantly allows players to reform at a much higher rate, than punishing them months after the facts [454,87].
This method also has the added value of updating the system very quickly whenever behavioural trends shift ; however, while the system adapts itself using reports, any newly found (and reported) bad behaviour gets manually checked, to ensure that it has been correctly perceived as toxic .
Machine learning is used to feed the Instant Feedback System with data on the context of phrases [149, 175], but even while taking context into account, players in a premade aren’t safe from punishments, if they use offensive language in a setting where other players aren’t aware that they’re talking to friends [171, 580]. Full premades don’t get automatically checked .
It’s a fairly common misconception that the chat log analysis only involves the in-game chat, but pre- and post-game chats are also recorded, but rarely used to review a case .
Usually, offenses are punished with an escalating system that starts from Chat Restrictions and goes up to 14 days and permanent bans , with the added ability to override punishments with heavier penalties if necessary ; homophobia, sexism and racism are considered particularly severe offenses , despite being rare (only 2% of matches globally) , and can get detected and punished with a 14 days or permanent ban , even without prior warning or punishment [254, 255, 344]. The reason being offered for such severity is that such remarks are “overwhelmingly rejected by the community, and hard to mistake” .
While it’s true that the system punishes toxicity, it actually doesn’t punish profanity -- not even vulgar words that belong to the in-game culture [305, 548] -- but it concentrates on both common and culture-related malicious or insulting phrases [259, 305, 353]. According to Riot, thanks to it there was a drastic reduction of racist, homophobic and sexist remarks, accompanied by a rise of common profanity, such as “fuck” or “shit” .
The reason for choosing only four steps of punishments was stated to be reform rates, which plummet after simple warnings . Each punishment is one chance less for the affected player to reform, because permanent bans due to behaviour are never lifted, unless an error occurred .
A pop up informing a player of a punishment being dealt to a reported offender might appear. It was initially set to be seen only by the player whose report triggered the punishment [46, 68, 127], but it has been changed since then, and doesn’t even appear every time action has been taken . Riot stated that tuning it to pop more often would have been seen akin to spamming, since about 20% of the community wasn’t interested in those pop ups , and the would have been worse after the inclusion of Chat and Ranked Restrictions , but this last part of the plan was never implemented.
The system is tuned fairly conservatively to avoid false positives as much as possible [65, 126]. It doesn’t scan private conversations by design; the reason given being that context there is much more difficult to discern .
The system was intended to have a 0,1% error rate since launch [260, 472], to minimize false positives, and accuracy is ensured through case sampling, performed by Player Support employees [175, 472, 602]. One of these quality checks in the very beginning of this system showed that 1 out of 6000 punishments is undeserved .
Since the introduction of the Instant Feedback System, verbal abuse has dropped by more than 40%, and 91,6% of negative players change their attitude after the first punishment, thanks to fast feedback and Reform Cards, therefore avoiding subsequent penalties [20, 67, 589, 601]. According to Riot, warnings are enough to convince 75% of the players to improve their behaviour, thus avoiding the penalty system altogether . Ranked games experienced a 40% drop in toxicity after the launch of the system .
This is important because the reform rate is very high, compared to traditional systems of manual reviews (which scored a 50% reform rate). Adding evidence, such as chat logs, to the punishment increases the reform rate to 75% , thanks to players getting a clear picture of what got them punished.
Another accomplishment is that it managed to basically equalize player behaviour standards across all servers, even those that never ran a Tribunal to begin with. European servers are considered to be more difficult to keep in check, due to the huge variety of languages spoken across the region [125, 475].
An interesting phenomenon about players not reporting behaviour that the Instant Feedback System believes to be toxic, has put onto the table the problem of understanding how the community thinks on certain issues .
67 http://gdcvault.com/play/1022160/More-Science-Behind-Shaping-Player [min 19:05 - 20:30]
80 https://ask.fm/RiotLyte/answers/132069455298 ; Page 140
117 https://ask.fm/RiotLyte/answers/130575469250 ; Page 326
122 https://ask.fm/RiotLyte/answers/130576421570 ; Page 324
125 https://ask.fm/RiotLyte/answers/130577031106 ; Page 324
126 https://ask.fm/RiotLyte/answers/130577014978 ; Page 324
127 https://ask.fm/RiotLyte/answers/130577014978 ; Page 324
171 https://ask.fm/RiotLyte/answers/130581249730 ; Page 324
175 https://ask.fm/RiotLyte/answers/130588741826 ; Page 316
179 https://ask.fm/RiotLyte/answers/130598831042 ; Page 315
182 https://ask.fm/RiotLyte/answers/130601189570 ; Page 314
278 https://ask.fm/RiotLyte/answers/134527663810 ; Page 6
279 https://ask.fm/RiotLyte/answers/134527205570 ; Page 7
304 https://ask.fm/RiotLyte/answers/130594064578 ; Page 312
305 https://ask.fm/RiotLyte/answers/130601915586 ; Page 311
330 https://ask.fm/RiotLyte/answers/134185966530 ; Page 28
336 https://ask.fm/RiotLyte/answers/132195767490 ; Page 121
376 https://ask.fm/RiotLyte/answers/130998053570 ; Page 210, 211
389 https://ask.fm/RiotLyte/answers/130717609922 ; Page 283
417 https://ask.fm/RiotLyte/answers/130722330562 ; Page 277
460 https://ask.fm/RiotLyte/answers/132338559426 ; Page 107
495 https://ask.fm/RiotLyte/answers/131165741506 ; Page 140
540 https://ask.fm/RiotLyte/answers/133520179906 ; Page 61
546 https://ask.fm/RiotLyte/answers/133489071298 ; Page 64
548 https://ask.fm/RiotLyte/answers/133478793154 ; Page 64, 65, 230, 24
557 https://ask.fm/RiotLyte/answers/132976907714 ; Page 77
574 https://ask.fm/RiotLyte/answers/132567836098 ; Page 133
580 https://ask.fm/RiotLyte/answers/132066956738 ; Page 243
584 https://ask.fm/RiotLyte/answers/132037756866 ; Page 152
589 https://ask.fm/RiotLyte/answers/131818129602 ; Page 163
591 https://ask.fm/RiotLyte/answers/131632237250 ; Page 167
594 https://ask.fm/RiotLyte/answers/131560051650 ; Page 118, 171
601 https://ask.fm/RiotLyte/answers/131185401794 ; Page 189
602 https://ask.fm/RiotLyte/answers/131188058306 ; Page 189