Player Behavior Knowledge Base > Instant feedback system

Instant feedback system

1. About

The Instant Feedback System, as the name implies, is the system that analyzes player reports and chat logs and issues punishments when due; all of this happens within 15 minutes from the report that triggers the review [417, 574].

The punishments being issued vary from Chat Restrictions, to bans according to the afflicted account’s behaviour history.


2. Implementation

The system initiates analysis on a reported player, and checks the in-game data (such as statistics, chats, item builds) [227]. The reports, however, are not used indiscriminately, but are first filtered to ignore the false (i.e. not justified) reports, which lead the reporter to lose some of their “report value” [179, 227]. Correct reports that do not lead to punishments are “stored” and can be used for the next analysis on the same player account [137, 227].

Behavioural standards that influence report validity are not dictated by Riot, but by the players themselves: this is accomplished through data gathering from old Tribunal, reports, honours and social media [80, 389]. For example, “gg ez” has become a reportable option fairly early in the system’s lifespan specifically thanks to this unique method [80, 144]. These data points also help Riot refine the Instant Feedback System, in order to catch and punish players that otherwise would stay unpunished for too long [117]. Speed of the response is the key here: feedback being delivered practically instantly allows players to reform at a much higher rate, than punishing them months after the facts [454,87].

This method also has the added value of updating the system very quickly whenever behavioural trends shift [389]; however, while the system adapts itself using reports, any newly found (and reported) bad behaviour gets manually checked, to ensure that it has been correctly perceived as toxic [495].


Machine learning is used to feed the Instant Feedback System with data on the context of phrases [149, 175], but even while taking context into account, players in a premade aren’t safe from punishments, if they use offensive language in a setting where other players aren’t aware that they’re talking to friends [171, 580]. Full premades don’t get automatically checked [546].

It’s a fairly common misconception that the chat log analysis only involves the in-game chat, but pre- and post-game chats are also recorded, but rarely used to review a case [540].


Usually, offenses are punished with an escalating system that starts from Chat Restrictions and goes up to 14 days and permanent bans [344], with the added ability to override punishments with heavier penalties if necessary [591]; homophobia, sexism and racism are considered particularly severe offenses [203], despite being rare (only 2% of matches globally) [221], and can get detected and punished with a 14 days or permanent ban [32], even without prior warning or punishment [254, 255, 344]. The reason being offered for such severity is that such remarks are “overwhelmingly rejected by the community, and hard to mistake” [224].

While it’s true that the system punishes toxicity, it actually doesn’t punish profanity -- not even vulgar words that belong to the in-game culture [305, 548] -- but it concentrates on both common and culture-related malicious or insulting phrases [259, 305, 353]. According to Riot, thanks to it there was a drastic reduction of racist, homophobic and sexist remarks, accompanied by a rise of common profanity, such as “fuck” or “shit” [304].

The reason for choosing only four steps of punishments was stated to be reform rates, which plummet after simple warnings [351]. Each punishment is one chance less for the affected player to reform, because permanent bans due to behaviour are never lifted, unless an error occurred [557].


A pop up informing a player of a punishment being dealt to a reported offender might appear. It was initially set to be seen only by the player whose report triggered the punishment [46, 68, 127], but it has been changed since then, and doesn’t even appear every time action has been taken [618]. Riot stated that tuning it to pop more often would have been seen akin to spamming, since about 20% of the community wasn’t interested in those pop ups [279], and the would have been worse after the inclusion of Chat and Ranked Restrictions [127], but this last part of the plan was never implemented.


The system is tuned fairly conservatively to avoid false positives as much as possible [65, 126]. It doesn’t scan private conversations by design; the reason given being that context there is much more difficult to discern [65].


3. Successes

The system was intended to have a 0,1% error rate since launch [260, 472], to minimize false positives, and accuracy is ensured through case sampling, performed by Player Support employees [175, 472, 602]. One of these quality checks in the very beginning of this system showed that 1 out of 6000 punishments is undeserved [250].


Since the introduction of the Instant Feedback System, verbal abuse has dropped by more than 40%, and 91,6% of negative players change their attitude after the first punishment, thanks to fast feedback and Reform Cards, therefore avoiding subsequent penalties [20, 67, 589, 601]. According to Riot, warnings are enough to convince 75% of the players to improve their behaviour, thus avoiding the penalty system altogether [351]. Ranked games experienced a 40% drop in toxicity after the launch of the system [67].

This is important because the reform rate is very high, compared to traditional systems of manual reviews (which scored a 50% reform rate). Adding evidence, such as chat logs, to the punishment increases the reform rate to 75% [376], thanks to players getting a clear picture of what got them punished.


Another accomplishment is that it managed to basically equalize player behaviour standards across all servers, even those that never ran a Tribunal to begin with. European servers are considered to be more difficult to keep in check, due to the huge variety of languages spoken across the region [125, 475].


An interesting phenomenon about players not reporting behaviour that the Instant Feedback System believes to be toxic, has put onto the table the problem of understanding how the community thinks on certain issues [330].


4. Miscellaneous

  • While the current Instant Feedback System works exclusively as a punishment system, at some point there were plans to include positive reinforcement, in the form of making the system learn and identify positive behaviour through reports and honours [187, 207, 208, 229] and send prizes to players [122, 208, 209]. Furthermore, with the new Tribunal voting system players would have been able to review both positive and negative behaviour [208, 209], and even receive a monthly “report card” showing the voters’ contributions for the betterment of the community [278]. It is currently unknown whether these plans are still in development, or have been iceboxed.
  • Another, less known, function of the system is that the data it collects is used by player support specialists, scientists, designers to review dubious cases, or analyze behavioural trends that can help designing new social systems [227].
  • Ranked Restrictions never impacted the overall toxicity in Normal Draft Pick [594].
  • One way to avoid restrictions would be not using the chat altogether, since smart pings are enough to communicate most information in game when used intelligently, and most players don’t use the chat anyway [182, 336, 584].
  • PBE used to be devoid of behaviour systems, and players exploited this loophole by performing all kinds of bad behaviours that are normally punished on the regular servers, which would in turn show how servers would look like if they didn’t have any punishment system implemented [460].
  • Asking other people to report someone is useless, since just one report already triggers an investigation of that players behavior and more reports don't change anything [650]


5. Sources

67 [min 19:05 - 20:30]
80 ; Page 140
117 ; Page 326
122 ; Page 324
125 ; Page 324
126 ; Page 324
127 ; Page 324
171 ; Page 324
175 ; Page 316
179 ; Page 315
182 ; Page 314
278 ; Page 6
279 ; Page 7
304 ; Page 312
305 ; Page 311
330 ; Page 28
336 ; Page 121
376 ; Page 210, 211
389 ; Page 283
417 ; Page 277
460 ; Page 107
495 ; Page 140
540 ; Page 61
546 ; Page 64
548 ; Page 64, 65, 230, 24
557 ; Page 77
574 ; Page 133
580 ; Page 243
584 ; Page 152
589 ; Page 163
591 ; Page 167
594 ; Page 118, 171
601 ; Page 189
602 ; Page 189

Created by Coxis on 16-01-2017. Last edited: 28-06-2017