Player Behavior Knowledge Base > Old Tribunal - General Information

Old Tribunal - General Information



 

1. Summary

 

The old Tribunal was Riots first attempt to provide the community with a system to fight toxic behavior. It was a crowdsourcing system in which players could review chatlogs and game data of reported players and judge whether this player should or should not be punished. The term “Tribunal” has various different meanings, which are explained here. This article only refers to the old Tribunal voting system.

 

2. Functionality

 

The main purpose of the Tribunal was to let the players (instead of Riot Games) be the ones who decide what is okay and what is not and define social norms, as it was described as a powerful way for the community to set behaviour standards [25, 26, 177, 186]. It was theorized to be an effective approach in fighting toxicity [98]. In addition, the data from the old Tribunal was also used to teach the artificial intelligence of the current Instant Feedback System [53, 390].

The system was in effect for all queues and game modes, except for custom games [249]. There were six levels of punishments [343].

There was a minimum time of 30 seconds before the vote could be cast on a case, therefore preventing vote spam. Riot was also in possession of the means to check whether a judge was reviewing the case, or simply voting without reading [415].

 

3. History of the Tribunal

 

The initial implementation of the Tribunal granted a 10 IP reward for each solved case, but Riot eliminated it in order to push for more positive reinforcement. The end result was that more players started being involved in the Tribunal and solving more cases [90, 102]. However, only players with level 20 accounts or above could examine cases [105].

In the end the Tribunal was closed because of it was simply too slow. On average it took a week to punish a player for toxic behavior [30].

 

Some servers never had a Tribunal running because of the huge costs of implementation, due to hardware and website specs that differed in each region and to the constant technical issues that afflicted the system in the regions where it was running, forcing Riot to keep weekly live support that subtracted resources from development [286].

 

4. Success and failure of the Tribunal

 

Potential abuse was a problem many players often worried about, but never really was a big problem for the Tribunal. Players spamming “punish” or “pardon” didn’t actually happen a lot, and even when it happened those players simply lost their voting rights [47, 141].

 

The introduction of the Reform Cards allowed punished players to learn the reason for their punishments and improve their behaviour, with the result that they got reported less often [101].

 

The biggest downside of the Tribunal was its speed. In the end it took about a week on average to punish a player for toxic behavior [30]. While, in average, reform rates were up to 70%-75% when players are punished swiftly and with evidence of their bad behaviour, taking too long meant that the effect of the punishment was almost unnoticeable [57, 123]. The Instant Feedback System was developed in response to this problem [103].


5. Additional Data

 

Data from May 2013 shows that over 105 million votes were cast, and over 280.000 players successfully reformed after receiving a Tribunal punishment [99, 234]. Roughly 100-150 votes were collected for every case [477].

In order to be judged by the Tribunal you need to get reported several hundred times [476]

Less than 1% of the Tribunal voters were casting wrong votes on purpose [105].

The smaller (1-3 days) bans of the old Tribunal showed that people weren’t affected much, while 14 days and upwards bans were more impactful in terms of reform [123].

 

About 95% of the players never received a Tribunal punishment (same as with the current system) [256].

 

6. Random Facts

 

At some point, Evelynn players used to be reported and end up in the Tribunal 300% more often than any other champion players, but it didn’t equate to an increase of punishments - Tribunal voters correctly pardoned those cases where no toxicity was displayed. In turn, this is a demonstration of the accuracy of the old Tribunal [121].

 

The introduction of the Instant Feedback System to servers that were never under the old Tribunal’s umbrella brought forth a huge shift in player behaviour in those servers [125].

 

In 2012 a player analyzed several thousand Tribunal cases, providing some additional data that can be seen here: [516]

In addition this paper brought up a lot of data about reporting behavior and judgement in the Tribunal: [480]

 

7. Sources

 

25 http://d1at8ppinvdju8.cloudfront.net/9/965/show_9965811_2017_04_17_22_19_03.mp3 as found on The Psychology of Video Games Podcast 4: Toxic Behaviour
26 http://d1at8ppinvdju8.cloudfront.net/9/965/show_9965811_2017_04_17_22_19_03.mp3 as found on The Psychology of Video Games Podcast 4: Toxic Behaviour
30 http://d1at8ppinvdju8.cloudfront.net/9/965/show_9965811_2017_04_17_22_19_03.mp3 as found on The Psychology of Video Games Podcast 4: Toxic Behaviour
47 https://www.reddit.com/r/leagueoflegends/comments/3h663c/askfm_riot_lyte_within_the_next_few_patches/cu4mru4
53 http://www.the-vital-edge.com/artificial-intelligence-behavior/#_=_
57 http://gdcvault.com/play/1022160/More-Science-Behind-Shaping-Player
90 http://d1at8ppinvdju8.cloudfront.net/9/965/show_9965811_2017_04_17_22_19_03.mp3 as found on The Psychology of Video Games Podcast 4: Toxic Behaviour
98 https://www.youtube.com/watch?v=nbYQ0AVVBGU (12:30)
99 https://www.youtube.com/watch?v=nbYQ0AVVBGU (14:10-15:20)
100 https://www.youtube.com/watch?v=nbYQ0AVVBGU (16:40-17:20)
101 https://www.youtube.com/watch?v=nbYQ0AVVBGU (18:10-19:00)
102 https://www.youtube.com/watch?v=nbYQ0AVVBGU (20:00-22:15)
103 https://www.youtube.com/watch?v=nbYQ0AVVBGU (23:25-25:25)
105 https://www.youtube.com/watch?v=nbYQ0AVVBGU (35:40-36:15)
121 https://ask.fm/RiotLyte/answers/130576333250 ; Page 324
123 https://ask.fm/RiotLyte/answers/130576396226 ; Page 324
125 https://ask.fm/RiotLyte/answers/130577031106 ; Page 323
141 https://www.reddit.com/r/leagueoflegends/comments/36xcya/banned_for_literally_nothing/crhy7a3
177 https://ask.fm/RiotLyte/answers/130589272002 ; Page 316
186 http://na.leagueoflegends.com/en/news/game-updates/player-behavior/upgrading-tribunal
234 http://gdcvault.com/play/1017940/The-Science-Behind-Shaping-Player
249 https://www.youtube.com/watch?v=a-5yKdpR0kU (09:35-30:42)
256 https://www.reddit.com/r/leagueoflegends/comments/36ow3s/new_player_reform_system_heads_into_testing/crg153u
286 https://www.reddit.com/r/leagueoflegends/comments/369a4k/reporting_feels_so_useless/crccl9s
343 https://www.reddit.com/r/leagueoflegends/comments/369a4k/reporting_feels_so_useless/crc0ite
390 https://ask.fm/RiotLyte/answers/130717576898 ; Page 283
415 https://ask.fm/RiotLyte/answers/130729829826 ; Page 274
476 https://arxiv.org/abs/1504.02305
477 https://arxiv.org/abs/1504.02305
480 https://arxiv.org/abs/1504.02305
516 http://forums.na.leagueoflegends.com/board/showthread.php?t=2445574

Created by on 16-01-2017. Last edited: 21-02-2018