Riot Games is Violating California Employment Law

An interesting study comes trolling across my twitter feed claiming that there’s a correlation between workplace behavior and in-game behavior at Riot games. I have a friend who works there so I thought, “Hey, that’s a neat study let’s see what they did.” What I read in the article turns out to be a massive employment law violation on many levels and I predict Riot will be sued for this and other practices in the near future.

In the article they describe how they went into employee’s past game behavior, mined it for “toxic speech” and then used that information to correlate with the past employees who have been fired. The article mentions “passive aggression” as one of the toxic behaviors, so keep this in mind as I discuss how illegal this is. People were being identified based on either a flawed sentiment analysis of how snarky they were, or a manager’s assessment of whether they were being snarky.

This was then used to identify other employees who needed to either be reprimanded or straight up fired:

Riot also found that a player’s toxicity was a fluid thing and not immutable. Like moods, toxicity levels can fluctuate. Riot could measure and predict toxicity trajectories of players over time, and so they set about seeing if they could improve the player behavior of their employees.

Riot identified the 30 most toxic employees (all of whom were more junior Rioters, new to the working world) and classified them into two categories:

• this person needs a stern warning
• this person should leave Riot, because their in-game chat was unusually toxic

Look at that last bullet point. The article says they used an analysis of employee’s game play to decide if they should be fired. This game play was based on behavior the employee did not know would be used to fire them and was collected without the employees' explicit permission. In addition to this, the employees are most likely expected to play the game or have to play it to do their jobs.

Riot games is now in the process of turning this into a way to exclude future employees and to automate this and place it in their applicant tracking system without informing the applicant about their decision. To quote the article :

“Riot is a quickly growing company with managers eager to fill seats, but the organization wants to make sure they don’t make any bad hires. The team is now trying to figure out how to use this in-game information as a signal during the hiring process. Riot asks applicants for their in-game handle during the application process so they can review their gameplay and identify any toxic chats and behaviors.

Riot is experimenting with displaying this information in their applicant tracking system using a simple stoplight code - red, yellow, green. For any applicant flagged “red” (the most toxic), there are sample chat logs so a recruiter or hiring manager can see how an applicant conducts themselves in the very product they’re hoping to work on."

All The Things Wrong

1. California Labor Code 980 (link) states they can’t do this, except that there’s a section that states, "Nothing in this section precludes an employer from requiring or requesting an employee to disclose a username, password, or other method for the purpose of accessing an employer-issued electronic device.” Which means a phone or laptop the employee was given, not anything else. The gray area here is in the fact that Riot games happens to own the system being mined, so that’s where a case would hinge. Given the spirit of the law I believe that Riot is violating it.

2. Their use of statistics to bucket employees into “reprimand” or “fire” buckets is incredibly dubious and I’m guaranteeing if I saw their methods I could rip them to shreds in a few minutes. I only have a couple of graduate level sociology classes and I can already see huge problems with this study, namely the classification of the speech as “passive aggressive” when categorizing something as vague as snark is incredibly difficult. Either this study was done by amateurs who know nothing of sociology survey design and experimental design, or the study was done by trained Sociologists and those people have violated human trials requirements and should be held accountable.

3. Before you can study any humans you have to meet a very strict set of rules to protect people from abuse by scientific authority. The Stanford Prison experiments are the reason why we have these requirements and if you ever want to see how abusive scientists were in the early days just read about that study. To conduct experiments on people you need to get their permission, follow strict guidelines, and provide counseling. If you have to trick them then it’s even harder to get approval for it. The repercussions for not following trials are vague though, but can be everything from a pulling of funding all the way to prison or lawsuits.

4. Ethically the study violates the rights of these people because it is using data based on their behavior without their knowledge that it would be used to fire them. This is key because, if you tell them that they’ll be fired for acting like dicks in the game then they’ll most likely stop doing it. However, if you go into the past before they knew it’d be used against them then you are both violating their right to a private non-work life, and their human trials/study rights to not be experimented on without their permission. Additionally using this to fire them violates human studies requirements to provide counseling and not harm the respondent (employees).

5. You notice how the article only talks about managers analyzing employees? The huge problem with this ethically and potentially legally is that this data is only being used to fire employees. The managers probably don’t even play the damn game. If managers do know that it gets used to fire people. I’ll bet you $100 the second this study was discussed all the managers went and deleted their history. I’ll also bet you there are zero managers fired because of this. In the same way that Netflix claims to fire all the “B players” but keeps Reid Hoffman around after he tanks the company with an email, Riot will keep managers around even after they abuse the shit out of their direct reports.

6. The probability that you’ll have bad things in your Riot Games past is proportional to how much you play, so this targets only the employees who play the game all day because their job requires it. Not every employee at Riot plays the game. I’m pretty sure there’s a decent number of support staff who could give a shit about the game and just want to do their job. That leaves employees in testing, development, and other support roles who have to play the game all day long. Given enough time, Riot will find some dirt on any employee that they can use to fire anyone they like. All Riot has to do is mandate everyone play it, and then wait to build the dossier to use when they want to fire them.

7. Remember that this was done to every employee that played the game. People will focus on the small percentage of toxic people and forget that Riot grabbed the chat logs of every person at the company to figure out who to fire without those employee’s permission. I’ve never played their game, but if the game supports audio and headsets then it’s conceivable that they could record the audio of employees while they played the game in their own home before joining the company. Think about that for a second. An employer could have potentially recorded the audio of your home to figure out if they should fire you or not. Not even the government has that right so it’s easy to say that no company should either.

8. Finally, most states have laws against arbitrary tests to get or keep a job. This comes from the era when employers used IQ tests to exclude non-whites from jobs. To understand how this happens you have to understand that the Stanford-Binet IQ test was explicitly created to re-segregate the California school system and prove that whites were better. One question in the early test asks, “A man comes into town but his feet do not touch the ground. What is he riding?” The story goes that a young Native American boy responded, “A horse” to which the tester yelled, “Wrong! It’s a bicycle!” That’s how the test excludes people who just aren’t from your home town with your culture, and using it or other super screwed up home grown tests was how employers kept non-whites from taking white people’s jobs. We outlawed using unfair tests to determine employment because it is way too easy to abuse. Tests that confirm you can do a skill are sometimes valid, but even then you have to be careful because if they’re based on cultural elements you can be in trouble. In this case it might be possible to prove that Riot games used their game as a cultural test to exclude people of a particular gender or race. If there is a trial, they should hire a sociologist to figure that out.

Know Your Rights

This act by Riot is something that deserves a class action law suit by every employee there. There are also ethical problems with the researchers who did this (assuming they used real Ph.D. types) because of human trials requirements in Sociology and Psychology. As I mentioned before, either they used real Ph.Ds and those people knowingly violated human trials; or, they used dumbass “data scientists” and then the test is an unfair invalid test that can’t be used. In fact, I would add the researchers to any lawsuit and make sure they are unable to receive government grants in the future.

More importantly though, I have no fucking idea why this is being touted as something awesome. Programmers need to get their heads out of Ayn Rand’s ass and stop worshipping corporations as some sort of infallible gods. Corporations do not have your best interest at heart, and at best they’re giving you a place to sit and do some job while they figure out how to fuck you over. It seems to me that too often a programmer will rant about how terrible the government is and how they violate people’s privacy and then turns a blind eye while a corporation does the exact same shit or worse.

More from Zed A. Shaw