It has gotten so bad that our industry should stop saying “evil maid” and say “evil SRE” instead. Anyway, has anyone made a list? I haven’t seen a good one yet so here’s a handy reference for those asking.
First, before I get into the list, don’t forget that engineers are expected to plan for people doing things incorrectly. It is not ok for any cloud provider to say they accept no responsibility for their own staff causing harm.
Do not accept the line that victims should be entirely responsible. Amazon has been forced to back down on this before.
After watching customer after customer screw up their AWS S3 security and expose highly sensitive files publicly to the internet, Amazon has responded. With a dashboard warning indicator…
And in the case of an ex-AWS engineer attacking a customer, the careless attitude of Amazon has become especially relevant.
“The impact of SSRF is being worsened by the offering of public clouds, and the major players like AWS are not doing anything to fix it,” said Cloudflare’s Evan Johnson. Now senators Ron Wyden and Elizabeth Warren have penned an open letter to the FTC, asking it to investigate if “Amazon’s failure to secure the servers it rented to Capital One may have violated federal law.” It noted that while Google and Microsoft have both taken steps to protect customers from SSRF attacks, “Amazon continues to sell defective cloud computing services to businesses, government agencies and to the general public.”
Second, a more complicated point regularly put forth by cloud companies including Amazon is how “at the time” means they can escape liability.
When an Uber driver brutally killed a little girl in a crosswalk, Uber tried to claim their driver “at the time” wasn’t their responsibility.
Attorneys for Uber said the ride-sharing company was not liable … because the driver was an independent contractor and had no reason to be actively engaged with the app at the time. […] “It’s their technology. They need to make it safe,” [family attorney] Dolan said in January, suggesting that a hands-free mode would bring the business into compliance with distracted-driving laws.
Uber paid the family of the dead girl an undisclosed sum a year later “to avoid a trial about its responsibility for drivers who serve its customers”.
Lyft ran into a similar situation recently when “rideshare rapists” around America operated “at the time” as private individuals who just happened to have Lyft signage in their car when they raped women. The women were intoxicated or otherwise unable to detect that drivers “at the time” were abusing intimate knowledge of gaps in a cloud service trust model (including weak background checks).
Imagine a bank telling you they have no responsibility for lost money in your account because “at the time” the person stealing from you was no longer their employee. It’s not quite so simple, otherwise companies would fire criminals in process and be able to instantly wash their hands without stopping actual harm.
Let’s not blindly walk into this thinking insiders, such as SRE, are off the hook when someone throws up pithy “at the time” arguments.
And now for the list
1) AWS engineer in 2019
The FBI has already arrested a suspect in the case: A former engineer at Amazon Web Services (AWS), Paige Thompson, after she boasted about the data theft on GitHub.
Remember how I said “at the time” arguments should not easily get people off the hook? The former engineer at AWS, who worked there around 2015-2016, used exploits known since 2012 on servers in 2019. “At the time” gets fuzzy here.
Let the timeline sink in, and then take note customers were making mistakes with Amazon for many years, and their staff would have front row seats to the vulnerabilities. Other companies treated the same situation very differently. “Little Man in My Head” put it like this:
The three biggest cloud providers are Amazon AWS, Microsoft Azure, and Google Cloud Platform (GCP). All three have very dangerous instance metadata endpoints, yet Azure and GCP applications seem to never get hit by this dangerous SSRF vulnerability. On the other hand, AWS applications continue to get hit over and over and over again. Does that tell you something?
Yes, it tells us that an inside engineer working for Amazon any time after 2012 would have seen a lot of vulnerabilities sitting ripe for the picking, unlike engineers at other cloud providers. Boasting about theft is what got the former engineer caught, which should make us wonder what about the engineers who never boasted?
2) Trend Micro engineer 2019
In early August 2019, Trend Micro became aware that some of our consumer customers running our home security solution had been receiving scam calls by criminals impersonating Trend Micro support personnel. The information that the criminals reportedly possessed in these scam calls led us to suspect a coordinated attack. Although we immediately launched a thorough investigation, it was not until the end of October 2019 that we were able to definitively conclude that it was an insider threat. A Trend Micro employee used fraudulent means to gain access to a customer support database that contained names, email addresses, Trend Micro support ticket numbers, and in some instances telephone numbers.
3) Google engineer in 2010
Google Site Reliability Engineer (SRE) David Barksdale was recently fired for stalking and spying on teenagers through various Google services…
And in case you were wondering how easily a Google SRE could get away with this kind of obvious bad stalking behavior over the last ten years, in 2018 the company said it was starting to fire people in management for crimes and nearly 50 were out the door:
…an increasingly hard line on inappropriate conduct by people in positions of authority: in the last two years, 48 people have been terminated for sexual harassment, including 13 who were senior managers and above.
Do you realize how huge the investigations team has to be to collect evidence let alone fire 48 people over two years for sexual harassment?
4) Uber employees in 2016
Uber’s lack of security regarding its customer data was resulting in Uber employees being able to track high profile politicians, celebrities, and even personal acquaintances of Uber employees, including ex-boyfriends/girlfriends, and ex-spouses…
5) Twitter engineers 2019
Ahmad Abouammo and Ali Alzabarah each worked for the company from 2013 to 2015. The complaint alleges that Alzabarah, a site reliability engineer, improperly accessed the data of more than 6,000 Twitter users. […] Even after leaving the company, Abouammo allegedly contacted friends at Twitter to facilitate Saudi government requests, such as for account verification and to shutter accounts that had violated the terms of service.
6) Yahoo engineer 2018
In pleading guilty, Ruiz, a former Yahoo software engineer, admitted to using his access through his work at the company to hack into about 6,000 Yahoo accounts.
Just for the record, Ruiz was hired into Okta (cloud identity provider) as their SRE for eight months before be pleaded guilty to massive identity-theft crimes at his former employer.
7) Facebook engineer 2018
Facebook fires engineer who allegedly used access to stalk women. The employee allegedly boasted he was a ‘professional stalker.’
9) Lyft employees 2018
…employees were able to use Lyft’s back-end software to “see pretty much everything including feedback, and yes, pick up and drop off coordinates.” Another anonymous employee posted on the workplace app Blind that access to clients’ private information was abused. While staffers warned one another that the data insights tool tracks all usage, there seemed to be little to no enforcement, giving employees free reign over it. They used that access to spy on exes, spouses and fellow Lyft passengers they found attractive. They even looked up celebrity phone numbers, with one employee boasting that he had Zuckerberg’s digits. One source admitted to looking up their significant other’s Lyft destinations: “It was addictive. People were definitely doing what I was.”
9) AirBnB employees 2015
Airbnb actually teaches classes in SQL to employees so everyone can learn to query the data warehouses it maintains, and it has also created a tool called Airpal to make it easier to design SQL queries and dispatch them to the Presto layer of the data warehouse. (This tool has also been open sourced.) Airpal was launched internally at Airbnb in the spring of 2014, and within the first year, over a third of all employees at the company had launched an SQL query against the data warehouse.
10) And last but definitely not least, in a story rarely told accurately, City of San Francisco engineer 2008
Childs made national headlines by refusing to hand over administrative control to the City of San Francisco’s FiberWAN network, [a cloud service] which he had spent years helping to create.
One of the most peculiar aspects of the San Francisco case was how the network was sending logs (and perhaps even more, given a tap on core infrastructure) to a cluster of encrypted linux servers, and only Terry Childs had keys. He installed those servers in a metal cabinet with wood reinforcements and padlocks outside the datacenter and behind his desk. Holes were drilled in the cabinet for cable runs back into the datacenter, where holes also were drilled to allow entry.
The details of his case are so bizarre I’m surprised nobody made a movie about it. Maybe if one had been made, we’d have garnered more attention about these insider risks to cloud than the book we published in 2012.
So who would be the equivalent today of Tony Curtis?
Updated December 2019 with yet another Facebook example
11) Journalists had to notify the massively wealthy company that insiders were taking bribes to easily circumvent security controls to intentionally harm users:
A Facebook employee was paid thousands of dollars in bribes by a shady affiliate marketer to reactivate ad accounts that had been banned due to policy violations, a BuzzFeed News investigation has found. A company spokesperson confirmed that an unnamed employee was fired after inquiries from BuzzFeed News sparked an internal investigation.