Skip to content

#Hotmailgate: Where Don’t You Want to Go Today?

I thought with all the opinions flowing about the Hotmail privacy incident I would throw my hat into the ring. Perhaps most notably Bruce Schneier has done an excellent job warning people not to believe everything Google or Facebook is saying about privacy.

Before I get to Bruce’s article (below) I’d like to give a quick summary of the details I found interesting when reading about the Microsoft Hotmail privacy incident.

How it Begins

We know, for example, that the story begins with a Microsoft employee named Alex Kibkalo who was given a less-than-stellar performance review. This employee, a Russian native who worked from both Russia and Lebanon, reacted unfavorably and stole Microsoft pre-release updates for Windows RT and an Activation Server SDK.

Russia? Lebanon?

Perhaps it is fair to say the software extraction was retaliatory, as the FBI claims, but that also seems to be speculation. He may have had other motives. Some could suggest Alex’s Russian/Lebanese associations could have some geopolitical significance as well, for example. I so far have not seen anyone even mention this angle to the story but it seems reasonable to consider. It also raises the thorny question of how rights differ by location and nationality, especially in terms of monitoring and privacy.

Microsoft Resources Involved

More to the point of this post, from Lebanon Alex was able to quickly pull the software he wanted off Microsoft servers to a Virtual Machine (VM) running in a US Microsoft facility. Apparently downloading software all the way to Lebanon would have taken too long so he remotely controlled a VM and leveraged high speeds and close proximity of systems within US Microsoft facilities.

Alex then moved the stolen software from the Microsoft internal VM to the Microsoft public Skydrive cloud-based file-sharing service. With the stolen goods now in a place easily accessible to anyone, he emailed a French blogger.

The blogger was advised to have a technical person use the stolen software to build a service that would allow users to bypass Microsoft’s official software activation. The blogger publicly advertised on eBay the activation keys for sale and sent an email, from a Hotmail account, to a technical person for assistance with the stolen software. This technical person instead contact Microsoft.


To recap, an internal Microsoft employee used a Microsoft internal VM and a Microsoft public file-sharing cloud to steal Microsoft assets.

He either really liked using Microsoft or knew that they would not notice him stealing.

The intended recipient of those assets also used a Microsoft public cloud email account to communicate with employee stealing software, as well as with a person friendly to Microsoft senior executives.

When All You Have is a Hammer

Microsoft missed several red flags. Their internal virtual environment as well as their public cloud clearly was not detecting a theft in progress. A poor-performance review could be tied to sensitivity of network monitoring, watching for movement of large assets or pointing to communication with other internal staff that may have been working on behalf of the employee. Absent more advanced detective capabilities, let alone preventive ones, someone like Alex moves freely across Microsoft resources to steal assets.

A 900-lb gorilla approach to this problem must have seemed like a good idea to someone in Microsoft management. I have heard people suggest a rogue legal staff member was driving the decisions, yet this doesn’t sound plausible.

Having worked with gigantic legal entities in these scenarios I suspect coordinated and top-down the investigation and legal teams. Ironically, perhaps the most damaging steps to customer trust might have been done by a team called Trustworthy Computer Investigations (TWCI). They asked the Office of Legal Compliance (OLC) for authorization to compromise customer accounts. That to me indicates the opposite of any rogue effort; it was a management-led mission based on an internal code-of-conduct and procedures.

Hotmail Broken

The real controversy should be that the TWCI target was not internal. Instead of digging around Microsoft’s own security logs and controls, looking at traces of employee activity for what they needed, Microsoft compromised a public customer Hotmail account (as well as a physical home) with the assistance of law enforcement in several countries. They found traces they were looking for in the home and Hotmail account; steps that explained how their software was stolen by an internal employee as well as signs of intent.

The moral of the story, unfortunately, seems to be Microsoft internal security controls were not sufficient on their own, in speed or cost or something else, which compelled the company to protect themselves with a rather overt compromise of customer privacy and trust. This naturally has led to a public outcry about whether anyone can trust a public cloud, or even webmail.

Microsoft, of course, says this case is the exception. They say they had the right under their service terms to protect their IP. These are hard arguments to dispute, since an employee stealing Microsoft IP and using Microsoft services, and even trying to sell the IP by contacting someone friendly with Microsoft, can not possibly be a normal situation.

On the other hand, what evidence do we have now that Microsoft would restrict themselves from treating public as private?

With that in mind, Microsoft has shown their hand; they struggle to detect or prevent IP-theft as it happens, so they clearly aim to shoot after-the-fact and as necessary. There seems to be no pressure to do things by any standard of privacy (e.g. one defined by the nationality of the customer) other than one they cook up internally weighted by their own best interests.

Note the explanation by their Deputy Counsel:

Courts do not issue orders authorizing someone to search themselves, since obviously no such order is needed. So even when we believe we have probable cause, it’s not feasible to ask a court to order us to search ourselves.

They appear to be defining customers as indistinguishable from Microsoft employees. If you are a Hotmail user, you are now a part of Microsoft’s corporate “body”. Before you send HR an email asking for healthcare coverage, however, note that they also distinguish Microsoft personal email from corporate email.

The only exception to these steps will be for internal investigations of Microsoft employees who we find in the course of a company investigation are using their personal accounts for Microsoft business.

So if I understand correctly Microsoft employees are allowed an illusion of distinguishing personal email on Hotmail from their business email, which doesn’t make any sense really because even public accounts on Hotmail are treated like part of corporate body. And there’s no protection from searches anywhere anyway. When Microsoft internal staff, and an external attorney they have hired, believe there is probable cause then they can search “themselves”.

And for good measure, I found a new Google statement that says essentially the same thing. They reserve the right to snoop public customer accounts, even journalists.

“[TechCrunch editor Michael Arrington] makes a serious allegation here — that Google opened email messages in his Gmail account to investigate a leak,” Kent Walker, Google general counsel, said in a statement. “While our terms of service might legally permit such access, we have never done this and it’s hard for me to imagine circumstances where we would investigate a leak in that way.”

Hard perhaps for Kent to imagine, but with nothing stopping them…is imagination really even relevant?

Back to Schneier

Given this story as background, I’d like to respond to Bruce Schneier’s excellent article with the long title: “Don’t Listen to Google and Facebook: The Public-Private Surveillance Partnership Is Still Going Strong

These companies are doing their best to convince users that their data is secure. But they’re relying on their users not understanding what real security looks like.

This I have to agree with. Reading the Microsoft story I first was shocked to hear they had cracked their own customer’s email account. Then after I read the details I realized they had probable cause and they followed procedures…until I reached the point where I realized there was nothing being said about real security. It begs a simple question:

Should the lack of Microsoft ability to detect or prevent a theft, utilizing their private and public services, a reasonable justification for very broad holes in customer terms-of-service?

Something Just Hit the Fan

Imagine you are sitting on a toilet in your apartment. That apartment was much more convenient to move into compared to building your own house. But then, suddenly the owner is standing over you. The owner says since they can’t tell when widgets are taken from their offices (e.g they can’t detect which of their employees might be stealing) and they have probable cause (e.g. someone says you were seen with a missing widget) they can enter your bathroom at any time to check.

Were you expecting privacy while you sat on your toilet in your apartment?

Microsoft clearly disagrees and says there’s no need to even knock since they’re entering their own bathroom…in fact, all the bathrooms are theirs and no-one should be able to lock them out. Enjoy your apartment stay.

Surveillance, Not Surveillance

Real security looks like the owners detecting theft or preventing theft in “their” space rather than popping “your” door open whenever they feel like it. I hate to say it this way but it’s a political problem, rather than a technical one: what guide should we use to do surveillance in places that are socially agreed-upon, such as watching a shared office to reduce risks of theft, rather than threaten surveillance in places people traditionally and reasonably expect privacy?

So here is where I disagree with Schneier

Google, and by extension, the U.S. government, still has access to your communications on Google’s servers. Google could change that. It could encrypt your e-mail so only you could decrypt and read it. It could provide for secure voice and video so no one outside the conversations could eavesdrop. It doesn’t. And neither does Microsoft, Facebook, Yahoo, Apple, or any of the others. Why not? They don’t partly because they want to keep the ability to eavesdrop on your conversations.

Ok, I actually sort of agree with that. Google could provide you with the ability to lock them out, prevent them from seeing your data. But saying they want to eavesdrop on your conversations is where I start to think differently from Bruce. They want to offer tailored services, marketing if you allow it. The issue is whether we must define an observation space for these tailored services as completely and always open (e.g. Microsoft’s crazy definition of everything as “self”) or whether there is room for privacy.

Give Me Private Cloud or Give Me Encryption…OK I’ll Take Both

Suddenly, and unexpectedly, I am seeing movement towards cloud encryption using private-keys unknown to the provider. Bruce says this is impossible because “the US government won’t permit it”. I disagree. For years I worked with product companies to create this capability and was often denied. But it was not based on some insidious back-door or government worry. Product managers had many reasons why they hated to allow encryption into the road-map and the most common was there simply was not enough demand from customers.

Ironically, the rise of isolated but vociferous demand actually could be the reason we now will see it happen. If Google and Apple move towards a private-key solution, even if only to fly the “we’re better than Microsoft flag,” only a fraction of users will adopt (there’s an unknown usability/cost factor here). And of those users that do adopt eagerly, what is the percentage that the government comes knocking for with a warrant or a subpoena to decrypt? Probably a high percentage, yet still a small population. Given that the cloud providers properly setup key management they should be able to tell the government they have no way to decrypt or access the data.

Economics to the Rescue

This means from a business view the cloud provider could improve their offering to customers by enhancing trust with privacy controls, while at the same time reducing a cost burden of dealing with government requests for data. It could be a small enough portion of the users it wouldn’t impact services offered to the majority of users. This balance also could be “nudged” using cost; those wanting enhanced privacy pay a premium. In the end, there would be no way a provider could turn over a key that was completely unknown to them. And if Bruce is right that the government gets in no matter what, then all the more reason for cloud providers to raise the bar above their own capabilities.

We should have been headed this way a long time ago but, as I’ve said, the product managers really did not believe us security folks when we begged, pleaded and even demanded privacy controls. Usability, performance and a list a mile long of priorities always came first. Things have changed dramatically in the past year and #Hotmailgate really shows us where we don’t want to go. I suspect Microsoft and its competitors are now contemplating whether and how to incorporate real private-key systems to establish better public cloud privacy options, given the new economic models and customer demands developing.

Posted in Security.

0 Responses

Stay in touch with the conversation, subscribe to the RSS feed for comments on this post.

Some HTML is OK

or, reply to this post via trackback.