Polish Mathematicians Broke Nazi Enigma

Sadly this topic has remained a simmering controversy for far too long, mostly because of lack of effort on all our part. It isn’t hard to get it right, yet for some reason Poland isn’t getting credit due. The BBC in 2014 described a hugely important and historic event as simply a “quiet gathering”.

The debt owed by British wartime codebreakers to their Polish colleagues was acknowledged this week at a quiet gathering of spy chiefs. […] On the outskirts of Warsaw, some of the most senior spy bosses from Poland, France and Britain gathered this week in a nondescript but well-guarded building used by the Polish secret services. Their coming together was a way of marking the anniversary of a moment three-quarters of a century earlier when their predecessors held a meeting in Warsaw that played a crucial role in the victory over Hitler in World War Two.

I feel guilty. What have I done, as a historian of sorts, to help elevate this from quiet obscure ceremony to normalcy?

Mostly, for at least five years, I have bored friends with stories and tweeted about Poland’s contributions, which doesn’t feel like enough. So here’s my blog post to move the ball forward.

This is inspired by a new story in The Telegraph that the Polish government says more needs to be done.

Polish codebreakers ‘cracked Enigma before Alan Turing’ Diplomats say Poland’s key part in the deciphering the German system of codes in WWII has largely been overlooked

Time to stop overlooking. Let’s do this. Say it loud and proud, Poland broke the Nazi Enigma.

The Telegraph in 2012 versus 2016

News from The Telegraph in 2012 was: “Honour for overlooked Poles who were first to crack Enigma code”

…decades after Nazi Germany’s Enigma code was cracked, Poland has gone on the offensive to reclaim the glory of a cryptological success it feels has been unjustly claimed by Britain.

Frustrated at watching the achievements of the British wartime code-breakers at Bletchley Park lauded while those of Poles go overlooked, Poland’s parliament has launched a campaign to “restore justice” to the Polish men and women who first broke the Enigma codes.

[…]

The 2001 film Enigma, in particular, ruffled Polish feathers. The British production starring Kate Winslet and set in Bletchley Park made little mention of the Polish contribution to cracking the codes, and rubbed salt into the wounds by depicting the only Pole in the film as a traitor.

Some really good background in this 2012 article in The Telegraph. It is well written and accurate. Curious then how different it is from the story told to us in 2016.

Instead of pulling forward the earlier work, The Telegraph wrote a whole new version in 2016 filled with poorly researched ideas, pointing more towards the recent Turing movie, “The Imitation Game”.

Here are some questionable statements that jumped out at me.

Telegraph 2016: Poland Passed the Baton

…few people realise that early Enigma codes had already been broken by the Poles who then passed on the knowledge to Britain shortly before the outbreak of war.

It was not so simple. The Poles did not just pass along knowledge “shortly before” war. More to the point, given the escalation path of 1938, why was Britain waiting to the last moment before fall of Poland and declaration of war on Germany to receive crucial intelligence on German Enigma? Why were Brits far more focused on the Soviets as a threat instead of Germany, and why so interested in Spanish and Italian Enigmas instead of German?

Perhaps another way of asking this is what did the 1938 Munich Agreement, British appeasement of Nazi Germany, tell the Poles about trust in potential allies and giving away secrets?

Codebreakers from Britain early in 1939 had a kind of stalemate with Poland via talks setup by France. The three sides weren’t aligned exactly. Simply put it was British arrogance that led them to believe that their ability to break Enigma was best. When they met with the Polish the first time the British left thinking there was nothing they could gain.

Once war with Germany seemed unavoidable by summer of 1939, Poland simply ran out of time waiting for better terms of collaboration or warmer relations with British intelligence. Just before Germany rolled over Poland, codebreaking basically shifted to France, where negotiations continued with real alignment on German Enigma as the most pressing concern.

Months were basically wasted before the British were caught out as laggards and had to realize they had mistaken French and Polish cautions about Germany for incompetence. England realized their error fortunately before it was too late and rushed to learn from Poland, as war with Germany was announced.

Telegraph 2016: Poles Needed Help

By the time war broke out the Germans had increased the sophistication of the machine and the Poles were struggling to make more headway.

I hate the way this sounds. Hope it goes without saying Poles were struggling because…betrayal by Soviet defenses and invasion by Nazis while the world stood by and didn’t help. A highly secretive code-breaking team wasn’t going to just carry on effortlessly while their entire country was carved up and dismantled.

Sure the Germans had made a change, but that wasn’t the first time they altered Enigma (see Rejewski’s leading work on the Enigma Eintrittwalze – “entry wheel” – before the British figured it out, or transfer of Zygalski sheets to Bletchley, where they were known as Netz, short for Netz verfahren – “lattice method”). Difference by the time war broke out? The Polish had to destroy all their secret decoding systems and escape to France. I’ve read at first they tried to go to Britain and were denied due to confusion and secrecy (British embassy could not verify their roles). I’ve also read they went straight to France, where politics prevented them from moving to Bletchley. The bottom line is from the end of 1939 through early 1940 Turing and other Brits visited and studied Polish methods, learning of plans for new machines and preparing to build up operations in Bletchley Park.

“Struggling to make headway” is not a fair characterization relative to the many earlier mathematical struggles, which Poles obviously overcame on their own. The Poles had reconstructed Enigma and solved for daily keys. What made it hard to continue making headway? Staying under difficult conditions in Vichy France.

One of the original three who cracked Enigma, Rozycki, was killed in 1942 (lost at sea). The remaining Poles tried to escape to Spain that year. Langer, Ciezki and Palluth were captured by Germans. Rejewski and Zygalski escaped and landed in a Spanish prison. Only in 1943 these two finally enter England, where they were pushed aside into the Polish army in exile.

Struggling to make headway shouldn’t be blithely blamed on sophistication of the Enigma. Poles already had made plans to step up their game, which were handed over to England, as they tried to fight in Vichy France and stay alive.

Telegraph 2016: Blame Hollywood

…despite their help, history and Hollywood has largely ignored their role. The most recent film The Imitation Game, starring Benedict Cumberbatch, barely mentioned the Poles.

That’s right. And it’s a damn shame. Given that The Telegraph wrote in 2012 that a 2001 movie gave an unfair portrayal of the Poles, how did Imitation Game repeat the error? I found the movie highly disappointing.

Even more to the point there was in 2001 a book called “Stealing Secrets” that should have given Imitation Game producers all the background they needed on the true Turing story. Stealing Secrets doesn’t mince words here:

With the tide of the war having changed for the better, Bletchley’s leaders must have concluded in the cold calculus of realpolitik that is no longer had anything to gain from the Poles. […] Even now that the facts of the Poles’ Enigma breakthrough are out in the open, they must still compete in the marketplace of knowledge with earlier fictions. […] For a decade before the truth emerged about the Polish achievement, however, most of the English-speaking public was fed a steady diet of fiction masquerading as fact. […] Therefore, anyone who believes that Bletchley Park paved the road to victory in World War II must give credit to Poland for designing the road and mixing the pavement.

“Must give credit to Poland” as sage advice in 2001 and yet Imitation Game does none of that.

While visiting Bletchley Park I talked with the keepers about how Turing was portrayed relative to the Poles. They told me the film was rubbish and unfair. Their frankness surprised me and I found it refreshing. They basically had nothing good to say about the movie’s portrayal of events.

Telegraph 2016: Blame the Soviets

“We were trapped on the wrong side of the Iron Curtain during the Cold War which meant we did not get the credit that we should have received and nobody wanted to admit that anyone in Eastern Europe had anything to do with Enigma.

The Americans and English weren’t trapped by Soviets yet they too chose not to give credit. Does the world really need the Poles to repeatedly convince us of these facts as if the West doesn’t get it? And were the Poles blocked by Soviets? Sort of.

First, put this in terms of the 1940 Katyn massacre.

The Soviets in 1940 rounded up and assassinated 22,000 Polish military and intellectual elites (doctors, lawyers, professors), taking them into the woods and shooting them all in the back of the head. This massacre aimed to destroy any Polish resistance to Soviet control. America learned these details in 1943 from American POW forced by Germans to look at mass graves left behind by Soviets. Instead of bringing the news to light, the US kept it all a secret under the pretense of avoiding friction with Stalin.

That context makes it highly plausible the West was not about to credit Polish intellectuals for breaking Enigma when Stalin was around. But here’s the problem, nobody before the 1970s (20 years after Stalin) got public credit for cracking Enigma. There was literally no risk.

Second, put this in terms of the 1980 Solidarność.

Being on the wrong side of the Iron Curtain at that time is more relevant to our topic because that’s when Bletchley Park started leaking the stories. Now we’re talking about a prime time for strong characters and thaw stories, a time of Polish greatness and the Solidarity movement.

Remember the hardships the Polish cryptographers faced in 1940s France? None of them, even during German capture, leaked details of their work to anyone. Secrecy was crucial to success even after the end of the war. It was a top secret operation that only started to be verified more than 20 years after Stalin was out of the picture.

So really it isn’t about the Iron Curtain. It is about lazy historians in the West not doing a proper job with the facts. Blame is global and can’t be put on the Soviets repressing Poland’s voice, especially since we’re actually talking about the 1990s when these secret stories reached public sources; started to appeal to wider audiences. Still, Poland has to tell the world again and again until we accept it.

Telegraph 2016: Enigma is From End of WWI

The Enigma machine was invented by German engineer Arthur Sherbius at the end of the First World Wat [sic] and were used by the military and government of several countries.

Sherbius was applying for a patent for the Enigma in February 1918. WWI ended in November. Given events between those months I wouldn’t say Enigma came at the end. To me that would imply December or the start of 1919. There may even be some significance in timing relative to 1917; that was the year American scientist Vernam was given a task to invent a communication channel the Germans could not break, as patented in 1918. So “developed during the war” would be most appropriate in my mind.

In terms of several countries use…in 1927 the British government gave Enigma plans to Foss and Knox, code breakers, for review. A book about Knox’s role in breaking Enigma explains how Foss reported in theory it “could be broken given certain conditions” knowing as little as fifteen letters to figure out the machine settings. This effort led to the British and French working together on deciphering Spanish (Civil War) and Italian (invasion of Ethiopia) military communications in 1936.

Dillwyn (Dilly) Knox was one of the [British] Room 40 codebreakers during World War I. Since 1925 he had been trying to break the Enigma machine and had his first success on 4 April 1937 when he broke Franco’s Enigma K during the Spanish Civil War. When Germany starts using the Steckered Enigma for communication between Germany and Spain in 1938, he mounts an attack on the military Enigma, but is not successful as he unable to work out the wiring of the entry disc (ETW).

Here’s the key issue (pun not intended). Britain was not as keen to monitor German Enigma traffic until long after the French and Polish had warned of its importance. France was able to extract German documentation and gave it to Poland, who then cracked even the most advanced Enigma by 1933. That should put in perspective Britain listening to “several countries” signals in 1936. That was the year Germany was pushing into Rhineland and getting no push-back from Britain.

Telegraph 2016: Poland Involvement Well Known in WWII

…Polish involvement was well known during World War Two but during the communist time it was not so convenient to admit that there had been so much cooperation between Britain and Poland. It was a very special and very secret alliance.

This just makes no sense to me. It was top secret work, as mentioned above. No one knew about involvement, except those working in secrecy who couldn’t tell anyone outside. The secrecy extended well into the 1970s. During the communist time is when the story was not actually known, rather than being a convenience issue.

Also, rather than “admit…so much cooperation” I’d call it acknowledge the lack of working relationship once the British realized the Polish were ahead and captured all their secrets, as forced by German invasion of France.

Revisting Bletchley Park

What really would be nice to see is Bletchley Park incorporate French and Polish exhibits, perhaps even curated by representatives from those countries, to give factual explanations of their roles. After all it is meant to be a place to read about the “allied” effort. The Park could benefit from the help for upkeep and maintaining records. Meanwhile, visitors would get a more robust and fair portrayal of a “world” war.

At some point maybe I’ll post my photos here from my trip there, which show some of the odd statements made by British historians, minimizing the efforts of the Polish.

Reasons Against Remembering

Some want to erase history to make others look good; ignoring the Polish role as Allies lets the British or Americans stand out more.

Some want to erase history to make themselves look less bad; ignoring Polish role as Axis lets the Germans stand out more.

Either way overlooking real Polish history is bad for WWII history as well as our understanding of security. Bringing facts forward today should have no risk.

If we give credit to Polish code-breakers we are not diminishing the still monumental contributions of Alan Turing during WWII. We can be more correct in the presentation of historic facts without much impact or edits to Bletchley Park.

When we give credit to those in Poland who fought against Nazis and did so much right, it does not mean we can forget wrongs done by others, such as Erich von Zelewski the Polish Nazi who proposed creation of Auschwitz (just one out more than 10,000 prisoner camps under Nazi control, let alone nearly 1,000 forced labor camps for Jews inside Poland). By 1946 Nuremburg trials this Polish Nazi testified while he had no issue with Jews sent to die in camps he had “tried to prevent the destruction of Warsaw” and his work “saved hundreds of thousands of civilians and tens of thousands of soldiers of Polish nationality”.

As more sunlight comes for the Poles who fought against Nazis, it may clear the air for us to also discuss and better understand their opposite, the Poles who collaborated. So far we have the book “Hunt for the Jews: Betrayal and Murder in German-Occupied Poland“, which discusses “how the Germans were able to mobilize segments of the Polish society to take part in their plan to hunt down the Jews”. And we have dramatization films like Ida and Poklosie (Aftermath)

The 1946 Kielce Pogrom provides a sad study of how some Poles continued to kill even after the war had ended to try and finish what Germans could not – elimination of Jews from Poland. With that in mind please note a bill has been introduced in Poland making it illegal to mention any Nazi collusion. Such a bill of denial would be a tragedy for those of us who try to bring out examples of bad as well as good and learn from the past.

Right now we should remember a Polish team of mathematicians working with human intelligence for what they were: the first to crack the Nazi Enigma.

As I said at the start, this is no quiet affair. Time to stop overlooking. Let’s do this. Say it loud and proud, Poland broke the Nazi Enigma.

And for those looking, there’s a physical commemoration to Polish codebreakers at Bletchley Park, built in 2002 on the ground in the far back area by residences, behind a brick wall.

“Commemorative Plaque” for Polish cryptanalysts, first to break the German Enigma. Source: Bletchley Park

Where is the Revolution in Intelligence? Public, Private or Shared?

Watching Richard Bejtlich’s recent “Revolution in Intelligence” talk about his government training and the ease of attribution is very enjoyable, although at times for me it brought to mind CIA factbook errors in the early 1990s.

Slides that go along with the video are available on Google drive

Let me say, to get this post off the ground, I will be the first one to stand up and defend US government officials as competent and highly skilled professionals. Yet I also will call out an error when I see one. This post is essentially that. Bejtlich is great, yet he often makes some silly errors.

Often I see people characterize a government as made up of inefficient troglodytes falling behind. That’s annoying. Meanwhile often I also see people lionize nation-state capabilities as superior to any other organization. Also annoying. The truth is somewhere in between. Sometimes the government does great work, sometimes it blows compared to private sector.

Take the CIA factbook I mentioned above as an example. It has been unclassified since the 1970s and by the early 1990s it was published on the web. Given wider distribution its “facts” came under closer scrutiny from academics. So non-gov people who long had studied places or lived in them (arguably the world’s true leading experts) read this fact book and wanted to help improve it — outsiders looking in and offering assistance. Perhaps some of you remember the “official” intelligence peddled by the US government at that time?

Bejtlich in his talk gives a nod towards academia being a thorough environment and even offers several criteria for why academic work is superior to some other governments (not realizing he should include his own). Perhaps this is because he is now working on a PhD. I mean it is odd to me he fails to realize this academic community was just as prolific and useful in the 1990s, gathering intelligence and publishing it, giving talks and sending documents to those who were interested. His presentation makes it sound like before search engines appeared it required nation-state sized military departments walking uphill both ways in a blizzard to gather data.

Aside from having this giant blind spot to what he calls the “outsider” community, I also fear I am listening to someone with no field experience gathering intelligence. Sure image analysis is a skill. Sure we can sit in a room and pore over every detail to build up a report on some faraway land. On one of my private sector security teams I had a former US Air Force technician who developed film from surveillance planes. He hated interacting with people, loved being in the darkroom. But what does Bejtlich think of actually walking into an environment as an equal, being on the ground, living among people, as a measure of “insider” intelligence skill?

Almost three decades ago I stepped off a plane into a crowd of unfamiliar faces in a small country in Asia. Over the next five weeks I embedded myself into mountain villages, lived with families on the great plains, wandered with groups through jungles and gathered as much information as I could on the decline of monarchial rule in the face of democratic pressure.

One sunny day on the side of a shoulder-mountain stands out in my memory. As I hiked down a dusty trail a teenage boy dressed all in black walked towards me. He carried a small book under his arm. He didn’t speak English. We communicated in broken phrases and hand gestures. He said he was a member of a new party.

Mao was his leader, he said. The poor villages felt they weren’t treated well, decided to do something about it. I asked about Lenin. The boy had never heard the name. Stalin? Again the boy didn’t know. Mao was the inspiration for his life and he was pleased about this future for his village.

This was before the 1990s. And by most “official” accounts there were no studies or theories about Maoists in this region until at least ten years later. I mention this here not because individual people with a little fieldwork can make a discovery. It should be obvious military schools don’t have a monopoly on intel. The question is what happened to that data. Where did information go and who asked about it? Did others have easy access to data gathered?

Yes, someone from private sector should talk about “The Revolution in Private Sector Intelligence”. Perhaps we can find someone with experience working on intelligence in the private sector for many, many years, to tell us what has changed for them. Maybe there will be stories of pre-ChoicePoint private sector missions to fly in on a moment’s notice into random places to gather intelligence on employees who were stealing money and IP. And maybe non-military experience will unravel why Russian operations in private sector had to be handled uniquely from other countries?

Going by Bejtlich’s talk it would seem that such information gathering simply didn’t exist if the US government wasn’t the one doing it. What I hear from his perspective is you go to a military school that teaches you how to do intelligence. And then you graduate and then you work in a military office. Then you leave that office to teach outsiders because they can learn too.

He sounds genuinely incredulous to discover that someone in the private sector is trainspotting. If you are familiar with the term you know many people enjoy as a hobby building highly detailed and very accurate logs of transportation. Bejtlich apparently is unaware, despite this being a well-known thing for a very long time.

A new record of trainspotting has been discovered from 1861, 80 years earlier than the hobby was first thought to have begun. The National Railway Museum found a reference to a 14 year old girl writing down the numbers of engines heading in and out of Paddington Station.

It reminds me a bit of how things must have moved away from military intelligence for the London School of Oriental and African Studies (now just called SOAS). The British cleverly setup in London a unique training school during the first World War, as explained in the 1917 publication “Nature”:

…war has opened our eyes to the necessity of making an effort to compete vigorously with the activities — political, commercial, and even scientific and linguistic — of the Germans in Asia and Africa. We have discovered that their industry was rarely disinterested, and that political propaganda was too often at the root of “peaceful penetration” in the field of missionary, scientific, and linguistic effort.

In other words, a counter-intelligence school was born. Here the empire could maintain its military grip around the world by developing the skills to better gather intelligence and understand enemy culture (German then, but ultimately native).

By the 1970s SOAS, a function of the rapidly changing British global position, seemed to take on wider purpose. It reached out and looked at new definitions of who might benefit from the study and art of intelligence gathering. By 1992 regulars like you or me could attend and sit within the shell of the former hulk of a global analysis engine. Academics there focused on intelligence gathering related to revolution and independence (e.g. how to maintain profits in trade without being a colonial power).

I was asked by one professor to consider staying on for a PhD to help peel apart Ghana’s 1956 transition away from colonial rule, for only academic purpose of course. Tempted as I was, LSE instead set the next chapters of my study, which itself seems to have become known sometime during the second World War as a public/private shared intelligence analyst training school (Bletchley Park staff tried to convince me Zygalski, inventor of equipment to break the Enigma, lectured at LSE although I could find no records to support that claim).

Fast forward five years to 1997 and the Corner House is a good example of academics in London who formalized public intelligence reports (starting in 1993?) into a commercial portfolio. In their case an “enemy” was more along the lines of companies or even countries harming the environment. This example might seem a bit tangential until you ask someone for expert insights, including field experience, to better understand the infamous pipeline caught in a cyberwar.

Anyway, without me droning on and on about the richness in an “outside” world, Bejtlich does a fine job describing some of the issues he had adjusting. He just seems to have been blind to communities outside his own and is pleased to now be discovering them. His “inside” perspective on intelligence is really just his view of inside/outside, rather than any absolute one. Despite pointing out how highly he regards academics who source material widely he then unfortunately doesn’t follow his own advice. His talk would have been so much better with a wee bit more depth of field and some history.

Let me drag into this an interesting example that may help make my point, that private analysts not only can be as good or better than government they may even be just as secretive and political.

Eastman Kodak investigated, and found something mighty peculiar: the corn husks from Indiana they were using as packing materials were contaminated with the radioactive isotope iodine-131 (I-131). Eastman Kodak at the time had some of the best researchers in the country on its team (the company even had its own nuclear reactor in the 1970s), and they discovered something that was not public knowledge: those farms in Indiana had been exposed to fallout from the 1945 Trinity Test in New Mexico — the world’s first atmospheric nuclear bomb explosions which ushered in the atomic age. Kodak kept this exposure silent.

The American film industry giant by 1946 realized, from clever digging into the corn husk material used for packaging, that the US government was poisoning its citizens. The company filed a formal complaint and kept quiet. Our government responded by warning Kodak of military research to help them understand how to hide from the public any signs of dangerous nuclear fallout.

Good work by the private sector helping the government more secretly screw the American public without detection, if you see what I mean.

My point is we do not need to say the government gives us the best capability for world-class intelligence skills. Putting pride aside there may be a wider world of training. So we also should not say private-sector makes someone the best in world at uncovering the many and ongoing flaws in government intelligence. Top skills can be achieved in different schools of thought, which serve different purposes. Kodak clearly worried about assets differently than the US government, while they still kind of ended up worrying about the same thing (colluding, if you will). Hard to say who evolved faster.

By the way, speaking of relativity, also I find it amusing Bejtlich’s talk is laced with his political preferences as landmines: Hillary Clinton is setup as so obviously guilty of dumb errors you’d be a fool not to convict her. President Obama is portrayed as maliciously sweeping present and clear danger of terrorism under the carpet, putting us all in grave danger.

And last but not least we’re led to believe if we get a scary black bag indicator we should suspect someone who had something to do with Krav Maga (historians might say an Austro-Hungarian or at least Slovakian man, but I’m sure we are supposed to think Israeli). Is that kind of like saying someone who had something to do with Karate (Bruce Lee!) when hinting at America?

And one last thought. Bejtlich also mentions gathering intelligence on soldiers in the Civil War as if it would be like waiting for letters in the mail. In fact there were many more routes of “real time” information. Soldiers were skilled at sneaking behind lines (pun not intended) tapping copper wires and listening, then riding back with updates. Poetry was a common method of passing time before a battle by creating clever turns of phrase about current events, perhaps a bit like twitter functions today. “Deserters” were a frequent source of updates as well, carrying news across lines.

I get what Bejtlich is trying to say about speed of information today being faster and have to technically agree with that one aspect of a revolution; of course he’s right about raw speed of a photo being posted to the Internet and seen by an analyst. Yet we shouldn’t under-sell what constituted “real-time” 150 years ago, especially if we think about those first trainspotters…

Hillary, Official Data Classification, and Personal Servers

The debate over Hillary Clinton’s use of email reminds me of a Goldilocks’ tech management dilemma. Users tend to think you are running too slow or too fast, never just right:

Too slow

You face user ire, potential revolt, as IT (let alone security) becomes seen as the obstacle to progress. Users want access to get their job done faster, better, etc. so they push data to cloud and apps, bring in their own devices and run like they have no fear because trust is shifted into clever new service providers.

We all know that has been the dominant trend and anyone caught saying “blackberry is safer” is at risk of being kicked out of the cool technology clubs. Even more to the point you have many security thought leaders saying over and over to choose cloud and ipad because safer.

I mentioned this in a blog post in 2011 when the Apple iPad was magically “waived” through security assessments for USAID.

Today it seems ironic to look back at Hillary’s ire. We expect our progressive politicians to look for modernization opportunities and here is a perfect example:

Many U.S. Agency for International Development workers are using iPads–a fact that recently drew the ire of Secretary of State Hillary Clinton when she sat next to a USAID official on a plane, said Jerry Horton, chief information officer at USAID. Horton spoke April 7 at a cloud computing forum at the National Institute of Standards and Technology in Gaithersburg, Md.

Clinton wanted to know why a USAID official could have an iPad while State Department officials still can’t. The secret, apparently, lies in the extensive use of waivers. It’s “hard to dot all the Is and cross all the Ts,” Horton said, admitting that not all USAID networked devices are formally certified and accredited under Federal Information Security Management Act.

“We are not DHS. We are not DoD,” he said.

While the State Department requires high-risk cybersecurity, USAID’s requirements are much lower, said Horton. “And for what is high-security it better be on SIPR.”

Modernizing, innovating, asking for government to reform is a risky venture. At the time I don’t remember anyone saying Hillary was being too risky, or her ire was misplaced in asking for technology improvements. There was a distinct lack of critique heard, despite my blog post sitting in the top three search results on Google for weeks. If anything I heard the opposite, that the government should trust and catch up to Apple’s latest whatever.

Too fast

Now let’s look at the other perspective. Dump the old safe and trusted Blackberry so you can let users consume iPads like candy going out of style, and you face watching them stumble and fall on their diabetic face. Consumption of data is the goal and yet it also is the danger.

Without getting into too many of the weeds for the blame game, figuring out who is responsible for a disaster, it may be better to look at why there will be accidents/misunderstandings in a highly politicized environment.

What will help us make sure we avoid someone extracting data off SIPR/NIPR without realizing there is a “TS/SAP” classification incident ahead? I mean what if the majority of data in question pertain to a controversial program, let say for example drones in Pakistan, which may or may not be secret depending on one’s politics. Colin Powell gives us some insight to the problem:

…emails were discovered during a State Department review of the email practices of the past five secretaries of state. It found that Powell received two emails that were classified and that the “immediate staff” working for Rice received 10 emails that were classified.

The information was deemed either “secret” or “confidential,” according to the report, which was viewed by CNN.

In all the cases, however — as well as Clinton’s — the information was not marked “classified” at the time the emails were sent, according to State Department investigators.

Powell noted that point in a statement on Thursday.

“The State Department cannot now say they were classified then because they weren’t,” Powell said. “If the Department wishes to say a dozen years later they should have been classified that is an opinion of the Department that I do not share.”

“I have reviewed the messages and I do not see what makes them classified,” Powell said.

This classification game is at the heart of the issue. Reclassification happens. Aggregate classification of not secret data can make it secret. If we characterize it as a judgment flaw by only one person, or even three, we may be postponing the critical need to review where there are wider systemic issues in decision-making and tools.

To paraphrase the ever insightful Daniel Barth-Jones: smart people at the top of their political game who make mistakes aren’t “stupid”; we have to evaluate whether systems that don’t prevent mistakes by design are….

Just right

Assuming we agree want to go faster than “too slow”, and we do not to run ahead “too fast” into disasters…a middle ground needs to come into better focus.

Giving up “too slow” means a move away from blocking change. And I don’t mean achieving FISMA certification. That is seen as a tedious low bar for security rather than the right vehicle for helping push towards the top end. We need to take compliance seriously as a guide as we also embrace hypothesis, creative thinking, to tease out a reasonable compromise.

We’re still very early in the dinosaur days of classification technology, sitting all the way over by the slow end of the equation. I’ve researched solutions for years, seen some of the best engines in the world (Varonis, Olive), and it’s not yet looking great. We have many more tough problems to solve, leaving open a market ripe for innovation.

Note the disclaimer on Microsoft’s “Data Classification Toolkit

Use of the Microsoft Data Classification Toolkit does not constitute advice from an auditor, accountant, attorney or other compliance professional, and does not guarantee fulfillment of your organization’s legal or compliance obligations. Conformance with these obligations requires input and interpretation by your organization’s compliance professionals.

Let me explain the problem by way of analogy, to be brief.

Cutting-edge research on robots focuses on predictive capabilities to enable driving off-road free from human control. A robot starts with near-field sensors, which gets them about 20 feet of vision ahead to avoid immediate danger. Then the robot needs to see much further to avoid danger altogether.

This really is the future of risk classification. The better your classification of risks, the better your predictive plan, and the less you have to make time-pressured disaster avoidance decisions. And of course being driver-less is a relative term. These automation systems still need human input.

In a DARPA LAGR Program video the narrator puts it simply:

A short-sighted robot makes poor decisions

Imagine longer-range vision algorithms that generate an “optimal path”, applied to massive amounts of data (different classes of email messages instead of trees and rocks in the great outdoors), dictating what you actually get to see.

LAGR-view

What I like about this optimal path illustration is the perpendicular alignment of two types of vision. The visible world is flat. And then there is the greater, optimal path theory, presented as a wall-like circle, easily queried without actually being “seen”. This is like putting your faith in a map because you can’t actually see all the way from San Francisco to New York.

The difference between the short and long highlights why any future of safe autonomous systems will depend on processing power of the end nodes, such that they can both create a larger areas of more “flat” rings as well as build out the “taller” optimal paths.

Here is where “personal” servers come into play. Power becomes a determinant of vision and autonomy. Personal investments often can increase processing power faster than government bureaucracy and depreciation schedules. I mean if the back-end system looks at the ground ahead and classifies as sand (unsafe to proceed), and the autonomous device does its own assessment on its own servers and decides it is looking at asphalt (safe for speed), who is right?

The better the predictive algorithms the taller the walls of vision into the future, and that begs for power and performance enhancements. Back to the start of this post, when IT isn’t providing users the kind of power they want for speed, we see users move their workloads towards BYOD and cloud. Classification becomes a power struggle, as forward-looking decisions depend on reliable data classification from an authoritative source.

If authoritative back-end services accidentally classify data safe and later reverse to unsafe (or vice-versa) the nodes/people depending on a classification service should not be the only target in an investigation of judgement error.

We can joke about how proper analysis always would chose a “just right” Goldilocks long-term path, yet in reality the debate is about building a high-performance data classification system that reduces her cost of error.