Skip to content


Polish Mathematicians Broke Nazi Enigma

Sadly this topic has remained a simmering controversy for far too long, mostly because of lack of effort on all our part. It isn’t hard to get it right, yet for some reason Poland isn’t getting credit due. The BBC in 2014 described a hugely important and historic event as simply a “quiet gathering”.

The debt owed by British wartime codebreakers to their Polish colleagues was acknowledged this week at a quiet gathering of spy chiefs. […] On the outskirts of Warsaw, some of the most senior spy bosses from Poland, France and Britain gathered this week in a nondescript but well-guarded building used by the Polish secret services. Their coming together was a way of marking the anniversary of a moment three-quarters of a century earlier when their predecessors held a meeting in Warsaw that played a crucial role in the victory over Hitler in World War Two.

I feel guilty. What have I done, as a historian of sorts, to help elevate this from quiet obscure ceremony to normalcy?

Mostly, for at least five years, I have bored friends with stories and tweeted about Poland’s contributions, which doesn’t feel like enough. So here’s my blog post to move the ball forward.

This is inspired by a new story in The Telegraph that the Polish government says more needs to be done.

Polish codebreakers ‘cracked Enigma before Alan Turing’
Diplomats say Poland’s key part in the deciphering the German system of codes in WWII has largely been overlooked

Time to stop overlooking. Let’s do this. Say it loud and proud, Poland broke the Nazi Enigma.

The Telegraph in 2012 versus 2016

News from The Telegraph in 2012 was: “Honour for overlooked Poles who were first to crack Enigma code”

…decades after Nazi Germany’s Enigma code was cracked, Poland has gone on the offensive to reclaim the glory of a cryptological success it feels has been unjustly claimed by Britain.

Frustrated at watching the achievements of the British wartime code-breakers at Bletchley Park lauded while those of Poles go overlooked, Poland’s parliament has launched a campaign to “restore justice” to the Polish men and women who first broke the Enigma codes.

[…]

The 2001 film Enigma, in particular, ruffled Polish feathers. The British production starring Kate Winslet and set in Bletchley Park made little mention of the Polish contribution to cracking the codes, and rubbed salt into the wounds by depicting the only Pole in the film as a traitor.

Some really good background in this 2012 article in The Telegraph. It is well written and accurate. Curious then how different it is from the story told to us in 2016.

Instead of pulling forward the earlier work, The Telegraph wrote a whole new version in 2016 filled with poorly researched ideas, pointing more towards the recent Turing movie, “The Imitation Game”.

Here are some questionable statements that jumped out at me.

Telegraph 2016: Poland Passed the Baton

…few people realise that early Enigma codes had already been broken by the Poles who then passed on the knowledge to Britain shortly before the outbreak of war.

It was not so simple. The Poles did not just pass along knowledge “shortly before” war. More to the point, given the escalation path of 1938, why was Britain waiting to the last moment before fall of Poland and declaration of war on Germany to receive crucial intelligence on German Enigma? Why were Brits far more focused on the Soviets as a threat instead of Germany, and why so interested in Spanish and Italian Enigmas instead of German?

Perhaps another way of asking this is what did the 1938 Munich Agreement, British appeasement of Nazi Germany, tell the Poles about trust in potential allies and giving away secrets?

Codebreakers from Britain early in 1939 had a kind of stalemate with Poland via talks setup by France. The three sides weren’t aligned exactly. Simply put it was British arrogance that led them to believe that their ability to break Enigma was best. When they met with the Polish the first time the British left thinking there was nothing they could gain.

Once war with Germany seemed unavoidable by summer of 1939, Poland simply ran out of time waiting for better terms of collaboration or warmer relations with British intelligence. Just before Germany rolled over Poland, codebreaking basically shifted to France, where negotiations continued with real alignment on German Enigma as the most pressing concern.

Months were basically wasted before the British were caught out as laggards and had to realize they had mistaken French and Polish cautions about Germany for incompetence. England realized their error fortunately before it was too late and rushed to learn from Poland, as war with Germany was announced.

Telegraph 2016: Poles Needed Help

By the time war broke out the Germans had increased the sophistication of the machine and the Poles were struggling to make more headway.

I hate the way this sounds. Hope it goes without saying Poles were struggling because…betrayal by Soviet defenses and invasion by Nazis while the world stood by and didn’t help. A highly secretive code-breaking team wasn’t going to just carry on effortlessly while their entire country was carved up and dismantled.

Sure the Germans had made a change, but that wasn’t the first time they altered Enigma (see Rajewski’s leading work on the Enigma Eintrittwalze – “entry wheel” – before the British figured it out, or transfer of Zygalski sheets to Bletchley, where they were known as Netz, short for Netz verfahren – “lattice method”). Difference by the time war broke out? The Polish had to destroy all their secret decoding systems and escape to France. I’ve read at first they tried to go to Britain and were denied due to confusion and secrecy (British embassy could not verify their roles). I’ve also read they went straight to France, where politics prevented them from moving to Bletchley. The bottom line is from the end of 1939 through early 1940 Turing and other Brits visited and studied Polish methods, learning of plans for new machines and preparing to build up operations in Bletchley Park.

“Struggling to make headway” is not a fair characterization relative to the many earlier mathematical struggles, which Poles obviously overcame on their own. The Poles had reconstructed Enigma and solved for daily keys. What made it hard to continue making headway? Staying under difficult conditions in Vichy France.

One of the original three who cracked Enigma, Rozycki, was killed in 1942 (lost at sea). The remaining Poles tried to escape to Spain that year. Langer, Ciezki and Palluth were captured by Germans. Rajewski and Zygalski escaped and landed in a Spanish prison. Only in 1943 these two finally enter England, where they were pushed aside into the Polish army in exile.

Struggling to make headway shouldn’t be blithely blamed on sophistication of the Enigma. Poles already had made plans to step up their game, which were handed over to England, as they tried to fight in Vichy France and stay alive.

Telegraph 2016: Blame Hollywood

…despite their help, history and Hollywood has largely ignored their role. The most recent film The Imitation Game, starring Benedict Cumberbatch, barely mentioned the Poles.

That’s right. And it’s a damn shame. Given that The Telegraph wrote in 2012 that a 2001 movie gave an unfair portrayal of the Poles, how did Imitation Game repeat the error? I found the movie highly disappointing.

Even more to the point there was in 2001 a book called “Stealing Secrets” that should have given Imitation Game producers all the background they needed on the true Turing story. Stealing Secrets doesn’t mince words here:

With the tide of the war having changed for the better, Bletchley’s leaders must have concluded in the cold calculus of realpolitik that is no longer had anything to gain from the Poles. […] Even now that the facts of the Poles’ Enigma breakthrough are out in the open, they must still compete in the marketplace of knowledge with earlier fictions. […] For a decade before the truth emerged about the Polish achievement, however, most of the English-speaking public was fed a steady diet of fiction masquerading as fact. […] Therefore, anyone who believes that Bletchley Park paved the road to victory in World War II must give credit to Poland for designing the road and mixing the pavement.

“Must give credit to Poland” as sage advice in 2001 and yet Imitation Game does none of that.

While visiting Bletchley Park I talked with the keepers about how Turing was portrayed relative to the Poles. They told me the film was rubbish and unfair. Their frankness surprised me and I found it refreshing. They basically had nothing good to say about the movie’s portrayal of events.

Telegraph 2016: Blame the Soviets

“We were trapped on the wrong side of the Iron Curtain during the Cold War which meant we did not get the credit that we should have received and nobody wanted to admit that anyone in Eastern Europe had anything to do with Enigma.

The Americans and English weren’t trapped by Soviets yet they too chose not to give credit. Does the world really need the Poles to repeatedly convince us of these facts as if the West doesn’t get it? And were the Poles blocked by Soviets? Sort of.

First, put this in terms of the 1940 Katyn massacre.

The Soviets in 1940 rounded up and assassinated 22,000 Polish military and intellectual elites (doctors, lawyers, professors), taking them into the woods and shooting them all in the back of the head. This massacre aimed to destroy any Polish resistance to Soviet control. America learned these details in 1943 from American POW forced by Germans to look at mass graves left behind by Soviets. Instead of bringing the news to light, the US kept it all a secret under the pretense of avoiding friction with Stalin.

That context makes it highly plausible the West was not about to credit Polish intellectuals for breaking Enigma when Stalin was around. But here’s the problem, nobody before the 1970s (20 years after Stalin) got public credit for cracking Enigma. There was literally no risk.

Second, put this in terms of the 1980 Solidarność.

Being on the wrong side of the Iron Curtain at that time is more relevant to our topic because that’s when Bletchley Park started leaking the stories. Now we’re talking about a prime time for strong characters and thaw stories, a time of Polish greatness and the Solidarity movement.

Remember the hardships the Polish cryptographers faced in 1940s France? None of them, even during German capture, leaked details of their work to anyone. Secrecy was crucial to success even after the end of the war. It was a top secret operation that only started to be verified more than 20 years after Stalin was out of the picture.

So really it isn’t about the Iron Curtain. It is about lazy historians in the West not doing a proper job with the facts. Blame is global and can’t be put on the Soviets repressing Poland’s voice, especially since we’re actually talking about the 1990s when these secret stories reached public sources; started to appeal to wider audiences. Still, Poland has to tell the world again and again until we accept it.

Telegraph 2016: Enigma is From End of WWI

The Enigma machine was invented by German engineer Arthur Sherbius at the end of the First World Wat [sic] and were used by the military and government of several countries.

Sherbius was applying for a patent for the Enigma in February 1918. WWI ended in November. Given events between those months I wouldn’t say Enigma came at the end. To me that would imply December or the start of 1919. There may even be some significance in timing relative to 1917; that was the year American scientist Vernam was given a task to invent a communication channel the Germans could not break, as patented in 1918. So “developed during the war” would be most appropriate in my mind.

In terms of several countries use…in 1927 the British government gave Enigma plans to Foss and Knox, code breakers, for review. A book about Knox’s role in breaking Enigma explains how Foss reported in theory it “could be broken given certain conditions” knowing as little as fifteen letters to figure out the machine settings. This effort led to the British and French working together on deciphering Spanish (Civil War) and Italian (invasion of Ethiopia) military communications in 1936.

Here’s the key issue. Britain was not as keen to monitor German Enigma traffic, despite it being the most advanced, until long after the French and Polish had warned of its importance. France was able to extract German documentation and gave it to Poland, who then cracked this most advanced Enigma by 1933. That should put in perspective Britain listening to “several countries” signals in 1936. That was the year Germany was pushing into Rhineland and getting no push-back from Britain.

Telegraph 2016: Poland Involvement Well Known in WWII

…Polish involvement was well known during World War Two but during the communist time it was not so convenient to admit that there had been so much cooperation between Britain and Poland. It was a very special and very secret alliance.

This just makes no sense to me. It was top secret work, as mentioned above. No one knew about involvement, except those working in secrecy who couldn’t tell anyone outside. The secrecy extended well into the 1970s. During the communist time is when the story was not actually known, rather than being a convenience issue.

Also, rather than “admit…so much cooperation” I’d call it acknowledge the lack of working relationship once the British realized the Polish were ahead and captured all their secrets, as forced by German invasion of France.

Revisting Bletchley Park

What really would be nice to see is Bletchley Park incorporate French and Polish exhibits, perhaps even curated by representatives from those countries, to give factual explanations of their roles. After all it is meant to be a place to read about the “allied” effort. The Park could benefit from the help for upkeep and maintaining records. Meanwhile, visitors would get a more robust and fair portrayal of a “world” war.

At some point maybe I’ll post my photos here from my trip there, which show some of the odd statements made by British historians, minimizing the efforts of the Polish.

Reasons Against Remembering

Some want to erase history to make others look good; ignoring the Polish role as Allies lets the British or Americans stand out more.

Some want to erase history to make themselves look less bad; ignoring Polish role as Axis lets the Germans stand out more.

Either way overlooking real Polish history is bad for WWII history as well as our understanding of security. Bringing facts forward today should have no risk.

If we give credit to Polish code-breakers we are not diminishing the still monumental contributions of Alan Turing during WWII. We can be more correct in the presentation of historic facts without much impact or edits to Bletchley Park.

When we give credit to those in Poland who fought against Nazis and did so much right, it does not mean we can forget wrongs done by others, such as Erich von Zelewski the Polish Nazi who proposed creation of Auschwitz (just one out more than 10,000 prisoner camps under Nazi control, let alone nearly 1,000 forced labor camps for Jews inside Poland). By 1946 Nuremburg trials this Polish Nazi testified while he had no issue with Jews sent to die in camps he had “tried to prevent the destruction of Warsaw” and his work “saved hundreds of thousands of civilians and tens of thousands of soldiers of Polish nationality”.

As more sunlight comes for the Poles who fought against Nazis, it may clear the air for us to also discuss and better understand their opposite, the Poles who collaborated. So far we have the book “Hunt for the Jews: Betrayal and Murder in German-Occupied Poland“, which discusses “how the Germans were able to mobilize segments of the Polish society to take part in their plan to hunt down the Jews”. And we have dramatization films like Ida and Poklosie (Aftermath)

The 1946 Kielce Pogrom provides a sad study of how some Poles continued to kill even after the war had ended to try and finish what Germans could not – elimination of Jews from Poland. With that in mind please note a bill has been introduced in Poland making it illegal to mention any Nazi collusion. Such a bill of denial would be a tragedy for those of us who try to bring out examples of bad as well as good and learn from the past.

Right now we should remember a Polish team of mathematicians working with human intelligence for what they were: the first to crack the Nazi Enigma.

As I said at the start, this is no quiet affair. Time to stop overlooking. Let’s do this. Say it loud and proud, Poland broke the Nazi Enigma.

Posted in History, Security.


1868 Bugle Call Semiotics: Melody as Communication in War

Archive.org presents this very fine example of a system of communication during war. Signs, sounds and symbols that have semantics, syntactics, and pragmatics basically boil down to a bugle melody telling soldiers their actions.

1868-bugle-calls

Posted in History, Security.


Where is the Revolution in Intelligence? Public, Private or Shared?

Watching Richard Bejtlich’s recent “Revolution in Intelligence” talk about his government training and the ease of attribution is very enjoyable, although at times for me it brought to mind CIA factbook errors in the early 1990s.

Slides that go along with the video are available on Google drive

Let me say, to get this post off the ground, I will be the first one to stand up and defend US government officials as competent and highly skilled professionals. Yet I also will call out an error when I see one. This post is essentially that. Bejtlich is great, yet he often makes some silly errors.

Often I see people characterize a government as made up of inefficient troglodytes falling behind. That’s annoying. Meanwhile often I also see people lionize nation-state capabilities as superior to any other organization. Also annoying. The truth is somewhere in between. Sometimes the government does great work, sometimes it blows compared to private sector.

Take the CIA factbook I mentioned above as an example. It has been unclassified since the 1970s and by the early 1990s it was published on the web. Given wider distribution its “facts” came under closer scrutiny from academics. So non-gov people who long had studied places or lived in them (arguably the world’s true leading experts) read this fact book and wanted to help improve it — outsiders looking in and offering assistance. Perhaps some of you remember the “official” intelligence peddled by the US government at that time?

Bejtlich in his talk gives a nod towards academia being a thorough environment and even offers several criteria for why academic work is superior to some other governments (not realizing he should include his own). Perhaps this is because he is now working on a PhD. I mean it is odd to me he fails to realize this academic community was just as prolific and useful in the 1990s, gathering intelligence and publishing it, giving talks and sending documents to those who were interested. His presentation makes it sound like before search engines appeared it required nation-state sized military departments walking uphill both ways in a blizzard to gather data.

Aside from having this giant blind spot to what he calls the “outsider” community, I also fear I am listening to someone with no field experience gathering intelligence. Sure image analysis is a skill. Sure we can sit in a room and pore over every detail to build up a report on some faraway land. On one of my private sector security teams I had a former US Air Force technician who developed film from surveillance planes. He hated interacting with people, loved being in the darkroom. But what does Bejtlich think of actually walking into an environment as an equal, being on the ground, living among people, as a measure of “insider” intelligence skill?

Almost three decades ago I stepped off a plane into a crowd of unfamiliar faces in a small country in Asia. Over the next five weeks I embedded myself into mountain villages, lived with families on the great plains, wandered with groups through jungles and gathered as much information as I could on the decline of monarchial rule in the face of democratic pressure.

One sunny day on the side of a shoulder-mountain stands out in my memory. As I hiked down a dusty trail a teenage boy dressed all in black walked towards me. He carried a small book under his arm. He didn’t speak English. We communicated in broken phrases and hand gestures. He said he was a member of a new party.

Mao was his leader, he said. The poor villages felt they weren’t treated well, decided to do something about it. I asked about Lenin. The boy had never heard the name. Stalin? Again the boy didn’t know. Mao was the inspiration for his life and he was pleased about this future for his village.

This was before the 1990s. And by most “official” accounts there were no studies or theories about Maoists in this region until at least ten years later. I mention this here not because individual people with a little fieldwork can make a discovery. It should be obvious military schools don’t have a monopoly on intel. The question is what happened to that data. Where did information go and who asked about it? Did others have easy access to data gathered?

Yes, someone from private sector should talk about “The Revolution in Private Sector Intelligence”. Perhaps we can find someone with experience working on intelligence in the private sector for many, many years, to tell us what has changed for them. Maybe there will be stories of pre-ChoicePoint private sector missions to fly in on a moment’s notice into random places to gather intelligence on employees who were stealing money and IP. And maybe non-military experience will unravel why Russian operations in private sector had to be handled uniquely from other countries?

Going by Bejtlich’s talk it would seem that such information gathering simply didn’t exist if the US government wasn’t the one doing it. What I hear from his perspective is you go to a military school that teaches you how to do intelligence. And then you graduate and then you work in a military office. Then you leave that office to teach outsiders because they can learn too.

He sounds genuinely incredulous to discover that someone in the private sector is trainspotting. If you are familiar with the term you know many people enjoy as a hobby building highly detailed and very accurate logs of transportation. Bejtlich apparently is unaware, despite this being a well-known thing for a very long time.

A new record of trainspotting has been discovered from 1861, 80 years earlier than the hobby was first thought to have begun. The National Railway Museum found a reference to a 14 year old girl writing down the numbers of engines heading in and out of Paddington Station.

It reminds me a bit of how things must have moved away from military intelligence for the London School of Oriental and African Studies (now just called SOAS). The British cleverly setup in London a unique training school during the first World War, as explained in the 1917 publication “Nature”:

…war has opened our eyes to the necessity of making an effort to compete vigorously with the activities — political, commercial, and even scientific and linguistic — of the Germans in Asia and Africa. We have discovered that their industry was rarely disinterested, and that political propaganda was too often at the root of “peaceful penetration” in the field of missionary, scientific, and linguistic effort.

In other words, a counter-intelligence school was born. Here the empire could maintain its military grip around the world by developing the skills to better gather intelligence and understand enemy culture (German then, but ultimately native).

By the 1970s SOAS, a function of the rapidly changing British global position, seemed to take on wider purpose. It reached out and looked at new definitions of who might benefit from the study and art of intelligence gathering. By 1992 regulars like you or me could attend and sit within the shell of the former hulk of a global analysis engine. Academics there focused on intelligence gathering related to revolution and independence (e.g. how to maintain profits in trade without being a colonial power).

I was asked by one professor to consider staying on for a PhD to help peel apart Ghana’s 1956 transition away from colonial rule, for only academic purpose of course. Tempted as I was, LSE instead set the next chapters of my study, which itself seems to have become known sometime during the second World War as a public/private shared intelligence analyst training school (Bletchley Park staff tried to convince me Zygalski, inventor of equipment to break the Enigma, lectured at LSE although I could find no records to support that claim).

Fast forward five years to 1997 and the Corner House is a good example of academics in London who formalized public intelligence reports (starting in 1993?) into a commercial portfolio. In their case an “enemy” was more along the lines of companies or even countries harming the environment. This example might seem a bit tangential until you ask someone for expert insights, including field experience, to better understand the infamous pipeline caught in a cyberwar.

Anyway, without me dragging on and on about the richness of an “outside” world, Bejtlich does a fine job describing some of the issues he had adjusting. He just seems to have been blind to communities outside his own and is pleased to now be discovering them. His “inside” perspective on intelligence is really just his view of inside/outside, rather than any absolute one. Despite pointing out how highly he regards academics who source material widely he then unfortunately doesn’t follow his own advice. His talk would have been so much better with a wee bit more depth of field and some history.

Let me drag into this an interesting example that may help make my point, that private analysts not only can be as good or better than government they may even be just as secretive and political.

Eastman Kodak investigated, and found something mighty peculiar: the corn husks from Indiana they were using as packing materials were contaminated with the radioactive isotope iodine-131 (I-131). Eastman Kodak at the time had some of the best researchers in the country on its team (the company even had its own nuclear reactor in the 1970s), and they discovered something that was not public knowledge: those farms in Indiana had been exposed to fallout from the 1945 Trinity Test in New Mexico — the world’s first atmospheric nuclear bomb explosions which ushered in the atomic age. Kodak kept this exposure silent.

The American film industry giant by 1946 realized, from clever digging into the corn husk material used for packaging, that the US government was poisoning its citizens. The company filed a formal complaint and kept quiet. Our government responded by warning Kodak of military research to help them understand how to hide from the public any signs of dangerous nuclear fallout.

Good work by the private sector helping the government more secretly screw the American public without detection, if you see what I mean.

My point is we do not need to say the government gives us the best capability for world-class intelligence skills. Putting pride aside there may be a wider world of training. So we also should not say private-sector makes someone the best in world at uncovering the many and ongoing flaws in government intelligence. Top skills can be achieved in different schools of thought, which serve different purposes. Kodak clearly worried about assets differently than the US government, while they still kind of ended up worrying about the same thing (colluding, if you will). Hard to say who evolved faster.

By the way, speaking of relativity, also I find it amusing Bejtlich’s talk is laced with his political preferences as landmines: Hillary Clinton is setup as so obviously guilty of dumb errors you’d be a fool not to convict her. President Obama is portrayed as maliciously sweeping present and clear danger of terrorism under the carpet, putting us all in grave danger.

And last but not least we’re led to believe if we get a scary black bag indicator we should suspect someone who had something to do with Krav Maga (historians might say an Austro-Hungarian or at least Slovakian man, but I’m sure we are supposed to think Israeli). Is that kind of like saying someone who had something to do with Karate (Bruce Lee!) when hinting at America?

And one last thought. Bejtlich also mentions gathering intelligence on soldiers in the Civil War as if it would be like waiting for letters in the mail. In fact there were many more routes of “real time” information. Soldiers were skilled at sneaking behind lines (pun not intended) tapping copper wires and listening, then riding back with updates. Poetry was a common method of passing time before a battle by creating clever turns of phrase about current events, perhaps a bit like twitter functions today. “Deserters” were a frequent source of updates as well, carrying news across lines.

I get what Bejtlich is trying to say about speed of information today being faster and have to technically agree with that one aspect of a revolution; of course he’s right about raw speed of a photo being posted to the Internet and seen by an analyst. Yet we shouldn’t under-sell what constituted “real-time” 150 years ago, especially if we think about those first trainspotters…

Posted in Energy, Food, History, Poetry, Security.


Hillary, Official Data Classification, and Personal Servers

The debate over Hillary Clinton’s use of email reminds me of a Goldilocks’ tech management dilemma. Users tend to think you are running too slow or too fast, never just right:

Too slow

You face user ire, potential revolt, as IT (let alone security) becomes seen as the obstacle to progress. Users want access to get their job done faster, better, etc. so they push data to cloud and apps, bring in their own devices and run like they have no fear because trust is shifted into clever new service providers.

We all know that has been the dominant trend and anyone caught saying “blackberry is safer” is at risk of being kicked out of the cool technology clubs. Even more to the point you have many security thought leaders saying over and over to choose cloud and ipad because safer.

I mentioned this in a blog post in 2011 when the Apple iPad was magically “waived” through security assessments for USAID.

Today it seems ironic to look back at Hillary’s ire. We expect our progressive politicians to look for modernization opportunities and here is a perfect example:

Many U.S. Agency for International Development workers are using iPads–a fact that recently drew the ire of Secretary of State Hillary Clinton when she sat next to a USAID official on a plane, said Jerry Horton, chief information officer at USAID. Horton spoke April 7 at a cloud computing forum at the National Institute of Standards and Technology in Gaithersburg, Md.

Clinton wanted to know why a USAID official could have an iPad while State Department officials still can’t. The secret, apparently, lies in the extensive use of waivers. It’s “hard to dot all the Is and cross all the Ts,” Horton said, admitting that not all USAID networked devices are formally certified and accredited under Federal Information Security Management Act.

“We are not DHS. We are not DoD,” he said.

While the State Department requires high-risk cybersecurity, USAID’s requirements are much lower, said Horton. “And for what is high-security it better be on SIPR.”

Modernizing, innovating, asking for government to reform is a risky venture. At the time I don’t remember anyone saying Hillary was being too risky, or her ire was misplaced in asking for technology improvements. There was a distinct lack of critique heard, despite my blog post sitting in the top three search results on Google for weeks. If anything I heard the opposite, that the government should trust and catch up to Apple’s latest whatever.

Too fast

Now let’s look at the other perspective. Dump the old safe and trusted Blackberry so you can let users consume iPads like candy going out of style, and you face watching them stumble and fall on their diabetic face. Consumption of data is the goal and yet it also is the danger.

Without getting into too many of the weeds for the blame game, figuring out who is responsible for a disaster, it may be better to look at why there will be accidents/misunderstandings in a highly politicized environment.

What will help us make sure we avoid someone extracting data off SIPR/NIPR without realizing there is a “TS/SAP” classification incident ahead? I mean what if the majority of data in question pertain to a controversial program, let say for example drones in Pakistan, which may or may not be secret depending on one’s politics. Colin Powell gives us some insight to the problem:

…emails were discovered during a State Department review of the email practices of the past five secretaries of state. It found that Powell received two emails that were classified and that the “immediate staff” working for Rice received 10 emails that were classified.

The information was deemed either “secret” or “confidential,” according to the report, which was viewed by CNN.

In all the cases, however — as well as Clinton’s — the information was not marked “classified” at the time the emails were sent, according to State Department investigators.

Powell noted that point in a statement on Thursday.

“The State Department cannot now say they were classified then because they weren’t,” Powell said. “If the Department wishes to say a dozen years later they should have been classified that is an opinion of the Department that I do not share.”

“I have reviewed the messages and I do not see what makes them classified,” Powell said.

This classification game is at the heart of the issue. Reclassification happens. Aggregate classification of not secret data can make it secret. If we characterize it as a judgment flaw by only one person, or even three, we may be postponing the critical need to review where there are wider systemic issues in decision-making and tools.

To paraphrase the ever insightful Daniel Barth-Jones: smart people at the top of their political game who make mistakes aren’t “stupid”; we have to evaluate whether systems that don’t prevent mistakes by design are….

Just right

Assuming we agree want to go faster than “too slow”, and we do not to run ahead “too fast” into disasters…a middle ground needs to come into better focus.

Giving up “too slow” means a move away from blocking change. And I don’t mean achieving FISMA certification. That is seen as a tedious low bar for security rather than the right vehicle for helping push towards the top end. We need to take compliance seriously as a guide as we also embrace hypothesis, creative thinking, to tease out a reasonable compromise.

We’re still very early in the dinosaur days of classification technology, sitting all the way over by the slow end of the equation. I’ve researched solutions for years, seen some of the best engines in the world (Varonis, Olive), and it’s not yet looking great. We have many more tough problems to solve, leaving open a market ripe for innovation.

Note the disclaimer on Microsoft’s “Data Classification Toolkit

Use of the Microsoft Data Classification Toolkit does not constitute advice from an auditor, accountant, attorney or other compliance professional, and does not guarantee fulfillment of your organization’s legal or compliance obligations. Conformance with these obligations requires input and interpretation by your organization’s compliance professionals.

Let me explain the problem by way of analogy, to be brief.

Cutting-edge research on robots focuses on predictive capabilities to enable driving off-road free from human control. A robot starts with near-field sensors, which gets them about 20 feet of vision ahead to avoid immediate danger. Then the robot needs to see much further to avoid danger altogether.

This really is the future of risk classification. The better your classification of risks, the better your predictive plan, and the less you have to make time-pressured disaster avoidance decisions. And of course being driver-less is a relative term. These automation systems still need human input.

In a DARPA LAGR Program video the narrator puts it simply:

A short-sighted robot makes poor decisions

Imagine longer-range vision algorithms that generate an “optimal path”, applied to massive amounts of data (different classes of email messages instead of trees and rocks in the great outdoors), dictating what you actually get to see.

LAGR-view

What I like about this optimal path illustration is the perpendicular alignment of two types of vision. The visible world is flat. And then there is the greater, optimal path theory, presented as a wall-like circle, easily queried without actually being “seen”. This is like putting your faith in a map because you can’t actually see all the way from San Francisco to New York.

The difference between the short and long highlights why any future of safe autonomous systems will depend on processing power of the end nodes, such that they can both create a larger areas of more “flat” rings as well as build out the “taller” optimal paths.

Here is where “personal” servers come into play. Power becomes a determinant of vision and autonomy. Personal investments often can increase processing power faster than government bureaucracy and depreciation schedules. I mean if the back-end system looks at the ground ahead and classifies as sand (unsafe to proceed), and the autonomous device does its own assessment on its own servers and decides it is looking at asphalt (safe for speed), who is right?

The better the predictive algorithms the taller the walls of vision into the future, and that begs for power and performance enhancements. Back to the start of this post, when IT isn’t providing users the kind of power they want for speed, we see users move their workloads towards BYOD and cloud. Classification becomes a power struggle, as forward-looking decisions depend on reliable data classification from an authoritative source.

If authoritative back-end services accidentally classify data safe and later reverse to unsafe (or vice-versa) the nodes/people depending on a classification service should not be the only target in an investigation of judgement error.

We can joke about how proper analysis always would chose a “just right” Goldilocks long-term path, yet in reality the debate is about building a high-performance data classification system that reduces her cost of error.

Posted in Security.


BBC’s false history of long distance communication

One might think history would be trivially easy, given how these days every fact is on the Internet at the tips of our fingers. However, being a historian still takes effort, perhaps even talent. Why?

The answer is simple: “the value of education is not the learning of many facts but the ability of the mind to think”. I’ll let you try and search to figure out the person who said that.

A historian is trained to apply expertise in thinking, run facts through a system of sound logic for others to validate, rather than just leave facts on their own. It is a bit like a chef cooking a delicious meal rather than offering you a bowl of raw ingredients. Analysis to get the right combinations of ingredients cooked together can be hard. And on top of finding the results desirable, we also need ways to know the preparations were clean an can be trusted.

Take for example a BBC magazine article written about long distance communication, that cooks up a soup called “How Napoleon’s semaphore telegraph changed the world”.

This article unfortunately offers factual conclusions that are poorly prepared and end up tasting all wrong. Let’s start with three basic assertions the BBC has asked readers to swallow:

  1. The last stations were built in 1849, but by then it was clear that the days of line-of-sight telegraphy were done.
  2. The military needs had disappeared, and latterly the operators’ main task was transmitting national lottery numbers.
  3. The shortcomings of visual communication were obvious. It only functioned in daytime and in good weather.

First point: Line-of-sight telegraphy is still used to this day. Anyone sailing the Thames, or any modern waterway for that matter, would happily tell you they rely on a system of lights and flags. I wrote it into our book on cloud security. The BBC itself has a story about semaphore adoption during nuclear disarmament campaigns. As long as we have visual sensors, these signal days will never be done. Dare I mention the line-of-sight communication scene in a futuristic sci-fi film The Martian?

Second point: Military needs are not the only need. This should be obvious from the first point, as well as from common sense. If this were true you would not be reading a blog, ever. More to the stupidity of this reasoning, the French system resorted to a lottery because it went bankrupt. The inventor had pinned all his hope for a very expensive system on military financing and that didn’t come through. So the lottery was a last-ditch attempt to find support after the military walked.

semaphore-lottery

A sad footnote to this is the French military didn’t see the Germans coming in latter wars. So I could dive into why military needs didn’t disappear, but that would be more complicated than proving there were other needs and the system just wasn’t funded properly to survive.

Third point: Anyone heard of a lighthouse? What does it do best? Functions at night and in bad weather, am I right? Fires on a hill (e.g. pyres) also work quite well at night. Or a flashlight, such as the one on your cell-phone.

Try out the Jolla phone app “Morse sender” if you want to communicate over distance at night and bad weather using Morse code. Real shortcomings of visual communication come during thick smoke (e.g. old gunpowder battles or near coal power), which leads to audio signals such as the talking drum, fog horns, bagpipes and songs or cries.

Ok, so all those three above points are false and easily disproved, tossed into the bin. Now for the harder part, the overall general conclusion in two sentences from BBC magazine:

Smoke, fire, light, flags – since time immemorial man had sought to speak over space.

What France did in the first half of the 19th Century was create the first ever system of distance communication.

Shame that the writer acknowledges fire and flags here because those are the facts we used above to disprove their own analysis (work at night, still in use). Now can we disprove “first ever system of distance communication”?

I say this is hard because I’m giving the writer benefit of the doubt. Putting myself in their shoes they obviously see a big difference between the “immemorial” methods used around the world and a brief French experiment with an expensive, unfunded militaristic system.

As hard as I try, honestly I don’t see why we should call the French system first. Consider this passage from archaeologist Charles Jones’ 1873 “Antiquities of the Southern Indians

southern-indian-smoke-signals

Note this is a low-cost and night-time resilient system that leaves no trace. Pretty damning evidence of being earlier and arguably better. We have fewer first-hand proofs from earlier yet it would be easy to argue there were complex fire signals as far back as 150 BCE.

The Greek historian Polybius explained in The Histories that fire signals were used to convey complex messages over distance via cipher. A flame would be raised and lowered, turned on or off, to signal column and row of a letter.

6 The most recent method, devised by Cleoxenus and Democleitus and perfected by myself, is quite definite and capable of dispatching with accuracy every kind of urgent messages, but in practice it requires care and exact attention. 7 It is as follows: We take the alphabet and divide it into five parts, each consisting of five letters. There is one letter less in the last division, but this makes no practical difference. 8 Each of the two parties who are about signal to each other must now get ready five p215tablets and write one division of the alphabet on each tablet, and then come to an agreement that the man who is going to signal is in the first place to raise two torches and wait until the other replies by doing the same. 10 This is for the purpose of conveying to each other that they are both at attention. 11 These torches having been lowered the dispatcher of the message will now raise the first set of torches on the left side indicating which tablet is to be consulted, i.e. one torch if it is the first, two if it is the second, and so on. 12 Next he will raise the second set on the right on the same principle to indicate what letter of the tablet the receiver should write down.

It even works at night and in bad weather!

Speaking of which there may even have been a system earlier, such as 247 BCE. Given the engineering marvel of the lighthouse Pharos of Alexandria, someone may know better of its use for long-distance communication by line-of-sight.

Has the point been made that the first ever system of distance communication was not the French during their revolution?

I think the real conclusion here, in consideration of BBC magazine’s attempt to persuade us, is someone was digging for reasons to be proud of French militarism. Had they bothered to think more deeply or seek more global sources of data they might have avoided releasing such a disappointing article.

When native Americans demonstrated excellent long distance communication systems, European settlers mocked them. Yet the French build one and suddenly we’re supposed to remember it and say…oh la la? No thanks, too hard to swallow. That’s poor analysis of facts.

Posted in Food, History, Poetry, Security.


The German New Year’s Eve Terror Alerts

On the one hand we have RT telling us credible predictions of threats to safety were based on a tip from foreign intelligence services

“We received names,” [Munich police chief Hubertus] Andrae said. “We can’t say if they are in Munich or in fact in Germany.”

“At this point, we don’t know if these names are correct, if these people even exist, or where they might be. If we knew this, we would be a clear step further,” he added.

According to the Turkish security agency, the wider European strategy by the five individuals included churches and the sites of mass gatherings.

This led to travel warnings for people to avoid train stations, such as this one:

munichNYE2016

On the other hand, did the predicted events happen? Consider a BBC story reflecting back on New Year’s Eve in Germany, which does not seem to be put in context of any advance warnings.

The scale of the attacks on women at the city’s central railway station has shocked Germany. About 1,000 drunk and aggressive young men were involved.

City police chief Wolfgang Albers called it “a completely new dimension of crime”. The men were of Arab or North African appearance, he said.

Women were also targeted in Hamburg.

But the Cologne assaults – near the city’s iconic cathedral – were the most serious, German media report. At least one woman was raped, and many were groped.

Most of the crimes reported to police were robberies. A volunteer policewoman was among those sexually molested.

[…]

What is particularly disturbing is that the attacks appear to have been organised. Around 1,000 young men arrived in large groups, seemingly with the specific intention of carrying out attacks on women.

The problem with these stories side-by-side is twofold. First, increased police vigilance at train stations across Germany was the defensive plan against people experiencing terror, yet we’re being told now these attacks happened without notice. Violence against women at scale deserve real-time detection and response. Are authorities capable?

Second, is there clarity on what constitutes “organized” attacks? As we learn more, puzzle pieces of conspiracy are being placed on the table: “there had been reports of similar attacks on New Year’s Eve in other cities such as Hamburg and Stuttgart, although not on as massive a scale”.

I have not yet seen anyone report events in this light. The BBC report holds out the train station as a scene of terror without any mention of prior warnings, and without the police warning locations were still unknown: “We can’t say if they are in Munich or in fact in Germany”.

The looming dilemma is whether we now can say planned terror attacks happened in Germany on New Year’s Eve. As time goes on the number of women coming forward has been increasing to report assault. Why would or we say this was not a terror attack, especially as women soon after said they now fear being in public places? If we call it terror, some will complain of a slide towards loss of rights. If we don’t call it terror, some will complain of ignoring rights.

Posted in Security.


2015 Fachhochschule St. Pölten: Ethical Foundations for Information Security

I have been asked to post a copy of my presentation at the Fachhochschule St. Pölten. Vielen Dank an alle fürs Kommen und für die ausgezeichnete Diskussion.

Please find a PDF version here.

title slide

Posted in History, Security.


US Restitution for Wartime Internment of Japanese-American Civilians

The mayor of Roanoke, Virginia on November 18 made the following argument to block refugees:

I’m reminded that President Franklin D. Roosevelt felt compelled to sequester Japanese foreign nationals after the bombing of Pearl Harbor, and it appears that the threat of harm to America from Isis now is just as real and serious as that from our enemies then.

There’s no good way to say what this really means. This mayor is motivated by prejudice, hysteria and poor leadership. He is so ignorant of history he is unfit for his job; an embarrassment to America he should voluntarily resign his post.

It is a tragedy on two fronts:

  1. Syrian refugees are just looking for a better future. They have nothing to do with ISIL attacks. From a risk perspective they not only are safe, actually they could become a valuable asset in the fight against ISIL.
  2. Xenophobia has a long dark history in America usually linked to groups such as the KKK who practice terrorism, torture, murder. Virginia local news recently has reported “biggest resurgence of the Klan since 1915″. The Roanoke Mayor should be investigated for his ties to these groups.

We must put any call for return to internment camps in proper historic context. Here is President Ronald Reagan’s speech on August 10th, 1988 saying America made a mistake, would apologize and make amends for internment camps. It is abundantly clear no American ever again, let alone an elected official, should try to frame the camps as positive in any way.

Speech by Ronald Reagan, as documented in the Ronald Reagan Presidential Library

Remarks on Signing the Bill Providing Restitution for the Wartime Internment of Japanese-American Civilians

August 10, 1988

The Members of Congress and distinguished guests, my fellow Americans, we gather here today to right a grave wrong. More than 40 years ago, shortly after the bombing of Pearl Harbor, 120,000 persons of Japanese ancestry living in the United States were forcibly removed from their homes and placed in makeshift internment camps. This action was taken without trial, without jury. It was based solely on race, for these 120,000 were Americans of Japanese descent.

Yes, the Nation was then at war, struggling for its survival and it’s not for us today to pass judgment upon those who may have made mistakes while engaged in that great struggle. Yet we must recognize that the internment of Japanese-Americans was just that: a mistake. For throughout the war, Japanese-Americans in the tens of thousands remained utterly loyal to the United States. Indeed, scores of Japanese-Americans volunteered for our Armed Forces, many stepping forward in the internment camps themselves. The 442d Regimental Combat Team, made up entirely of Japanese-Americans, served with immense distinction to defend this nation, their nation. Yet back at home, the soldier’s families were being denied the very freedom for which so many of the soldiers themselves were laying down their lives.

Congressman Norman Mineta, with us today, was 10 years old when his family was interned. In the Congressman’s words: ”My own family was sent first to Santa Anita Racetrack. We showered in the horse paddocks. Some families lived in converted stables, others in hastily thrown together barracks. We were then moved to Heart Mountain, Wyoming, where our entire family lived in one small room of a rude tar paper barrack.” Like so many tens of thousands of others, the members of the Mineta family lived in those conditions not for a matter of weeks or months but for 3 long years.

The legislation that I am about to sign provides for a restitution payment to each of the 60,000 surviving Japanese-Americans of the 120,000 who were relocated or detained. Yet no payment can make up for those lost years. So, what is most important in this bill has less to do with property than with honor. For here we admit a wrong; here we reaffirm our commitment as a nation to equal justice under the law.

I’d like to note that the bill I’m about to sign also provides funds for members of the Aleut community who were evacuated from the Aleutian and Pribilof Islands after a Japanese attack in 1942. This action was taken for the Aleuts’ own protection, but property was lost or damaged that has never been replaced.

And now in closing, I wonder whether you’d permit me one personal reminiscence, one prompted by an old newspaper report sent to me by Rose Ochi, a former internee. The clipping comes from the Pacific Citizen and is dated December 1945.

“Arriving by plane from Washington,” the article begins, “General Joseph W. Stilwell pinned the Distinguished Service Cross on Mary Masuda in a simple ceremony on the porch of her small frame shack near Talbert, Orange County. She was one of the first Americans of Japanese ancestry to return from relocation centers to California’s farmlands.” “Vinegar Joe” Stilwell was there that day to honor Kazuo Masuda, Mary’s brother. You see, while Mary and her parents were in an internment camp, Kazuo served as staff sergeant to the 442d Regimental Combat Team. In one action, Kazuo ordered his men back and advanced through heavy fire, hauling a mortar. For 12 hours, he engaged in a singlehanded barrage of Nazi positions. Several weeks later at Cassino, Kazuo staged another lone advance. This time it cost him his life.

The newspaper clipping notes that her two surviving brothers were with Mary and her parents on the little porch that morning. These two brothers, like the heroic Kazuo, had served in the United States Army. After General Stilwell made the award, the motion picture actress Louise Allbritton, a Texas girl, told how a Texas battalion had been saved by the 442d. Other show business personalities paid tribute–Robert Young, Will Rogers, Jr. And one young actor said: “Blood that has soaked into the sands of a beach is all of one color. America stands unique in the world: the only country not founded on race but on a way, an ideal. Not in spite of but because of our polyglot background, we have had all the strength in the world. That is the American way.” The name of that young actor–I hope I pronounce this right–was Ronald Reagan. And, yes, the ideal of liberty and justice for all–that is still the American way.

Thank you, and God bless you. And now let me sign H.R. 442, so fittingly named in honor of the 442d.

Thank you all again, and God bless you all. I think this is a fine day.

Note: The President spoke at 2:33 p.m. in Room 450 of the Old Executive Office Building. H.R. 442, approved August 10, was assigned Public Law No. 100-383.

Posted in History, Security.


Why Do We Hack?

I’ve seen recently some weird speculations on motive of a hacker. Personally I prefer to focus on consequence because that defines our control options best. I learned to make this switch while studying the history of Vietnam War and seeking motives.(1) What motivated American leaders to kill so many people? Try reading “Advice for Soldiers in Vietnam: The Fish is Good

…young, uneducated soldiers…had to be told why they were going to Vietnam, from which, after all, they might not return. “It is interesting, that the [US Department of Defense guide for soldiers] accurately and briefly describes the history of the Vietnamese resisting outsiders—the Chinese and others—while assuming that we could never be cast in this light.” To do this required telling some of the same lies that the government was telling the public and, for the most part, telling itself.

It’s basically impossible to clarify motive in this sort of context, whereas understanding the consequences is comparatively easy and can greatly affect motives in future: outsiders faced resistance. So time spent studying history really was learning to distill accurate consequences from action to help inform future paths; avoid predictable mistakes.

But still I understand that discussion of motive is attractive to many and there’s some merit to getting lost in speculation so here’s mine:

As I’ve said before I believe everyone is a hacker. In brief it seems to me to be a condition of economics and politics, laced with philosophy. If you find an obstacle in your path then hacking is a way to work around or even through resistance instead of using more direct methods. The asymmetry, disobedience to routine or expectation, is what I find at the foundation of hacking.

Theories of hacker motive that settle on addiction or male sexual fantasy as foundations have mistaken a small tree for the entire forest; symptoms such as these are woefully lacking in perspective.

Let us take as assumption that given a choice humans tend to go a path of lesser resistance. The more privilege or authority one has the more choices of low resistance, and less cause for a hack to get around resistance spots. Having total control therefore means the least hack incentives. Got root? Whereas, having the least control options for a desired changes brings highest incentive to start hacking.

Why would the an intelligence agency hack? They calculate a path to greater control for less cost (including blowback) than other options. Why would the activist hack? They calculate a path to greater justice for less cost than other options.

Perhaps I can explain using a counter-example. Addiction makes no logical sense to me as THE hacker motive. It is just one shade or flavor. The word addict comes from latin addictus, which means a person enslaved as a consequence of debt or crime.

Imagine a child abandoned by parents, or bullied by older schoolmates, and you have someone with potential incentive to see asymmetry as a best option against the obstacles in the way of their personal success. Tempting as it might be to describe them as addicts; it is false to assume use of asymmetric methods to overcome would lead to a form of slavery. They are not addicts if a control level they seek through resource-constrained methods is reasonable and achievable.

The addiction theory says hackers want more control because they are addicted to more control. This sounds like an administrator, not a hacker. You want more control? You get a job that gives you more control, and a promotion to more control, and another one. Hacking not required. Should we call a promoted system administrator an addict because increased authority achieved and desired? If they can choose to exit of free will, no. Addiction is a way to describe those with a high exit barrier/cost.

Moreover a tautology such as “want more power because power is wanted” should have been shot-down in the very first presentation review-cycle. Addiction to growth of power is separate from and does not pre-suppose any need for hacking because not-hacking (following procedures) also can end in the same place of more power. Obviously if one wanted to amass power and be enslaved by it (e.g. run a debt and be unable to pay) hacking still is not necessary, so it is hard to see it as THE logical justification to hack.

An asymmetry theory even explains away the (incredibly vapid) accusation that “penetration testing” could be a manifestation of man’s desire to stick their penis into everything. Hopefully I don’t have to explain why a male-only theory of motivation fails at first blush. Let it suffice to say people without a penis also see penetration opportunity to gain entry where they aren’t authorized. The risks of unauthorized entry is a much broader subject (i.e. women stealing) than just men being dicks.

Let’s face it, hacking is really about power, which brings me to think of it in terms of economics, politics and philosophy. Psychology may help study why a child abandoned by a parent feels transfer power and needs to react in a non-standard way. I don’t think that will really explain when and how authority, or let’s just call it privilege, will have to deal with those who learn and engage with asymmetry rather than sit bored because symmetry is a pipe dream. And hacking therefore also is not always bad. Asymmetric approaches can be known by their more common labels of innovation or creativity.

The question people really should be answering is when is it ethical to innovate or use creativity instead of following routines.


(1) John Stuart Mill, Utilitarianism (London: Parker, Son and Bourn, 1863), page 26-27, argued good behavior comes from questionable intentions so best to ignore and focus on outcome.

Posted in Security.


Would removing DMCA reduce pollution?

In response to my earlier posts on VW cheating I have heard several people say “I don’t know engines well so I don’t follow most of what you’re saying”. This is a familiar hurdle, true for most specialized technical fields.

I don’t mind hearing this because I am a believer in bridging. I see no point in shaming people who lack hands-on engine experience or have not thought deeply about the economics of transportation. A technical argument should be able to stand on its own, such that it can be explained to anyone.

So here I will attempt to build a bridge from being a long-time engine tuner to the growing number of very smart IT and infosec people without any real engine experience who suddenly now are looking into smog topics.

More specifically I will answer from experience whether removing engine DMCA immediately would help in the case of VW cheating.

Three Levels of Analytics

On the beginning end of an analytic spectrum, the thought that immediate DMCA removal “probably would help” is a binary form of assessment: see something say something. DMCA is a prior known harm. It has done harm elsewhere. When DMCA is noticed therefore its removal is a simple reaction.

Next on the spectrum is knowing that DMCA can be a harm yet wondering based on ranked data if removal will achieve an objective. Seeing DMCA used by a German car company could mean every German car company is suspect. A ranking system begs the question of how to know when and if safe transition away from DMCA is possible? Is it after German cars no longer are available for sale?

The training examples I suggest to answer this question are from other scandals related to privacy. Lance Armstrong, like VW, was a winner caught cheating. However Lance wasn’t the problem, he was a symptom of demand. He represented a far wider problem.

Using first level analytics (see Lance with privacy and say something) would not be the right approach. Likewise second level analytics are insufficient because Lance was not the only cheater.

Getting beyond level two analytics is very hard. Anyone with audit experience knows it can be a losing battle on the ground unless you have real infrastructure in place to support a search for knowledge. You have to be able to store data, evaluate and adapt. The better your tests the more your cheating adversaries will circumvent them so you need some way to win that race.

A sophisticated level of knowledge is a third level of analytics, which I will call heatmap. As signs of cheating emerge, none very special on their own, the probability is warmer overall. Privacy is not completely lost, but reliable indicators of cheating are developed broadly. This involves sensors so fast, unique and rich in detail that the cheater can not afford to keep ahead of them.

There are two more levels of analytics above heatmap unnecessary to discuss here. Suffice it to say a third level gets us to where we need; it should answer whether and when removing DMCA would be improving air quality.

I use my own experience to work through finding a third level analytics answer. It comes from tuning many engines and even making my own fuel over the last decade. Here are two reasons why I think removing DMCA is a distraction from the main issue: free market risks and the economics of performance tuning.

Free Market Risks

Removing DMCA would be great for innovation and cost improvements from shared knowledge. It would create a more free and unregulated market. That however is not going to magically make pollution stop.

More of something and cheaper doesn’t imply clean. In fact it could be the opposite as the market innovates toward more power for less money. Removing DMCA arguably means the market continues in the worst possible direction and pollution simply increases.

Can we avoid innovation going awry? Yes, with regulation specific to the objective. DMCA is a weak control for issues of competitiveness and innovation, only slightly related to the issue of keeping air clean. Removing it should come when we are able to regulate for clean air.

Removing Lance Armstrong’s privacy could actually make his cheats more pervasive and harder to detect by auditors. So could we improve detection without removing privacy completely? Absolutely yes.

Some suggest the VW cheat was caught using sophisticated testing. I think that’s an exaggeration but we still should look at the tests as an example to model. The auditor success really was in perseverance and perspective more than doing anything clever or novel. Someone kept thinking mpg and power advertised were too good to be clean, so they applied a clean-specific test where VW did not.

Take a moment to think of the VW cheat this way:

  • When you are stationary (garage, warm up in snow) you get cleaner air
  • When you drive, you get more power but it is dirty

This is exactly, and I mean exactly, what typical American customers demand of manufacturers. It is considered acceptable to pollute in the areas least likely to be measured. This is why you can buy “off-road use only” performance parts (meant to be used privately) and then drive them around on roads (publicly) without any real risk of prosecution or fines.

So with pervasive cheating and cheating ingrained in the American engine market why did regulators focus on one company? In brief because it is harder to ban pollution by cheating American consumers than it is to go after a wealthy German company with a minority of vehicles on the road.

To put this in perspective VW already had their cars banned from the California market in 2004. They came back in 2008 with some incredible new numbers and sales took off. All of this has been blogged here extensively before.

A good auditor sees improvement and immediately starts thinking skeptically; how did a small car sales winner get so good so fast (the answer is Bosch, who actually developed “off-road use only” codes). And then the auditor hunts. Sending a car across the country with sensors is not a super special or novel idea, which perhaps you have read in my prior blog posts (e.g. Jaguar boasted 62mpg in cross country test).

Auditors today are closing in on manufacturers because the market functions in a somewhat predictable manner. Changing this abruptly by opening up innovation could lead to many more polluters, groundswell of people acting more like VW (because you’ve removed VW from the equation) and even take us towards weakening of other controls focused on clean air. A focus on a winner with a clear-cut case is a very efficient form of regulation but insufficient, since the problem is widespread.

All of this says to me removing DMCA and opening up a free market without other forms of regulation in place would likely be a clean air setback. It would be like demanding the recipe for cyclist performance enhancing drugs be public in order to reduce their use. Unless cyclists and race organizers are prepared to regulate against use, releasing the recipe can lead to far more cheating and less chance of stopping it.

Performance Tuner Economics

It is well known in the engine market that DMCA does not stop people from completely reverse engineering their cars. Performance tuning firms, not to mention customers themselves, often reverse firmware and/or write their own. In fact you could say there is a symbiotic relationship where the weak enforcement of DMCA allows manufacturers to learn from the after-market crowd what power enhancements to sell next.

Note here there is literally no market for clean enhancements. You simply can not find after-market products designed to get the cleanest possible emissions from your engine.

What VW did was realize that customers wanted more power, more mpg, as they always do. This translates to more convenient “workarounds” and double-speak to avoid regulations of being clean. Thus instead of customers paying $100 and taking 10 minutes to after-market tune their engine, VW essentially modeled customer behavior and provided a solution in software.

VW probably figured why leave the fixes to after-market performance companies. They also likely saw it as a temporary workaround to get back into the market sooner (2008) instead of when they had figured out how to actually comply: both power and clean (2013). Classic product manager risk behavior.

The pervasive cheating that drives VW to do the same is both good and bad. On the one hand it is bad because the market obviously and flagrantly pollutes and no one has budget or tools to stop it at the widespread consumer level. On the other hand it is good because VW took the unrepentant customer bait for better cheats, brought it in-house, and gave regulators a one-stop shop to issue a fine and make an example for everyone to see.

Using our Lance Armstrong example, he cheated more and better than all the other cheaters, which made him the best person to take-down in front of everyone as an example. Some people say VW had 11 million cars affected and this is a lot. Unfortunately this is not a lot in the big picture of cheating.

I mentioned before that California took action in 2005 and knocked VW out of the market. This was because VW was big enough to be a centralized high-profile target but small enough and consumer-centric enough to be made into an easy example. Much more difficult would be for regulators to go after Ford, GM, Kenworth, Caterpillar, John Deere, etc..

Instead of only affecting a few million consumers a regulation at the much larger cheater level could seriously impact business processes and even shut them down. It is common to hear truck drivers complain that if they have to drive a clean engine in order to operate in California they will go out of business; lower mpg or less power to stop polluting is a very hard business decision for hundreds of millions of drivers.

DMCA therefore doesn’t really stop people from innovating (albeit in non-clean direction). So it would have to be enforced far more strictly to help keep air clean. That would be a very bad thing. Harming innovation to reduce pollution sounds backwards because it is. The same resources instead of trying to enforce DMCA could be used directly for enforcing actual clean air controls. The goal being when you finally remove DMCA the resulting innovation would be pointed in a positive direction.

This is why I say stop wasting time talking about DMCA in pollution circles (a mostly non-barrier to reversing and tuning) when you directly could be addressing the actual problems of cheating for actual air quality controls.

Building a Better Solution

In conclusion, I hope I’ve built the argument well enough to stand on its own, no special engine experience necessary. We need to be building a far better surveillance network to monitor for clean air and a far more effective response system for enforcement. This probably sounds shocking so the ethics and norms of behavior have to be ironed out. We should put it in terms of other pollution success stories.

When you see someone smoking a cigarette you say something to them. If that person doesn’t comply you invoke authority. Obviously you can’t tell on the spot you are getting cancer but you have it on good authority that seeing a smoker is reason to act. DMCA of the cigarette industry, such as recipes for mixing and rolling, seem mostly irrelevant because they are.

Thus we really should ask ourselves for engines how do we build a comfortable living environment still capable of finding and stopping engine-smokers?

Imagine every loud pipe you hear is reason enough to say something. Generally loud pipes are after market power improvements that intentionally increase pollution. The ear is no perfect sensor but it’s a start (albeit California regulators have been arguing they can decouple noise from pollution). Imagine neighborhoods using air quality sensors deployed to help build a heatmap; for example monitoring outside popular restaurants collecting data on SUV emissions left behind. You then deliver to the restaurant their pollution results and fine them based on their customer behavior.

There are many possibilities of great impact to consider and plan. Arguments about removing DMCA are mostly irrelevant to clean air economics and technical problems.

Posted in Energy, History, Security.