Mike Greenfield has some really insightful things to say on his blog about big data statistical risk and the difficulty in predicting human behavior. Take for example his experience with starting a company, which proved how dangerous it was to rely on a sole supplier.

So Facebook acted rationally, optimizing for their own best interests and those of their users. They killed the notifications feature (which we used to tell someone her friend’s child was turning two). They removed boxes and tabs from profile pages (which over a million moms had added to show off their kids’ accomplishments). And they hid invitations (which moms used to tell their friends about our product).

At that time, we were almost completely dependent on Facebook’s channels to communicate with our users and find new ones. We felt like a beer maker preparing for the government banning beer sales in markets, shutting down bars, and only allowing people to drink in restaurants on Tuesdays. Not quite prohibition, but pretty darned close.

I want reiterate that Greenfield is in the business of predicting human behavior based on data analysis. Although he says “Facebook acted rationally” he actually started his blog post with “Facebook, the VCs said, could suddenly turn off all of their communication channels and we’d collapse. We thought they were full of it…”.

Why didn’t he see it coming?

It sounds to me that VCs predicted the danger of losing a sole supplier. That makes sense in a simple predictive risk model. A “rational” behavior model for suppliers who see economic opportunity, however, is a complex and messy business. It really shouldn’t be so casually described as if a supplier who kills their distribution channel is predicted easily or is rational/optimizing.

Although I love the prohibition analogy it probably is not for the reasons Greenfield uses it. Prohibition is a good example of bad regulation and resulting security risks.

Consider for a moment how the consumption of alcohol actually increased in America after it was banned. If Facebook’s regulation of data were like prohibition then we should predict an illegal data running/smuggling boom.

That didn’t happen, as documented by Greenfield. Instead his story centers on “cutting the cord” and walking away from Facebook forever.

Also consider that prohibition in America was led by popular religious extremists (well, popular in Kansas anyway) who violently forced into power a bunch of blatent hypocrites.

The “conservative” politicians who said they favored a “dry” country ended up meaning someone who drank but refused to admit it. In today’s terms it is similar in nature to the radically homophobic politicans.

Those calling for regulation thus can be mired in complex psychological and cultural issues, which makes “rational” predictions of their economic behavior less than obvious. Was Greenfield accounting for a fundamentalist Carrie Nation element to Facebook when he was threatened by “hatchetation” of his data?

The really interesting point of Greenfield’s story is that at the same time he (like most people) predicted a demise of email and replacement with social networking (risk of staying on email), he also was using the venerable traditional direct-communication path of email to save his company from destruction.

As 2010 came to a close, the proverbial feces was hitting the proverbial fan, and we started to look at email as a way out of the ditch. […] Over the course of 2011, we streamlined our content-writing and emailing operations, in the process turning email into a viable re-engagement channel for millions of moms.

The lesson of course is to predict and manage risk related to distribution channels to your customers, which is what the VCs told them in the first place. It sounds to me had he followed his own risk analysis based on a prediction of the future he would have been far worse off. In other words don’t stop using email unless you realize the true risk of giving up ownership and control over your communication.

Fast forward to Greenfield’s more recent post called “Predicting the Future is the Future” and he extols automation.

Automation is incredibly important. It democratizes the process of building and using statistical models, so that a small startup (with lots of data) can build pretty good statistical models without a team of statisticians. These automated statistical models will almost inevitably perform more poorly than their human-built counterparts, but they’re close enough to be competitive.

I really want to agree with him, because technology can make data more accessible and therefore more democratic. Giving out statistical model tools to everyone means they too can start a company and make money from mining your personal data.

But again he leaves out an essential part of behavior — who gets to own and control access to data. This part of risk has to be better defined before we can celebrate democracy and a risk reduction.

His description of the troubles with Facebook give a clear example of how automation can be rendered completely useless — it runs straight into severe power inequality in terms of resource control and management risks.

Alas, back to the Facebook prohibition analogy, every farm in America used to have an apple tree, if not an orchard. Yet the saying “as American as apple pie” is a subtle reminder of the strange story of hard cider in America.

150 years ago, in the 1840s, hard cider held the position now held by beer as the preferred alcoholic beverage of the working class.

Where did it go? It turns out that while technology democratized the process of building farms and making goods it alone was unable to prevent the extinction of the preferred beverage in America.

…the temperance movement remains as a major culprit responsible for the decline of cider consumption in the U.S., but the association of cider with rural WASP culture was the added factor which distinguishes cider from beer or wine. Add to this the economics of beer production, growing urbanization, German immigration, a predatory beer industry, and a substitute drink in coca-cola, and there seems to be enough factors working together to explain why and how cider so completely disappeared.

A statistician looking at data in 1840 might have said cider was the future, but the question is whether they could or would have predicted a much more complicated mix of risk factors related to irrational human behavior (e.g. religious fervor and ethnic prejudice) that killed the market.

England’s farmers were insulated from the risk of politics and industry in early 1900s America, so they still make cider:

Source: Broome Farm on Flickr.

Mother Earth News says it is not too late to learn how to make your own American cider…assuming you can find a reliable apple distributor.