Courage of Simplicity and Big Data Science

General Patton may be the first to initiate this trend of Speed, Simplicity and Boldness.

Jack Welch later customized this to business world as Speed, Simplicity and Self-Confidence. In that HBR interview, he ridiculed people with 4-inch thick books, full of what every 5th grader knows.


Now Philipp K. Janert talks about his experience in data analysis field, in the book Data Analysis with Open Source Tools.

Once he noted a team trying to solve a case by running a neural network on a computer cluster. Yet, a working solution would take a mere 1-line calculation of some very simple techniques. He saw "an excess of sophistication" everywhere.

Before he recommends his cure, he diagnoses the cause as discomfort arising from unfamiliarity and uncertainty. In a new domain or problem space, people become unsure as to how to proceed and thus begin looking for silver bullets and miracle cures. The more impressive, the more expensive, the more hyped, the more complicated, the better for them.

His recommendat‭ions:

1. Methods you know for sure, methods you understand fully
2. Simple, transparent approaches and methods, because they are cheaper, less error-prone, and more revealing of insights.
3. Stay with the simple methods until they prove insufficient

He then goes on to say :

1. "Big Data" is often small, small enough to reside in Excel of the Finance Departments.
2. "Big Data" is not always better since the classical statistics's concept of sampling obviates "the need to examine every single member of a population ..."

Big data, and therefore "Big Guns"?

You decide.