This article was first published on June 27 2016 on my linkedin page: https://www.linkedin.com/pulse/from-gods-guts-science-golden-era-data-analytics-gyasi-dapaa/
Part I: What’s the Craze?
The chief role of executives has always been to make decisions but this function has been executed differently at different eras of history. In the times preceding the seventeenth century, when human understanding of numbers was frail if at all, decisions were based on faith, hope and guesswork (see HBR article, A Brief History of Decision-Making by Leigh Buchanan & Andrew O’Connell). It was not uncommon for people to bank their fates on the presumed telegnosis of the stars, oracles, priests, and other cultic bodies. However, as our knowledge in numeral systems and many intellectual disciplines such as mathematics, economics, psychology, political science and others evolved over the centuries, our abilities to make decisions and assess associated risks matured as well. The toolkit of the twentieth century executive sophisticatedly developed to include cost benefit analysis, net present value analysis, capital asset pricing models (CAPM), valuation models, demand and supply models and many others.
Nevertheless, as tools for decisions became sharper, the dynamics of modern business also became increasingly complex, sometimes incapacitating their (the tools’) ability to generate any meaningfully actionable insights, a situation popularly referred to as analysis paralysis. In his book, ‘Organized Mind’, Dan Levitin helps paint the picture of how markedly complex our modern, in contradistinction to earlier worlds, have been: Dan explains that we take in five times as much information (the equivalence of 175 newspapers) as was absorbed in 1986. In addition to the complexity of information, most companies struggled with the institution and maintenance of good data structures, which would have helped them to capitalize on the nuggets of winning insights buried in the messy, unstructured data they collected. These consequently ignited a romantic appeal of gut calls to executives, who sought to benefit from the palpable benefits of first mover advantages and speed to the market, enabled by fast but frugal decision-making culture.
The advent of big data─ data being produced in high volumes and velocity such as telematics ─ coupled with breakthroughs in computational power has however presented an immense competitive advantage for companies to strategize based on data insights than on the traditional lure of the gut calls and hip shots. Daniel Khaneman and other cognitive scientists have continuously reminded us, in a rich set of published literature, about how vulnerable our human intuition is. Common biases such as sunk cost fallacy (tendency to allow historic irrecoverable losses to influence future decisions), overconfidence bias (our utter belief in our ability to deliver despite contrary historic results), confirmation bias (selectively seeking only counsel that confirms our prior beliefs), only to mention a few, continue to affect the quality of our intuitive decisions. All of these forces have sparked strong desire in companies to leverage cutting edge analytics and evidence-based solutions to uncover new territories of immense values otherwise considered impossible a few decades ago.
Today, data analytics are being used to help drive business strategies in all sorts of ways in various industries (Financial, Political, Education, Energy, Manufacturing, etc): From figuring out the target customers or market appetite of companies, to recruiting best of best employees, to devising marketing strategies (profiles of customers with highest propensity to purchase), to detection of fraud, to forecasting what revenue of a company will be in a specified future year, to you just name it. Most companies, to illustrate, know from repeated experiences, which industries, or states, or customer business sizes are profitable. Such one-dimensional views are normally called univariate perspectives, and are common across many companies. However, analytical companies leverage data to develop multi-dimensional perspectives to deepen their competitive edge in the market. For instance, a company may generally know that its construction segment is unprofitable; however, such a business knowledge can be misleading because there could be a subsegment within the construction segment (such as construction businesses with number of employees above a specified threshold) that’s highly profitable. Sophisticated data science analysis therefore provides a powerful lens into multi-dimensional insights otherwise obscured by intuition and experience. This (i.e. the nuanced lens of multi-dimensional analysis) explains why most analytical companies expend many resources to building sophisticated multivariate predictive models with many explanatory variables.
The various types of analytical solutions can be categorized into four main rungs in the following order of sophistication:
Reporting Analysis: What is the state of my company?
Forecasting: What will be the state of my company?
Predictive Modeling: Determine the drivers of a target variable of interest (such as Customer profitability, employee productivity, fraudulent score, etc.)
Optimization: Synthesize current state, all the relevant driver variables, and all possible future paths to determine an optimized strategy for company.
What’s hot in the job market has definitely changed over the years. In the ’80s-90s, being a quant with mathematical dexterity to model financial derivatives was the envy of jobseekers; in the middle to late 90s, having the ability to operate the computer and manage information technology systems was the ideal job of the modal job candidate; the twenty-first century has borne a new favorite and it’s having the ability and agility to massage data in creative ways. Hal Varian, the chief economist of Google, is known to have deemed data science as the sexiest job in the next decade. Join me in Part II of this article for a discussion on how you can tap into the industrial brio of fun, reward, and value.
References:
- Leigh Buchanan & Andrew O’connell ,‘The Brief History of Decision-Making’, HBR Article
- Thomas Davenport & D.J. Patil, ‘Data Scientist: The Sexiest Job of the 21st Century’, HBR Article
- Daniel Khaneman, ‘Thinking Fast and Slow’, New York: Farrar, Strous, Girroux
