Archive for the ‘Big Data’ category

My Experiences -Last time (MELT) – 0214- “In God we trust. All others must bring data”

March 17, 2014

If you are reading my blog for the first time, please read my previous ( MELT 0114) blog to get the context and continuity. Let me start my blog with a famous quote “In God we trust. All others must bring data”
Elementary Mr Suresh your topic for next blog is “Big Data” This was the answer I was expecting. Unfortunately no one really predicted this although some were close but not on target (sic).

We did not target your daughter but future moms
Last week I wrote about how a father got upset on Target for sending e-mail promotion for baby cloths to his 16-year-old daughter. “The father visited the store again within a week and met the manager. Here is what transpired
“I had a talk with my daughter,” he said. “It turns out there’s been some activities in my house I haven’t been completely aware of.” He took a deep breath. “She’s due in August. I owe you an apology.”
In some 600 hospitals across the United States, new mothers get gifts from the various companies. Those companies have started a separate division to focus on parents of infants. Procter & Gamble sends gifts to new moms at the hospitals where they deliver the child.
Target thought it was too late to approach in the maternity ward and wanted to predict future moms through their buying patterns. They asked Andrew Pole, a data expert to develop pregnancy prediction algorithms based on certain trends such as buying lotions, unscented soaps, vitamins etc.
Here is the sample of what those analytics did based on buying patterns
Jenny Ward, a twenty-three-year-old bought cocoa butter lotion, a purse large enough to double as a diaper bag, zinc, magnesium, and a bright blue rug? The tool predicted that there is 87 percent chance that she’s pregnant and that her delivery date is sometime in late August. Liz Alter, a thirty-five-year-old who purchased five packs of wash cloths, a bottle of “sensitive skin” laundry detergent, baggy jeans, vitamins containing DHA, and a slew of moisturizers? She’s got a 96 percent chance of pregnancy, and she’ll probably give birth in early May. Caitlin Pike, a thirty-nine-year-old in San Francisco who purchased a $ 250 stroller, but nothing else? She’s probably buying for a friend’s baby shower. Besides, her demographic data shows she got divorced two years ago. Amazing isn’t it?
Who is the big brother of the modern world?
We are undergoing a revolution in the way that information about our purchases, our conversations, our social networks, our movements, and even our physical identities are collected, stored, analyzed and used. The immense volume, diversity, velocity, and potential value of data will have profound implications on privacy, the economy, public policy and corporate strategies.
Government collects data for various reasons and so do corporates for improving their market share and profits. Social network collect data to attract advertisements. Imagine the implication where are all data are combined to start predicting and tracking your behavior. This is the big brother of modern world- BIG DATA.

Big Data- What is it
We have been using Data for analytics since ages immemorial so my readers must be wondering what the hype all about is. In the last five decades data have been used for decision making in various terms such as decision support, executive support, online analytical processing, business intelligence and analytics. Typical characteristics of data analytics in the past have been based on sampling, trends and/ or structured data (most often static). Inability to process data faster and judicious allocation of budgets for analytics were limiting factor in leaping to “Big Data” world. . We often heard “Data Costs Money”.
Netflix, Amazon, Google and others have proved through robust analytics that Data costs money but also could make tons of money. Target retail chain in this blog is a classic example of using data to enhance sales and market share. Amazon tries to predict your next purchase based on your shopping history, and gives their recommendation. They are using data although voluminous but mostly generated internally. This is where the shift is happening.
Big Data is much more than this. Typical characteristics of big data are as follows:
1. Large volumes, unstructured, continuous and in multiple formats (texts, pictures, voice)
2. Move away from sample and use whole (all) Data
3. Accept messiness for the reasons mentioned in one and two above
4. More importantly to accept correlation
5. Multiple location of data
Big data comprise of 100 terabytes to petabytes as compared to few terabytes or less in traditional analytics. To give you a feel, if we have to store 100 terabytes in any digital format, it will occupy every inch of the tallest building in the world (somebody prove me wrong here).

What will you do with the big data ? Torture the data until they confess
Ronald Coase said” If you torture the data long enough, it will confess”
An abortion lead to drop in crime rate was the conclusion drawn by the authors of Freakonomics but by a sheer coincidence and not by structured correlation. Now through Big Data we have a tool and methodology to look for other correlation for example drop in prices of condoms or arrival of new birth control pills.
We are living in age of data abundance and business leaders are ready to exploit this abundance and bring transformational change in the way we do business.
Some provocative thought on how data could transform some business
1. Auditing: No more statistical sampling. Audit the whole data as it only takes a fraction of time to process data and throw the deviations. Will auditors be responsible to report on frauds?
2. Human: Resourcing: Use intranet social network (surf through chatting trends) to formulate teams and determine effectiveness of collaboration
3. IT: Instead of reacting to hacking or security breaches how about predicting and preventing it.

What is your new career?
Leaders who took decision on hunch (due to lack of robust analytics) can now dump those hunches and rely on Big Data. What skills they need to develop? Correlate unrelated events as in the future as there are no casual occurrences. As I write this blog, MH 370 disappearance has shaken the world and this can’t be treated as casual or one off occurrence. Big Data is certainly going to take over aviation and military space (countries will now be more willing to share data in this regard to prevent such happening).
If you are young and want to build career, think about learning machine language ( Coursera offers some good on line course for free). As someone said ‘Data Scientist” will be the sexiest job of the future.

Dessert
One blog is not sufficient to write on Big Data and I will be writing more on this and hence watch out in this space. I bid you good bye saying “Data is always predictive” and of course “life is beautiful”.

Advertisements

%d bloggers like this: