Posted tagged ‘big data’

My Experiences -Last time (MELT) – 0214- “In God we trust. All others must bring data”

March 17, 2014

If you are reading my blog for the first time, please read my previous ( MELT 0114) blog to get the context and continuity. Let me start my blog with a famous quote “In God we trust. All others must bring data”
Elementary Mr Suresh your topic for next blog is “Big Data” This was the answer I was expecting. Unfortunately no one really predicted this although some were close but not on target (sic).

We did not target your daughter but future moms
Last week I wrote about how a father got upset on Target for sending e-mail promotion for baby cloths to his 16-year-old daughter. “The father visited the store again within a week and met the manager. Here is what transpired
“I had a talk with my daughter,” he said. “It turns out there’s been some activities in my house I haven’t been completely aware of.” He took a deep breath. “She’s due in August. I owe you an apology.”
In some 600 hospitals across the United States, new mothers get gifts from the various companies. Those companies have started a separate division to focus on parents of infants. Procter & Gamble sends gifts to new moms at the hospitals where they deliver the child.
Target thought it was too late to approach in the maternity ward and wanted to predict future moms through their buying patterns. They asked Andrew Pole, a data expert to develop pregnancy prediction algorithms based on certain trends such as buying lotions, unscented soaps, vitamins etc.
Here is the sample of what those analytics did based on buying patterns
Jenny Ward, a twenty-three-year-old bought cocoa butter lotion, a purse large enough to double as a diaper bag, zinc, magnesium, and a bright blue rug? The tool predicted that there is 87 percent chance that she’s pregnant and that her delivery date is sometime in late August. Liz Alter, a thirty-five-year-old who purchased five packs of wash cloths, a bottle of “sensitive skin” laundry detergent, baggy jeans, vitamins containing DHA, and a slew of moisturizers? She’s got a 96 percent chance of pregnancy, and she’ll probably give birth in early May. Caitlin Pike, a thirty-nine-year-old in San Francisco who purchased a $ 250 stroller, but nothing else? She’s probably buying for a friend’s baby shower. Besides, her demographic data shows she got divorced two years ago. Amazing isn’t it?
Who is the big brother of the modern world?
We are undergoing a revolution in the way that information about our purchases, our conversations, our social networks, our movements, and even our physical identities are collected, stored, analyzed and used. The immense volume, diversity, velocity, and potential value of data will have profound implications on privacy, the economy, public policy and corporate strategies.
Government collects data for various reasons and so do corporates for improving their market share and profits. Social network collect data to attract advertisements. Imagine the implication where are all data are combined to start predicting and tracking your behavior. This is the big brother of modern world- BIG DATA.

Big Data- What is it
We have been using Data for analytics since ages immemorial so my readers must be wondering what the hype all about is. In the last five decades data have been used for decision making in various terms such as decision support, executive support, online analytical processing, business intelligence and analytics. Typical characteristics of data analytics in the past have been based on sampling, trends and/ or structured data (most often static). Inability to process data faster and judicious allocation of budgets for analytics were limiting factor in leaping to “Big Data” world. . We often heard “Data Costs Money”.
Netflix, Amazon, Google and others have proved through robust analytics that Data costs money but also could make tons of money. Target retail chain in this blog is a classic example of using data to enhance sales and market share. Amazon tries to predict your next purchase based on your shopping history, and gives their recommendation. They are using data although voluminous but mostly generated internally. This is where the shift is happening.
Big Data is much more than this. Typical characteristics of big data are as follows:
1. Large volumes, unstructured, continuous and in multiple formats (texts, pictures, voice)
2. Move away from sample and use whole (all) Data
3. Accept messiness for the reasons mentioned in one and two above
4. More importantly to accept correlation
5. Multiple location of data
Big data comprise of 100 terabytes to petabytes as compared to few terabytes or less in traditional analytics. To give you a feel, if we have to store 100 terabytes in any digital format, it will occupy every inch of the tallest building in the world (somebody prove me wrong here).

What will you do with the big data ? Torture the data until they confess
Ronald Coase said” If you torture the data long enough, it will confess”
An abortion lead to drop in crime rate was the conclusion drawn by the authors of Freakonomics but by a sheer coincidence and not by structured correlation. Now through Big Data we have a tool and methodology to look for other correlation for example drop in prices of condoms or arrival of new birth control pills.
We are living in age of data abundance and business leaders are ready to exploit this abundance and bring transformational change in the way we do business.
Some provocative thought on how data could transform some business
1. Auditing: No more statistical sampling. Audit the whole data as it only takes a fraction of time to process data and throw the deviations. Will auditors be responsible to report on frauds?
2. Human: Resourcing: Use intranet social network (surf through chatting trends) to formulate teams and determine effectiveness of collaboration
3. IT: Instead of reacting to hacking or security breaches how about predicting and preventing it.

What is your new career?
Leaders who took decision on hunch (due to lack of robust analytics) can now dump those hunches and rely on Big Data. What skills they need to develop? Correlate unrelated events as in the future as there are no casual occurrences. As I write this blog, MH 370 disappearance has shaken the world and this can’t be treated as casual or one off occurrence. Big Data is certainly going to take over aviation and military space (countries will now be more willing to share data in this regard to prevent such happening).
If you are young and want to build career, think about learning machine language ( Coursera offers some good on line course for free). As someone said ‘Data Scientist” will be the sexiest job of the future.

Dessert
One blog is not sufficient to write on Big Data and I will be writing more on this and hence watch out in this space. I bid you good bye saying “Data is always predictive” and of course “life is beautiful”.

Advertisements

My Experiences -Last time (MELT) – 0114

March 4, 2014

Here is the challenge for my readers. Kindly connect the dots between various unrelated events listed below and predict the subject of my next blog. Wear your Sherlock Holmes hat and tell me Elementary  Mr Suresh your topic for next blog is ……..

Whatever your answer might be one thing is certain that my topic of the next blog has transformational impact. Is it a clue? You may think so.

Why are you targeting my daughter?

It was business as usual in a large retail store in Minnesota and suddenly an angry man stormed into the store and wanted to see the manager. Seeing the mood of the visitor, the staff hurried him to fulfill his desire to meet the manager. As he entered the manager’s office the visitor raised his voice and thundered “My daughter got this in the e-mail. She is still in high school and you are sending coupons for baby clothes and cribs. Are you encouraging her to get pregnant?

The manager looked at the e-mail and was perplexed and was wondering how this could have happened and made a mental note to discuss with the IT folks. He then turned towards the angry father and apologized profusely and promised that these type of mistakes will never happen again.

The father still in his angry tone quipped “you may be big target store but don’t target my daughter” ( wow what a pun) and walked away.

The father visited the store again within a week and met the manager. Guess the conversation that transpired after a week.

1948 returns but is not 1984 but 2014

It was cool morning in October 1984; I was reading George Orwell novel 1984 and suddenly my father walked into my reading room and seeing 1984 book in my hand made a sarcastic observation” I never knew that 1984 was part of your syllabus for Auditing exams” and walked away. Yes I had my Chartered Accountancy final exams scheduled the next day.

I sheepishly kept the book aside and switched to Spicer and Pegler’s Practical auditing. But my thoughts were around big brother a symbolic character in the book. Is my father a big brother? Is he monitoring all my actions?

For the benefit of people who have not heard about 1984, here is the gist about the book:

The book was written in 1948 , the year that gives the book its title was still almost 40 years in the future. Some of the things Orwell imagined that would come to pass were the telescreen, a TV that observes those who are watching it, and a world consisting of three megastates rather than hundreds of countries. In the novel, the country of Eastasia apparently consists of China and its satellite nations; Eurasia is the Soviet Union; and Oceania comprises the United States, the United Kingdom, and their allies.

Winston Smith , character in the novel wrestles with oppression in Oceania, a place where the Party scrutinizes human actions with ever-watchful Big Brother. Defying a ban on individuality, Winston dares to express his thoughts in a diary and pursues a relationship with Julia. These criminal deeds bring Winston into the eye of the opposition, who then must reform the nonconformist. George Orwell’s 1984 introduced the watchwords for life without freedom: BIG BROTHER IS WATCHING YOU.”

Who is the big brother of the modern world? Keep thinking!

The tip: Abortions leads to drop in crime rate so does  broken window.

Malcolm Gladwell in his book “The Tipping point”   discusses the incredible transformation of New York in the late ’80s and early ’90s from a crime and drug ridden city to a modern, relatively safe metropolis. While crime rates went down nationwide, New York saw a massive reduction in crime. He eventually proposes that the cause was a new policy that was adapted by the administration called “The Broken Windows Theory“. It goes something along these lines: A single broken windows can have a far greater influence than one may realise. It can spread a certain attitude and create assumptions that it’s acceptable to behave in a certain way or, in other words, “ maintaining and monitoring urban environments in a well-ordered condition may stop further vandalism and escalation into more serious crime. By the way the Broken Window theory was initially formulated in 1982 by James Q Wilson and George L Kelling.

No one predicted it. Everyone thought that high crime rates were a permanent feature of urban life. And the standard arguments to explain why crime falls don’t seem to work in this case. Levitt and Dubner ( author of Freakonomics) went through all the usual explanations for crime decreases—a booming economy, decline in the crack trade, innovative policing strategies, tougher gun laws, aging of the population—and find only two that they think really matter. Putting more police on the street  and number of young men put away in prison in that same period. But neither of those two factors were sufficient to explain the full magnitude of the crime drop. There has to be something else—and their candidate for the missing explanation is the legalization of abortion.

People undergo abortions, in other words, for a reason: because they are poor, or don’t want a child, or live in an environment where it is hard to raise children. An unwanted child has a higher chance, when he or she grows up, of becoming a criminal. By removing a large number of unwanted children, legalized abortion ended up lowering the crime rate

So be the jury and decide whether it was Broken Window theory or Abortion the reason for decline in crime rate in New York back in 1990.

 

 

 

Performance Report or Minority Report ( this event has not happened but may happen)

John walks to Jim’s work place and stood behind as he continue to work without acknowledging any human presence. John taps Jim’s shoulder which evokes a freezing reaction as he did not expect John to show up at his work place. John says “congratulations Jim you are rated 5 for the performance year 2016”

Jim mumbled to John “ Sir we are still 2014 and how can you give performance rating for 2016? “John replies that “we went through your past performance and have come to this conclusion based on your potential”. John prophesized “you will execute transformation program in Japan successfully in 2016”.  In his bubbling voice said “Good stuff son” and he moved on. Jim was still in shock and was wondering could this be real. This is something similar to “Minority Report” where some is arrested before a crime is committed.

Now readers connect or correlate these events and predict my next topic of my Blog. If you have time try reading the book “Black Swan”

Dessert

Success is how high you bounce when you hit the bottom- General George S Patton. With those inspiring thoughts I bid you good bye. As always “life is beautiful”.


%d bloggers like this: