Sunday 31 December 2017

Big Data: New Elixir, New Aether of Life!!

Few days ago one of my friends sent this forwarded joke on WhatsApp:

Me- Hello! Tony’s pizza? 
Google- No it's Google's pizza.
Me- So it's a wrong number? 
Google- No sir, Google bought Tony’s. 
Me- OK. Take my order. 
Google- Well sir, you want the usual? 
Me- The usual?
Google- According to our caller ID, the last 12 times, you ordered pizza with cheeses, sausage, thick crust.
Me- Yes! 
Google- May I suggest to you add ricotta, arugula with tomato? 
Me- No, I hate vegetables 
Google- But your cholesterol is not good
Me- How do you know this?
Google- We have the results of your blood test for the past 7 years 
Me- I already take medication. 
Google- You have not taken medicine recently. 4 months ago you only purchased 30 tablets online
Me- I bought more from another Pharmacy
Google- It's not showing on your credit card
Me- I paid in cash
Google- But you did not withdraw that much cash according to your bank statement
Me- I have another source of cash
Google- According to your Tax files you do not have another source.... unless you did not report it. 
Me- I'm hanging up. 
Google- The pizza is already at the GPS location your are calling from. Have a good day.

I passed it off as a joke at first instance, however after one deep read, I realized, It’s now a reality.

Big data and analytics are the new drivers and growth engines for all the businesses. User data collection and its analysis is a next big revolution and it is going to cause huge disruption, if not done already, in the way businesses are run, grown. 

Every click on Social Media, every search on search engine creates data; adds on to our digital profile and lets someone know about your preferences your interests your likes, your dislikes... just imagine how much data we leave behind. Gone are the days when someone used to leave footprints on the sands of time..... We now leave digiprints!!!! Gone are the days when archaeologists used to trace fossils, today the modern day archaeologists- data scavengers, those Hadoop trained analysts trace your digiprints!!!  

As it stands more than 2.7 zettabytes of data is currently floating around in today’s binary universe, and is expected to exponentially grow to 180 zettabytes in by 2025 (for the uninitiated zettabytes has 21 zeroes on 1 or 1021 bytes)

As Mark Cuban calls it; data is new gold, new oil, new plastics...... I would go even beyond and call Big Data the new Elixir, new Ether of life!! 

There are several companies, organizations use, analyze this data to generate insights that can help organizations run, grow their business. Big Data combined with analytics is a potent combination to influence user choices..

There are various data analysis techniques available which help businesses serve, understand customer better. Techniques like: 
  • Data Mining
  • Data Management
  • Text Mining
  • Psychographic profiling
are the big levers on which business houses are now spending huge amounts of money and are reaping its rewards.

The world has already seen how the public opinion can be influenced using these techniques:

Cases in point: Last Presidential elections in America- How Donald Trump influenced voters in 2016 US election (and eventually making Cambridge Analytica household name), how Brexit decision was influenced and how Narendra Modi swept the 2014 election in India… The only reason to select these 3 examples is to show the scale and impact of psychologic profiling off Big Data!!! 

I just want to put some chilling facts in front of readers:

Circa-2008. 

Michal Kosinski and David Stillwell; the students of Cambridge University, Psycometrics Centre; introduced an app on Facebook which allowed users to take a quiz to understand and share their own profiles. They got way too many responses, much higher than their expectations. Kosinski correlated the results of the psychographic profiles to the Facebook likes of the respondents…and voila- he was able to make deductions based on these correlations..... This opened the ‘Ocean’ (pun intended) of secrets in front of the world!! (Ocean = 5 personality traits on which psychographic model is based = Openness, Contentiousness, Extroversion, Agreeableness, Neuroticism)

Having analyzed the data and user profiles and Facebook likes; Kosinksi was able to assess and predict an individual's profile. Just 70 likes by the user and Kosinski knew more about the person than his friends. He could predict the skin color, sexual orientation and political affiliations with greater than 85 percent accuracy. With increasing number of likes, the profiling got even deeper and at 150 likes, he could better predict the person than the parents of the individual.... double the likes to 300 and Kolinski knew better than the partners. More likes, and Kosinski could know more about the person than the person knew about himself... phew!! What can I say.. watch your Likes mate!! Someone’s watching you...err.. profiling you..  (read more at http://www.firstpost.com/tech/news-analysis/big-data-and-psychographic-profiling-helped-donald-trump-win-the-us-presidential-election-3696847.html)  

These are not the only areas where big data and analytics play a role…. the range is really vast and varied


  1. Hospitality:
  • Creating customized guest experience: 
Just imagine after tiresome travel and grueling business meeting, when you check-in into the hotel; your personal favorite welcome drink is served to you by hotel even without you asking it; or as you enter the room you can smell the aroma that too just at the right note- too mild not strong; of your favorite room freshener you will have that relaxing, soothing feeling. When you enter the shower bath and open the faucet you realize that the water temperature is exactly set as you would have liked to!!!!! You would feel as if you are a Sultan of Brunei!!
  • Increasing Business Profitability:
Business houses are consuming user data big time. One such case is of Finnair. Using passenger volume, demand data it can model its network, anticipate future passenger flow, predict market share by route, and analyze route-specific and market-specific performance in comparison to competitors. Analytics also helps the company price and market routes and services. (read more at: https://www.sas.com/en_us/customers/finnair.html)

    2. Healthcare:
  • Better Predicting Health outcomes:
By analyzing huge volumes of structured and unstructured clinical and operational data; the various analytics tools can uncover hidden insights on indications and then turn that insight into evidence-based knowledge that can help healthcare professionals predict and improve outcomes.

It also helps doctors to make big data-informed decisions within seconds and improve patients’ treatment. This is particularly useful in case of patients with complex medical histories, suffering from multiple conditions.

  3. Retail:

Based on a customer’s purchase history, retailers can predict what the customer is likely to purchase next. Machine learning models are trained in analyzing historical data which allows the retailer to generate accurate recommendations, helps in forecasting trends, Optimizing product pricing.

Predictive data searches on Google, predictive typing are day to day examples of how user data is used for enhancing customer experience. When I first encountered the predictive typing or predictive search- I was clueless, surprised, happy all at the same time.... so naive I was….I didn’t know I had left my own digitrace; every time when I searched something or typed!!! Another recent example is Predictive Email Responses- a feature added by Gmail. It shows the use of data gathered from user responses and utilization of deep machine learning.

As the world gets increasingly Facebooked and Googled and Ubered and Amazoned, we leave more and more data about ourselves in this binary world. The Data Tide, the Data Waves can’t be abated.

Such data waves are making users empowered and vulnerable at the same time. While I am not undermining the huge positive impact already created by Big Data and the promise it holds for future but the current and gradually increasing thereat of Privacy or the lack it poses some serious questions. “Are we Socially secure in this Social Media profiling days” a question of gigantic proportion has been posed to us by Big Data, Data Analytics; very sadly the answer to this question is overwhelmingly No. It’s a classic case of Data, Data everywhere, not a place to hide!!!!

P.S: while I searched for data for this article my digital profiling had been updated. I immediately started to get ads related to Big Data training on Hadoop. Machine learning is an abyss!! machine learning is relentless, tireless and continual process. While you are reading this blog post, data centers located in remote areas have updated your profile and you have opened a new digi-thread for the machines to explore…


Brace yourself for the topsy-turvy Big Data ride in years to come!! A whole new ecosystem, a whole new economy thriving on Big Data, Data Analytics is going to open new vistas to us. Only time will tell how!!!!


Saturday 31 December 2016

My; 'The Terminal'; moment

On one lazy Sunday afternoon in demonetized India (yeah... I can say lazy… because even in demonetized India, banks have Sunday off!!!), for umpteenth time, I was watching a highly entertaining yet soulful movie ‘The Terminal’. While Viktor Navorski was going through those helpless situations in the movie, I happened to receive a message from one of my friends. He was super irritated because his visit to Germany had to be postponed as Lufthansa had canceled 1,800 flights because of a strike by the German airline's pilots over pay. 

As soon as I read that message I had a sense of Déjà vu... I had been Viktor Navorski myself because of Lufthansa strike and as cashless as I am now (off course due to demonetization!!!!!). Strangely enough these three (The Terminal, Lufthansa Airlines strike and 'Cashless India') mutually exclusive events were very much intertwined in my ‘The Terminal’ moment..... Akin to PIP TV, while I was watching Terminal, another movie started to roll in my mind....

Scene 1: Lufthansa Strike and 'Being Cashless'

Turning back few personal memory pages... here's how the events unfolded:

Circa 2008.... I was to attend a 2 week training in our Buenos Aires office. So began the visa activities... one of the major deviation in this visa process was the requirement of a recommendation letter from State Chamber of Commerce.. Luckily enough our company was already an associate member... so getting that letter was whole lot simpler than I expected and it was one of the few activities that were completed smoothly and effortlessly as far as my entire LatAm Sojourn was concerned...

So then...... Visa photos…Tick; Visa Form and compiling related documents....Tick....; the checklist of requirements was getting completed at a fast clip... I was all set for dossier submission to Argentina Embassy. Soon after all the docs were submitted and I was waiting for the visa to be granted. Few days after, I received a call from travel agent and while I hoped that he would give me a good news of my Argentina visa being stamped, he informed me that I will need to apply for Schengen Visa as well since I was traveling via Europe, my pitstop being Frankfurt Airport... so began the activities for Schengen activities... within 2-3 days, documents of Schengen visa were submitted. Finally after a gap of few anxious days.. I received both Schengen and Argentina visa duly stamped on my passport... I was elated and looking for my first trip to any Lat Am country.

But a certain Mr. Murphy standing at a distant corner was laughing and waiting for his turn to propel me into the realms of uncertainty!!!!!!

As the travel day was approaching, the first Murphy salvo was fired!!! Lufthansa staff decided to go on strike!!! The travel agent informed me that the airlines is forking out an arrangement with another airlines..... So much for their customer centricity, I thought!!!!

My LatAm journey had 3 travel sectors: Ahmedabad to Mumbai, Mumbai to Frankfurt and finally from Frankfurt to Buenos Aires. There was huge uncertainty about travel leg from Mumbai, as reticent Lufthansa staff was in no mood to roll back the strike. I still didn't have confirmed tickets, Passport with stamped visas for BA travel in my hand nevertheless I decided to start my journey from Ahmedabad.... As I landed in Mumbai and switched on my cell I saw 12 missed calls from the travel agent... I hoped that there is some good news in offing... Without even waiting to collect my bags I dialed for the travel agent.

“Sir, we have confirmed tickets for your travel to BA... (Sigh of relief form me), but Sir there's one problem, since we had Schengen visa we had to search for the route via Europe and the only route available is via Charles de Gaulle (CDG), Paris by Air France"

“I am fine with it... as long as we have confirmed booking"

“Yes Sir, we have already mailed you the itinerary and tickets few minutes back. But there's one problem... (Mr Murphy was in his elements I thought)... You land at CDG 7 am and your onward flight to BA is at 10 pm from CDG"

“I can roam around Paris city before I board my next flight, right?"

Again there was huge uncertainty whether I could roam around Paris city with just transit visa, as per my travel agent I needed Tourist Visa!!!!! Mr Murphy was shooting at will and has a license to kill; I thought...

Now that it was certain that finally I am flying to Argentina, another reality dawned on me!!! Because of such prevailing chaos, I was advised not to purchase any foreign currency for travel. So then, I had no foreign currency in my hand and I was traveling several thousands of kilometers. It was a case of Bravado or Foolhardiness!!! I can't choose between the two!!!!

Scene 2: The Terminal Moment:

"Ladies and gentlemen, please fasten your seatbelts and secure all baggage underneath your seat or in the overhead compartments. Ensure that your seats and table trays are in the upright position for take-off. Please turn off all personal electronic devices" instructed the Air France airhostess.... after all the chaos and uncertainty these words sounded like music to my ears as it was assuring that I am headed to CDG....one more step closer to final destination!!!

As the adrenaline surge returned to normalcy, I started to think about the options that I could trade after my landing at CDG.

Just like Viktor Navorski, I had to explain the entire situation to French Airport Authorities in an alien language (English!!???). He had inability to communicate in English and I was challenged in French. My previous experience of asking direction of Eiffel Tower in English to French man was so great (??!!) that ultimately I had to play dumb charades with that French guy. I later realized that what I was pronouncing as 'I Fell Tower', he knew it as ‘Tour 'Ee fell'... so keeping in mind this experience and uncertain moments I had gone through, I had decided to stay put at CDG airport.

Once I landed at CDG, I collected my luggage and from Arrival lounge I was ferried by airport bus to Departure terminal, I searched for a nice runway facing recliner seat and I parked myself there. My journey of being Viktor Navorski had just begun.... The Terminal Moment had finally arrived..

Watching zillions of planes taking off and landing at CDG, calmed my thumping heart and finally I started to believe that after this stopover of 15 hrs, I will be in BA; this feeling was really assuring, atleast they are not On Strike!!

After I reenergized myself with some great homemade food that I had carried with myself, I decided to explore CDG. I saw a bunch some happy and some very active travelers, bang in the middle of Terminal. As I moved towards that active area, I realized that it was 'Stay Connected Zone'!!!! For few Euros there was an opportunity to connect to world even though for some limited time... Just to put few things in perspective, I had no foreign exchange on me, I could not activate my International roaming and the smartest smartphone generally doled out in corporate India in those days was Blackberry Curve!!! iPhone was a luxury and style statement, whereas the Samsungs, the Micromaxs, the Oppos and the Vivos of the current world, were not the household names they are now, India still being a monetized economy free flowing currency notes was the preferred way of moeny transaction; digital wallets existed only in sci-fi movies.....so seeing that ‘Stay Connected Zone’ I was as happy as a kid in a candy shop!!! At that instant I fully realized what Mastercard, very proudly proclaimed in their ‘Priceless’ ad campaign…. "There are some things money can't buy. For everything else, there's MasterCard" as I could buy for myself the time slot on CDG airport even when I was completely broke and could connect to the world.

..... First I mailed my near and dear ones back in India that I could manage to reach this far... second mail was naturally sent to my office colleagues to let them know I am finally on my way.

Once that time slot was over, I turned into Viktor Navorski again!!! I decided to do some shop hopping and first in pecking order was…. treating myself with some aromatherapy!!! I entered into 'Body and Perfume Shops' and started testing all the perfumes available on the sale. When I finally exited from the shop, I had successfully managed to knock off 1 hour from my ‘15 Terminal hours’!! Another funny thought crossed my mind, I had tested so many perfumes, colognes during this self-help aromatherapy session that I felt I was like this big potpourri dried flower smelling of Armani, Hugo Boss, Grey Fennel, Versace!!!!

Buoyed by this first experience I set my eyes on luxurious and some uber luxurious accessories from the fashion houses of Bulgari, Mont Blanc, Hermès, Lois Vitton, Cartier, Swarovski. After this 1.5 hrs of fashion binge; I stepped out with refreshed mind and famished body!!!

After a session of serious gluttony, I was raring to go on my CDG expedition. Had another window shopping round of Relay and other accessories stores.

It was around evening now, the Sun was setting on Paris and so was my energy level. I wasn't the same enthusiastic CDG explorer that I was few hours ago!! So I posited myself on a nice tarmac facing recliner and started playing Solitaire (seriously???!!!) on my Laptop!!!! I could see many flights taking off/landing and few others plying across various parking bays. It was really fascinating to watch such a high number of flights controlled so seamlessly 24*7 by Air Traffic Control.

Zzzzzz...snoozed my phone and I awoke from the slumber. I didn't even realize when I fell asleep on that recliner!!! It was almost the time for my final flight of the journey that started from Ahmedabad.

"Welcome to Buenos Aires International Airport" when I heard these words, I couldn't believe that I had finally reached BA!!!

Elated by this new status- ‘Landed in BA’, I collected my bags from belt, seated myself in the cab that was already waiting for me and began an absolute last leg of my final destination: cozy and secure environs of my room!!! As soon as I reached in room, I just dropped my bags and collapsed on the bed and woke up next morning completely rejuvenated.

…..by the time I had reached last frame of my own PIP movie; I realized that Viktor has arrived in New York at the hotel where Benny Golson is performing, and he could finally collect the last autograph. "I am going home" says very satisfied Viktor as he steps into a taxi and places the last signature into the can; I thought my reaction would've been similar, "I am at home" when I woke up rejuvenated next morning.

Visit to Floralis Generica, watching Tango in the land of it's origin, visiting Eva Peron's tomb; were the perfect sweet fruits of labor; that I had endured to reach Buenos Aires.






P.S.: The Terminal is partially inspired by the 18-year stay of Mehran Karimi Nasseri in Terminal 1 of Charles de Gaulle International AirportParisFrance, from 1988 to 2006!!!!! What a coincidence!!!


Happy New Year