I recently met Big Data in person.
In real life.
THE BIG DATA.
Yes, I did.
And here he is …
Do you recognise him?
Maybe this will help you to remember.
Yes, Big Data.
The man, the legend, the person …
… ah, you thought I meant Big Data, the idea.
Do you know where the term Big Data came from?
According to some, the term was inspired by Laurie Anderson’s album Big Science from 1982.
However, the term actually dates back to World War II when the phrase Big Science was used to describe the rapid cycle of changes that occurred in scientific disciplines during and after World War II.
A good example of this rapid cycle of change during World War II was the invention of computing.
John W. Mauchly was a physicist working at the Moore School of Electrical Engineering at Pennsylvania when the war began, and he believed that an electronic computing device could be created to predict the weather.
The idea was floated in August 1942 and, by June 1943, funds were allocated to the project.
Unfortunately, by the time the system was developed, the war was over. Nevertheless, the system that was developed (called ENIAC, the Electronic Numerical Integrator And Computer) led to the creation of the industry I have spent all my life working with – technology and information – as well as the weather forecasting service.
Ah well, you can’t win ‘em all.
The term Big Data was first used in the late 1990s and no-one can quite put their finger on how it came about. It appears to be through a combination of academic papers and META Group (now Gartner) who say that:
“Big Data are high-volume, high-velocity, and/or high-variety information assets that require new forms of processing to enable enhanced decision making, insight discovery and process optimization.”
Yea, yea, yea … surely it’s simpler to say that Big Data is just this phenomenon we have today, where every person on the planet is creating digital records such that we are drowning in data.
This is well illustrated by Google’s CEO Eric Schmidt who stated:
“There were 5 exabytes of information created by the entire world between the dawn of civilization and 2003. Now that same amount is created every two days”
during a Techonomy debate in 2010.
An exabyte is a 1 with 18 zeroes after it – or 1,000,000,000,000,000,000 bytes – an incomprehensibly large number.
Drowning in data.
Some call it a tsunami – not an appropriate term – whilst others call it oil. It doesn’t matter what you call it, it’s a big amount and is illustrated by the amazing growth of unlimited bandwidth and unlimited storage., combined with device ubiquity and cheap chips.
Why is this important?
Because those who make sense of Big Data will win.
Data is the new source of competitive warfare, and data mining will be the weapon of mass destruction of the competition.
Many industries get this, particularly the new tech players like Amazon, Apple, Google and Facebook.
It is the reason why banks fear these players because these players know how to sift the data and make sense of it.
As one bank Chief Executive recently stated: “Our peers I can handle. We’re in the same boat. But if Google opens a bank, with all their data – then we’re in big trouble.”
And the biggest issue for a bank is one that I’ve raised many times, but it is easiest to illustrate by just repeating a little from an earlier blog.
It amazed me to see the back office of one of the largest operations in the world in action today.
Yes, I’m talking about Amazon.
A place you would think would be a seamless operation of high technology but, in reality, is a bit of a mess.
The book division doesn’t talk to the music division. The music division won’t talk with the electronics division. The retail business won’t talk with the wholesale business. The wholesale business won’t talk with the cloud business. Oh, and the Kindle division won’t even talk with the book division!
None of them will share customer information with each other and, as a result, no-one knows what customers buy from Amazon as a group, when or with what sort of payment type.
Yes, the silo divisions of banks mean that they ignore their greatest competitive weapon: data, and let others potentially gain this opportunity at their expense.
Something’s gotta change.
Meantime, if you want to hear what Big Science sounds like, have a listen …
p.s. if you didn't recognise him, the actor with me in the photograph is Brent Spiner, who plays the character Data in Star Trek.