Big Data --- From Priests to Data Scientists and Back

While the term Big Data may be the most over hyped term of 2012 there is some substance behind the buzz. From historical times to today Priests, Accountants, Statisticians, Quants, and Data Scientists have been given access to the most critical and sensitive data from churches and businesses. In the past few years it seems there is a trend back towards High Priest status.

image

For those that haven’t paid that much attention Big Data has been around for a long time – very much like the concept of Cloud Computing. Which has also been around for many years.

What’s Changed? 

  • Access to the data
  • The tools for capturing, modeling and displaying have only recently come to the masses.

image

Historically, from ancient Egyptian times even through the middle ages and to some extent it’s still true today that only the “chosen ones” were given access to the mission critical information. In the past the Priests controlled the information as they were allowed to speak to the gods on behalf of the Pharaohs. Later and in other societies this task was handed down to Monks and Clerics. In more modern times the task and privilege of access to the sacred numbers and words was assigned to Accountants and more recently to Statisticians, Quants and to the latest buzzword title “Data Scientists

If you were paying attention over the past few years to the book and subsequent Brad Pitt movie Moneyball and more recently to the Presidential Election in the USA there was a relatively new face in the crowd. That was the face of Nate Silver. He was the brains behind the FiveThirtyEight blog and was generally dismissed by many of the pundits. He proved all of them wrong by using data and his incisive mental skills to slice through the data and build accurate models of how the election worked (and previously how to win at baseball).

The High Priest is in The House

Nate Silver might just be the new definition of the high priest of data and the slicing and dicing of this massive flow of data we have taken to calling Big Data. Nate Silver is defining a new type of role … one that morphs the functions of Priests, Accountants, Statisticians, Quants and Data Scientists all into one.

imageWe are back to the times when the critical and sensitive data is being offered up to these High Priests for tweaking, transforming and generally flipping data around in as many ways as they can think of to try and extract some nuance of information that might have otherwise been missed. There is a lot of value in this effort and there will be a lot of jobs for doing the digging, interpreting the results and creating new business opportunities in the coming years.

Here’s a Hint: If you are looking for a job in 2013 … Think Big Data

If you have skills in Records Management, Statistics, Programming, Risk Management and/or skills in Marketing, Sales and Business Development … you should think bout playing a role in the Rise of Big Data.

So, next time you are cruising down the street and you hear this … Make Way! I’m a Data Scientist! Perhaps you’ll be able to join the parade. This trend is going to go on for a while and there will be skills needed for everything from the traditional math and science oriented aspects of Big Data Analysis. There will also be significant opportunities for marketeers and business people to put it all together in a cohesive and useful package.

I’m calling on you Millennials. I’m looking for your skills to create powerful, creative and useful ways to capture, manage and display this Big Data. I’m not discounting the Gen X crowd and I’m especially not discounting the Baby Boomers.

With 10,000 Baby Boomers retiring every day for the next 19 years … that’s 69 million people reaching retirement age in less than 20 years.

Gen X’ers and Baby Boomers are not going to go into this lightly. I suspect there will be numerous businesses popping up to help government agencies and private businesses manage their Big Data overload.

I suspect we’ll see a few more High Priests of Big Data crop up in the coming weeks, months and years. Who will be the next Nate Silver? I‘m sure they are already out there diving into piles of data and coming up with data gems every day.

What do you think?

  • Are Data Scientists the new High Priests?
  • Is Big Data here to stay?
  • What will you build for your first Big Data app?

Drop me a note or comment here. I do want to know what you think and to start a dialogue. Who knows … maybe we’ll even start a company?

My Offer and Request to you:

I have a few more posts on this Big Data series in my head. If you are interested in collaborating on a post about Big Data, Cloud Computing or Social Computing drop me a line.

clip_image001Jeff Shuey is a veteran in the Enterprise Content Management industry. Over the past 20 years he has worked with customers and partners to design, develop and deploy solutions around the world. Jeff is currently the Director of Strategic Alliances at Winshuttle. He has worked for Microsoft, FileNet, K2, Captaris, Open Text, Kofax and Kodak. He speaks and blogs about ECM and the Intersection between Social, Mobile and Cloud Computing.

Follow me on Twitter, check my blog, send email or find me on Facebook, LinkedIn or Google+

Comments