Big Data for Broadcasters

3 questions that I want to answer here:

1. What is 'Big Data' ?
2. How big is big to justify it being labelled 'Big Data'?
3. What does Big Data mean for Broadcasters?

So what is Big Data? 

Well lets start at looking at the book of knowledge to enlighten us. 

Big Data is the term for a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications. 

From this definition we are saying we have so much data that current conventional technologies are not any use when it comes to cataloging, indexing and reviewing this data?

The book of knowledge continues... The challenges include capture, curation, storage, search, sharing, transfer, analysis and visualization. The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data, as compared to separate smaller sets with the same total amount of data, allowing correlations to be found to "spot business trends, determine quality of research, prevent diseases, combat crime, and determine real-time roadway traffic conditions.

As of 2012, limits on the size of data sets that are feasible to process in a reasonable amount of time were on the order of exabytes of data. Scientists regularly encounter limitations due to large data sets in many areas, including meteorology, genomics, connectomics, complex physics simulations.

I take the wiki extract as a summary of how Big Data came about and the challenges that certain industries and arenas face when trying to house keep this data and produce tangible trends and analysis. 

But are we saying that broadcasters are being challenged with Big Data issues? Do they really have so much data that it is to be classed as Big Data? I am not an expert, just an observer. But I highly doubt that current broadcasters face this issue or am i being incredibly short sighted, probably you tell me?? One thing is true however, broadcasters do need use data collected in a more efficient and intelligent manner.

I found this posting incredibly useful when it comes to further describing Big Data. 

Source: http://timoelliott.com/

The Original Big Data

Big Data as the three Vs: Volume, Velocity, and Variety. This is the most venerable and well-known definition, first coined by Doug Laney of Gartner over twelve years ago. Since then, many others have tried to take it to 11 with additional Vs including Validity, Veracity, Value, and Visibility.

Big Data as Technology

Why did a 12-year old term suddenly zoom into the spotlight? It wasn’t simply because we do indeed now have a lot more volume, velocity, and variety than a decade ago. Instead, it was fueled by new technology, and in particular the fast rise of open source technologies such as Hadoop and other NoSQL ways of storing and manipulating data.

The users of these new tools needed a term that differentiated them from previous technologies, and–somehow–ended up settling on the woefully inadequate term Big Data. If you go to a big data conference, you can be assured that sessions featuring relational databases–no matter how many Vs they boast–will be in the minority.

Big Data as Data Distinctions

The problem with big-data-as-technology is that (a) it’s vague enough that every vendor in the industry jumped in to claim it for themselves and (b) everybody ‘knew’ that they were supposed to elevate the debate and talk about something more business-y and useful.

Here are two good attempts to help organizations understand why Big Data now is different from mere big data in the past:

Transactions, Interactions, and Observations. 

This one is from Shaun Connolly of Hortonworks.  Transactions make up the majority of what we have collected, stored and analyzed in the past. Interactions are data that comes from things like people clicking on web pages. Observations are data collected automatically.

Process-Mediated Data, Human-Sourced Information, and Machine-Generated Data. 

This is brought to us by Barry Devlin, who co-wrote the first paper on data warehousing. It is basically the same as the above, but with clearer names.

Big Data as Signals

This is another business-y approach that divides the world by intent and timing rather than the type of data, courtesy of SAP’s Steve Lucas. The ‘old world’ is about transactions, and by the time these transactions are recorded, it’s too late to do anything about them: companies are constantly ‘managing out of the rear-view mirror’. In the ‘new world,’ companies can instead use new ‘signal’ data to anticipate what’s going to happen, and intervene to improve the situation.

Examples include tracking brand sentiment on social media (if your ‘likes’ fall off a cliff, your sales will surely follow) and predictive maintenance (complex algorithms determine when you need to replace an aircraft part, before the plane gets expensively stuck on the runway).

Big Data as Opportunity

This one is from 451 Research’s Matt Aslett and broadly defines big data as ‘analyzing data that was previously ignored because of technology limitations.’ (OK, so technically, Matt used the term ‘Dark Data’ rather than Big Data, but it’s close enough). This is my personal favorite, since I believe it lines up best with how the term is actually used in most articles and discussions.

Big Data as Metaphor

In his wonderful book The Human Face of Big Data, journalist Rick Smolan says big data is “the process of helping the planet grow a nervous system, one in which we are just another, human, type of sensor.” Deep, huh? But by the time you’ve read some of stories in the book or the mobile app, you’ll be nodding your head in agreement.

Big Data as New Term for Old Stuff

This is the laziest and most cynical use of the term, where projects that were possible using previous technology, and would have been called BI or analytics in the past have suddenly been rebaptized in a fairly blatant attempt to jump on the big data bandwagon.

How big is big then?

I think the answer here, is it depends. Or nobody actually knows how much data is required to categorize it as big data.  A rather loose and fuzzy term to associate. Secondly how is big data handled and managed? Who is currently doing big data???? I think this image sums it up for me :)

 

What does Big Data mean for Broadcasters?

With new / current broadcast delivery methods, broadcasters actually do have a a great deal of metric and analytic data combined with global and regional torrent activities (see Netflix !How they judge what content to publish). Linked with social media activity we end up with a number of data sets that need to be combined, indexed, reviewed, cultivated etc to provide signals and opportunities for broadcasters. 

In summary Big Data is the new buzz word and is a broad brush in my humble opinion. 

James