Here’s where Big Data is headed in 2014: BigInsights

2014 is well underway. So what does the new  year mean for Big Data? BigInsights Research Director Shayum Rahim outlines 5 broad trends around data that will entrench themselves this year.

1.       The Commercialisation of Information Assets

As organisations large and small increasingly realise the enormous value of data, the latter itself becomes a highly sought after and valuable resource. For those organisations that do not have large information assets, the benefits to be gained from data analytics are lost unless they are able to acquire information externally. This is where organisations that do have information assets that have been collected by virtue of the type of businesses they are in, stand to profit from a revenue stream previously unimagined.

These organisations are the ones that collect information with each transaction - valuable data on consumer trends and behaviour - the most notable of these being credit card companies. Each transaction on a credit card generates a slice of information about the purchase, the age and gender of the consumer, where the purchase was made, the date and time of the transaction, and so on. The information created from each credit card use will throw up all kinds of data related to the cardholder. No doubt this information is already being used by the credit card companies for their own internal purposes such as marketing and risk mitigation, but the credit card companies also have an enormous market for their information assets in the form of their merchants.

A significant number of these would be small retailers and operators, who may not have the resources themselves to be able to derive insights about the performance of their products and services, let alone customers, but would be willing to pay a premium for the value added service of information analysis on the fees they already pay to the credit card companies.

Credit card companies are just one example. The model can be adapted by any commercial member organisation that owns information assets. Think of some of the online coupon companies like Groupon with the businesses that advertise with them, and the database of consumers who buy the deals. The advertisers will benefit tremendously from the information Groupon has of its customers.

Information assets, thus, is all set to become the next commodity in an increasingly data driven world.

2.       The Rise of Hadoop Applications

As Big Data skills shortage continues, many organisations will start looking at technology vendors to provide fully automated, fully integrated applications to resolve their data analytics issues. But unless you are a bank or a big financial institution that can boast billion dollar profit margins, you’re probably not going to be rushing out to buy an IBM Watson any time soon.

This, then, will give rise to a variety of Hadoop-based applications from a new breed of specialist, Big Data technology vendors, who will pose a real challenge to the larger traditional technology vendors. We’ve already seen a number of these come into existence over the last two years, but this is just the beginning.

The market will turn to new cloud based applications to solve specific business problems as the cost of on-premises appliances becomes an inhibitor. These applications will be industry and vertical specific in a return to the ‘best of breed’ rather than the ‘all-in-one’ concept.

3.       The Decline of the Relational Database

In the context of new data types and sources that continue to build the chaos that is unstructured data, the stalwart of organised, structured data of the last thirty years becomes part of the chaos and becomes unstructured itself. SQL is no longer the be all and end all of data interrogation, and is increasingly giving way to NoSQL.

This, by no means, spells an immediate death of the relational database, but it will only be one type of data that will be called on to provide some context to other more dynamic data sets within the organisation. As the Hadoop Distributed File System evolves, and the new breed of Hadoop applications comes to the fore, we will see less of a reliance on the relational database.

4.       Precision Marketing

With the abundance of data available to marketers, marketing is poised to become an exact science. The outcome of marketing campaigns can soon be predicted through a series of variables including historical, real-time and predictive data.

No dollar will be spent in vain and will exhort a near guaranteed return on its investment. Data will override all marketing efforts and the marketing meeting will resemble a war room of generals and field marshals planning a major offensive.  This, in turn, will help organisations personalise their offers to a greater extent than they already are by knowing their customers’ individual preferences, thus strengthening the brand and promoting brand loyalty.

Brands will not be the only winners.  Customers will also benefit from precision marketing through the medium of mobility and social media, being able to take advantage of offers designed specifically for them from brands they prefer.

5.       LOBs to Become More Data-centric

Big Data Analytics has found a home in the marketing department of enterprise. Social media, mobility, spatial data and predictive analysis provided a smorgasbord of information to marketers to help them gain insights into who their customers were, their buying preferences, where they come from and how and what to sell to them next.

This, in turn, gave rise to the CMO as an influencer in the organisation, a person who was found increasingly circumventing the need to involve IT in technology decisions. Yet, the CMO was only responsible for one segment of the organisation’s total operations and the concern of the other LOBs may not be his own.  With each LOB manager responsible for the performance of his own department, they will be looking at data to streamline many of their own functions and improve P&L margins. Manufacturers will be looking at data to reduce production times and reduce wastage; warehouse managers will be looking at how to manage supply and reduce inventory costs etc. Thus, LOB managers can no longer afford to rely entirely on the IT department to deliver information on a scheduled or even ad-hoc basis. They need to be able to make real-time decisions from actionable insights.

Big Data presents a whole new set of challenges to organisations. Almost every aspect of our lives is now digitised and information is available to us at our fingertips. Though data is created predominantly by individuals, it is organisations that have the responsibility of managing it, and this has brought about significant changes to the way organisations do business in what is an ever-changing landscape.

Change is nothing new to organisations, but the technology renaissance of the present era has forced change at a rate never seen before, and they will need to keep a watchful eye on current trends to be able to anticipate what comes next. This will be the key in determining the winners and losers of the technology renaissance.  

About the Author

Shayum Rahim is the Research Director at BigInsights and former Head of Australian Software Research at IDC. Shayum has previously essayed senior roles at  JD Edwards, Oracle and Microsoft. He has spoken about Big Data at several industry events and has authored several reports and articles on Big Data. He can be reached via shayum@biginsights.co and can be followed @BigInsights (Twitter).