2014 Big Data Prediction #2: Big Data Will Go External

In 2013, people were beginning to understand the true potential of large data sets. We began to see companies that were successfully using internal proprietary data sets to help increase revenue.

Retailers were increasing their target marketing utilizing data from current customers, car manufacturers relied on sensor data to report potential issues with vehicles, and insurance companies utilized structured customer information to increase the efficiency of underwriting; making pricing for policies much more accurate.

Based on this trend in 2013, we have pulled together our second Big Data prediction for 2014 in this post.

Big Data Image 2As companies begin to grasp and continue to work with data internally, a point is around the corner where the focus will shift to external Big Data. This leads to our second prediction; companies will begin relying on external data sets to help augment and supplement data they don’t currently have or doesn’t currently exist internally.

Where are companies going to find that data?

Larger organizations looking for external data will likely lean on the expertise of data service providers to get that data (Data-as-a-Service [DaaS] is the next big thing). We are already seeing innovative companies across multiple industries (big pharma manufacturers, risk management professionals, financial services, and insurance providers) begin to turn to external data for their own projects.

One specific industry that we work with seems to be ahead of the competition in almost all areas in the use of external data sets to assess risk, increase efficiency, and build revenue; insurance. You’ll find a few examples of how some insurance providers have utilized external data sets in the following sections.


Insurance companies are tasked with the challenge of identifying risk for their clients and often rely on large amounts of data to assist in the qualification of customers for their policies.  For many companies in the United States, data freely exists that can be obtained about individuals through credit scores and arrest look-ups.

However, “for almost three-quarters of the world population, there is no reliable information on creditworthiness”, explains Alexander Graubner-Muller, a co-founder of a German start-up attempting to develop a social score for this lack of data.

Companies are looking at social media and other external sources to begin augmenting creditworthiness for underwriting. Some areas that we have begun to gather data for insurance providers in are:

  1. Social Media Data – Do they have a profile, and what are they saying on it? Who are they connected to on LinkedIn?  What level within companies are they connected to (Vice Presidents, Directors, Entry Level)?
  2. Local News Data – Have they been mentioned in the local news much? Does a lot of crime happen in their neighborhood? What life events have been mentioned?

Claims Management

In a previous blog post, we talked about how one insurance provider is using social media data collected by BrightPlanet to help triage claims. We are finding additional uses for external data in claims management:

  1. Online E-Commerce Data – E-commerce data allows for automated collection and aggregation of multiple price points for items that need replacing for an insurance claim, whether it’s a specific car part or a rare collectible figurine.
  2. Social Media – Social media has been a major area of focus for insurance providers as they are increasingly turning to it to assist in claims management.  Watch for more and more insurance companies turn to social media to identify witnesses to accidents and validate worker’s compensation claims.

The Power of External Big Data

What we are finding is that external data sets are being used to completely augment data that people thought didn’t exist or was impossible to obtain.

Watch in 2014 for substantially more groups looking for external data sets to augment their internal data sets in more increasingly creative ways greatly raising the bar with the level of sophistication required.

Stay tuned next week for our third and final Big Data prediction for 2014. If you missed our first, catch up here.



Photo: kentbye