Using Big Data from the Web to Improve the Underwriting Process

Six months have passed since we made our Big Industry Threats dataset, also known as BITS, available to customers. BITS, our flagship dataset, contains a collection of over 11 million news stories from the past 18 months spanning 9,000 global sources with over 93 languages. Our customers found a number of different uses for our BITS dataset including risk managers tracking disease outbreaks and insurance professionals improving the underwriting process.

In today’s post, we’re going to explain how we helped a chief actuary for a major insurance company improve her data to help increase efficiency and accuracy of pricing homeowners insurance policies for customers overseas.

The Data Needed to Price an Insurance Policy

We were approached by a chief actuary for a major insurance organization that wanted additional data to further increase underwriting efficiency. Don’t worry if you are confused by what underwriting means – put simply the chief actuary wanted additional data to help determine how qualified specific customers were for homeowners insurance as well as more accurate data to help better price homeowners insurance.

A tremendous amount of data goes into determining the price of your insurance, whether that’s homeowners, car, or health.  We learned that for determining the appropriate price for homeowners insurance for one policy, this insurance company utilizes over 1,000 datasets for U.S.-based customers.

Datasets help accurately price and qualify customers by providing statistics such as crime rates in the neighborhood, frequency of severe weather, and credit standing and occupation. The chief actuary explained that they were already leveraging this data in the US, but were not able to identify any providers of data in countries outside of the United States.

International Data

They found that providers of data were more difficult to identify in countries such as Portugal, Poland, and Turkey. This lack of data stemmed mostly from the fact that crime data wasn’t provided by police forces as it commonly is in the United States.

This lack of data forced the actuary and her staff to simply guess on crime rates to help with pricing policies instead of relying on actual data to support pricing.  To help this insurance company better understand crime, we helped create our own crime dataset using BITS data.

We worked on developing a crime to location entity to track where crime was occurring most often within cities. We tagged on mentions of specific crimes and leveraged Rosoka Software Solutions to tag the entities and assist with relationship mapping to identify crime outbreaks by location. An example of what we extracted from the data is shown below.


We then delivered location-based data of where crime was occurring directly to the insurance company. They ingested the data and used it as an additional data source to help determine how much crime was occurring in the neighborhood of the specific homes they were trying to price insurance. The pricing was more efficient and more accurate resulting in the company’s ability to provide the right policy at the right price to their customers.

Want to see visual examples of BITS data?

We just had a webinar that features our BITS dataset, where we developed visuals to help track disease outbreaks for a mining organization. Use the link below to request access to these visualizations.