From Web Pages to Insight: How Web Harvesting is Used to Combat Pharmaceutical Fraud

The Pharmaceutical Fraud Challenge

Illegal online pharmaceutical sales contribute to billions of dollars in lost revenues to pharmaceutical companies each year. More damaging than lost revenues is the potential for lost lives from consumers taking counterfeit and fraudulent drugs ordered online.

In this walk-through, we uncover how harvesting technology can be used to harvest, curate, and then develop insights into the world of online pharmaceutical sales.

Identifying and Harvesting Websites

The first step in tackling the sale of online pharmacies was creating a system that can identify and detect websites that sell drugs online. To do this, we asked ourselves how current people are finding the websites and making purchases. We used the answers to those questions to automate the process of finding new domains. Some of the techniques we implemented were:

Automating queries into search engines
Monitoring spam e-mail lists of known spamming domains

In addition to this reactive approach which found established domains, we also monitored domain registration lists of all .COM domains that were newly registered in real-time. This meant we were identifying domains, even before customers were visiting the sites. Approximately 100,000 domains are registered on a daily basis and BrightPlanet monitored those domain registrations based on indicators in the domain name such as RX and drug names (FreeViagra.com, CanadaRX.com, etc.).

Video

Click below to watch a short video explaining how we identified websites for our clients.

Curate

Once websites were identified, we further curated the harvested web pages to eventually help develop insights. This involves collecting all the text from each of the pages within the identified domains and identifying important terms on each page. We needed to identify key terms that were going to help us accomplish our goal of first uncovering which drugs were being targeted the most and second who were the biggest offenders. With those goals in mind, we extracted the following entities:

Names of drugs being sold
Dollar amounts
Dosages
E-mail addresses
Phone numbers
Payment processing / shopping cart information

Using External Data

In addition to the data harvested from the online pharmacy pages, we also paired a few external datasets for further analysis.

Alexa Ranking Data – Alexa ranking data was done on each of the identified domains to determine how popular each website was. This gave us the ability to identify and gauge which online pharmacies were the biggest threat and receiving the most traffic from customers.
WHOIS Lookup Information – WHOIS lookups were also completed on each of the domains to get information about the entity that registered the domain as well as where the web server was physically residing.

Develop Insights

Once the data was curated, we could begin developing insights from the data. We offered a few different options for drug manufacturers to ingest and analyze the harvested data.

Search Dashboard – BrightPlanet’s Search Dashboard gave drug manufacturers the ability to quickly query all harvested web pages at once, providing a Google-like interface for only online pharmacy webpages.
API Access/Report Access – API access and custom reports were given to drug manufacturers to ingest all harvested data in a structured format. This allowed drug manufacturers to ingest the harvested and curated data for their own analysis in data tools like Tableau, Tibco Spotfire, and Palantir.
Custom Data Visualization – We also created our own data visualizations to help better understand the data. A sample visualization is shown below, but these visualizations helped us understand which drugs were being targeted, from which websites, and how the websites were potentially connected.