How to Collect Competitive Intelligence Without Becoming Competitive Intelligence

Companies big and small seek competitive intelligence on the biggest threats to their business.

Competitive intelligence collection typically involves harvesting large amounts of unstructured data from competitor’s websites. But what most collectors of competitive intelligence don’t know, is that by doing this, you yourself become competitive intelligence.

In today’s post, we uncover the importance of anonymization when collecting data.

Becoming Competitive Intelligence

Companies employ all sorts of technologies that track their visitors’ activity: what they search on the site, which pages they visit, for how long, and how often they come back to revisit the site. This information is all stored and easily accessible by the website’s owners.

If you fill out any Web forms with your name on the site, all your previous activity on the site can be linked to that current session. Even if you don’t fill out the form, the ISP that is making the request often is displayed. For most of us, this means the business name that we’re researching from is displayed.

That’s how easy it is to become competitive intelligence when you really set out to search for competitive intelligence on a rival’s website.

We’ve generated an example using college and university data shown on the right. The data displayed from these higher ed Web servers doesn’t give us individual visitor names, but it gives us an insight into what the visitors were looking for. For example, the Dartmouth visits were mostly directed to a blog post on the differences between Surface Web and the Deep Web, while, the Wayne State College visits were looking at the BrightPlanet careers page.

It’s not difficult to see how your visits and data collection on competitor websites quickly allows you to become competitive intelligence yourself.

Something people rarely think of, though, is opening yourself up for competitive intelligence on sites related to your secret projects. Security analysts and officers turn to search engines, social media, paste sites, and forums to search for mentions of secretive projects they don’t want leaked. Ironically, in the attempt to protect and track their biggest secrets, searching directly on these sites now ties their company servers to keywords and project names they wanted to keep secret.

Safely Harvesting Competitive Intelligence

The question remains then: how do companies protect themselves and make their data collection processes and standards more efficient? The answer lies in the concept of anonymization.

Put simply, anonymization is the process of concealing (or anonymizing) your personal identifying information online. The most common way that companies use anonymization is through the use of a proxy.

BrightPlanet deploys anonymization and rotating IP addresses in their data harvest by using the AWS Cloud (Amazon Web Services). The proxy acts as a buffer in between the Web server and your company server to help conceal your identity.

A free option for groups doing small scale data collection is installing the TOR browser and operating all your competitive intelligence through the anonymous TOR network. Even with the ability to anonymize; however, the true value in collecting Web data comes from being able to scale and automate the process of Web collection.

Start Anonymizing Today

Want to learn how BrightPlanet can help you not only anonymize your data collection, but also automate that process?

Request a demo of our Data-as-a-Service solution below.

//