Drowning in Facts, Thirsting for Answers

As corporate business intelligence efforts become more mature, executives are finding that bigger and faster are not necessarily better.

By Evan Schuman

To companies trying to understand and predict customer behavior, the Web promised the kind of information of which marketing dreams are made.

The Web can electronically track every customer's action and make some very good guesses about their thoughts. For instance, are buyers just looking at products that are blue and green? How many seconds did they spend looking at a particular product? Which competitor did they visit right before they came to us? What does a five-year history of a particular customer's purchases look like?

Over the last few years, companies have become remarkably effective at grabbing every single piece of information available. And that's not merely for Web sites. Today's technology allows customer and prospect interactions via company kiosks, toll-free numbers, in-store visits and dozens of other methods to be tracked and categorized as well as merged and contrasted with both historical files and files purchased from others.

For many veteran marketers, such a mass of data is an answer to long-held wishes. Or is it? Many businesses today are discovering what seems to be a contradiction: gathering too much data about customers, distributors and inventory can—in the short term—make it much more difficult to quickly and cost-effectively obtain useful answers.

 

Where Does BI Run Into Trouble?

* IT, business manager disconnects

* Too much of the wrong data

* Insufficient data sharing between units

* Awkward and unfriendly GUIs

* Lack of clear business objectives for the data

* Training

Much of that customer analysis fits into the technology category of customer relationship management (CRM). But CRM is just one element of the much broader business intelligence (BI) category. BI programs try to gather and make sense of all the information within a company. This includes not only customer information but also the experience and knowledge within the heads and file cabinets of all employees and contractors, known as knowledge management (KM), as well as competitor intelligence.

The market for such software is undeniably huge. International Data Corp. (IDC) saw $4 billion spent on BI products in 2002, and the industry analyst firm sees that figure almost doubling—to $7.5 billion—by 2006.

"It's not the data's fault. It's the internal politics, inefficient
decision-making processes and inefficient
use of tools." —IDC Research Manager Dan Vesset

Many companies define business intelligence differently, but BI is generally accepted to refer to efforts to analyze all of a company's information—wherever it resides—and to use it to help make strategic and tactical business decisions. BI software is intended to automate that process and help with the intelligent manipulation of what might be trillions of pieces of information, with more pouring in every hour.

That's the rub. The more information companies can pour into their KM packages, the more difficult, time-consuming, inefficient and expensive it becomes to extract meaningful data. E-commerce, CRM and other tools—along with sharply lower storage costs—are what psychologists might refer to as enablers. They make it frighteningly easy for companies to gather far more information than they might need.

With many businesses frustrated at BI projects that fail to deliver ambitious internal goals, IDC Research Manager Dan Vesset cautions executives to not blame the hapless data.

"It's not the data's fault. It's the internal politics, inefficient decision-making processes and inefficient use of tools," Vesset said. "Don't stop collecting data because it's an irretrievable asset."

Ruth Stevens, an adjunct business professor at both New York University and Columbia University, sees the unlimited collection of data as potentially troublesome. "Companies are frequently awash in data and unable to turn it into intelligence. The problem lies in a failure to clarify objectives in advance. Establishing why we are gathering the data and what business results we can get from it—this is the hard work of marketing strategy," she said. "Sometimes it's just easier to say 'Let's collect everything and figure out later what to do with it,' which is a recipe for analysis paralysis."

With the costs involved in gathering this newly available data so high, this waste is especially regrettable. "Data is expensive to gather and maintain," Stevens said. "Its purpose needs to be established up front. With a clear picture of what you want, and how you will use it to increase the value of the business, then data can become a huge source of competitive advantage."

"Companies are frequently awash in data and unable to turn
it into intelligence." —Adjunct Professor Ruth Stevens

Part of the problem stems from the abundance of informational riches that e-commerce, Web sites, CRM programs and sophisticated payment systems have made available to business managers. Then there are the sharply falling storage costs. Taken together, these key elements make it more practical to store much of that new data.

"The IT shop usually has a kind of packrat mentality, were they want to collect data by default," said Adam Kornegay, an informational intelligence consultant with credit-reporting bureau Experian Information. "'If storage is cheap, let's go ahead and collect it.' [Unfortunately,] the IT unit generally has no sense of how the line-of-business manager needs to use that data."

Added Columbia's Stevens: "Web logs are a tool of the devil to distract us from the real opportunity in our relationships with our online customers."

Although one way to address this problem is to be pickier about the data collected, it can be difficult to decide what data ultimately will yield valuable information and what will not.

"In some large organizations, it's hard to know what information is important and what isn't," said Ralph Welborn, managing director of BI vendor BearingPoint. "And so you need a mechanism for finding out what's there and what's not. That's the real challenge of life in the 'real-time' enterprise."

"The IT shop usually has a kind of packrat mentality,
were they want to collect data by default." —Experian Information
Informational Intelligence Consultant Adam Kornegay

The problem can intensify when IT starts collecting data without a sophisticated sense of what the business needs and when business managers start requesting information without an understanding of what that request involves.

"The business user doesn't ask questions in SQL code. He doesn't say, 'I want to know this, so do this kind of a join,'" Kornegay said. "Doing database queries and understanding how a business works are two very different skill sets."

Guy Bonaldo, the director of financial systems and analysis at Staples, learned that lesson shortly after he started working with the retail giant's data warehouse. Bonaldo tried a two-pronged approach to bridging the business-to-IT data warehouse gap. It involved both high-end training (for hundreds of employees a year, in many cases mandatory) and a simpler interface.

Delivering the easier frontdoor to the data warehouse was achieved by building it out of off-the-shelf spreadsheet product Microsoft Excel—with which most of his users were familiar—as well as creating a wide range of standard research templates.

"The fact is that we don't have to worry about today's sales today.
It's sufficient to worry about today's sales tomorrow." —Burlington Coat Factory
Database Systems Administrator Gerry Claggett

Toyota also discovered that bridging the gap between IT and business meant hiding as much of the data's complexity as possible and—at the same time—making it easier for managers to see patterns and connections within the data. Software analytical tools can only go so far in spotting business trends without human intervention.

"At Toyota, we work really hard to try and put our hands around the data," said Mike Burkes, a data access manager at Toyota Motor Sales USA. To do that, the company has been working with a dashboard-like interface. But Toyota has found the efforts to bring business managers and IT database experts essential. Before, said Burkes, "IT was trying to deploy something that the business didn't ask for."

Other approaches, such as the real-time creation of colorful infographics, are available to more easily convert the massive amounts of data into actionable information. "Most business organizations have spent at least 20 years setting up systems to capture data about their activities. Today, businesses are drowning in data, ranging from supplier inventories to Web site traffic," said Corda Technologies CEO Neal Williams.

"Part of this failure is the tacit assumption by business IT managers and analysts that the best way to understand numbers is to look at numbers. Though numbers represent the data, they may not tell the best story to those who need to comprehend those numbers," said Williams. "The need is to understand the data, not merely to represent it. By formatting data into visual images, the integration of meaning and understanding can give strength and confidence to the decision maker. Information can thus be made a tool and not a distraction."

One issue companies can look at is how quickly they truly need answers. Gerry Claggett, the database systems administrator for $2.6 billion Burlington Coat Factory, concluded that the real-time data his people wanted him to gather wasn't necessary.

"The fact is that we don't have to worry about today's sales today. It's sufficient to worry about today's sales tomorrow," Claggett said. "If we can get all of Saturday into the database before we run our Sunday results, then the buyers will have a consistent view of what last week looked like."

Another critical BI implementation challenge is purely political. Many business managers are nervous about sharing information and access with others. That's especially bad news for BI, because a successful enterprisewide BI project needs to be able to pull data from practically everywhere.

"There are still information silos in place where people can't get to the data from a related area," said IDC's Vesset. "People are protecting their data for job security. To get a handle on true profitability, it's not enough to just look at sales. You also have to look at customer service. That may show you that you are spending a lot of money to service these people. Maybe you then need to see inventory and tech support report records. Why are customers waiting so long? You'll need to examine the supply chain. Maybe you need to change a certain supplier."

Knowledge is power, and corporate citizens who are understandably hesitant to share their knowledge know that fact instinctively. But hoarded knowledge won't likely do the company—or the employee—much good. Perhaps BI initiatives ultimately will be a good test for separating the companies that want business intelligence and those that want to be truly intelligent businesses.