The Integration Challenge of Web Data

By Mark Price

Note: This is the final article in a two-part series focusing on portal technology. The first one is "Using Business Portals to Leverage Information"

In my previous article I described the evolution of business portals and related those to the development of consumer portals such as MyYahoo! and MyExcite!. In both cases, portals developed from generic "one size fits all" models to customized – then personalized – versions. These personalized portals seek to match specific customer preferences to new information as soon as that information becomes available. When the portal discovers new information, it is brought to the user’s desktop for easy access and viewing, making it easier for the user to find needed information quickly and efficiently. Consumer portals conduct such searches among a limited series of information sources that are located on the Web; business portals function similarly with the internal company Web sites, databases and documents.

With today’s consumer and business portals, the tools search among a limited group of information sources to find and bring back what the user has specified. However, with the introduction of intelligent agents, "Web spiders" as they are sometimes called, and the leverage of business rules to manage the information as it is discovered, the potential leverage of Web information dramatically increases. Business portals always delivered on the core promise of access to internal company information. Now, business users can also access relevant Web information and integrate that information with decision support tools to bring a new level of analysis and insight to the organization.

More Familiar than You Would Think

What is an intelligent agent, and how can it help companies bring their businesses into real time? The concept is actually easier than it looks. We use intelligent agents every day, although those agents tend to be more of the "flesh and blood" than the digital variety. For example, a realtor is an intelligent agent working on your behalf. In many cases, you meet with the agent and provide him/her with a series of business rules – how many bedrooms, price range, location parameters, style of house, etc. Once the realtor has confirmed the understanding of the business rules, you go back to work or home and go about your daily routine.

The agent, working on your behalf, begins to search all the available data sources – newspapers, electronic sources, networks with other realtors, etc. – to find a match with your business rules. Once matches are found, the agent contacts you and provides the tools to take advantage of the new data. The agent will also keep checking with you to make sure the matches truly are fitting your needs. Through this reconfirmation process, the agent is able to get more precise in the search until you find the correct match. In this example, intelligent agents simplify our lives and permit us to take advantage of information that we would not otherwise be able to access in order to make better and smarter business and personal decisions.

The Agents of Change

The digital variety of intelligent agent plays the same function, assisting business managers in discovering information that could be valuable to the company and integrating that information into the overall business workflow.

How do intelligent agents assist in this process? First, the user enters the business rules by selecting the types of information that would be valuable. For example, a salesperson might want to see press releases that come from their customers or their customers’ competitors. A marketer might want to see published articles on advertising campaigns in a specific category.

The user can then add conditions to the search in terms of recency or specific performance conditions. For example, a CFO might wish to review stock prices for major players in the company’s industry, but only if the stock prices change by more than 5 percent in a week. That salesperson might want to look at competitors’ press releases, but only the new ones. A doctor might want to keep current on all new information on a specific disease, but only from a select list of respected publications. The conditions help to manage the information glut available on the Web and bring more specific information to the portal news page.

Finally, business portals will soon be able to reconfirm the accuracy of the information that is brought back with the user on an ongoing basis and use the responses to improve the searches. Managers will be able rank the value of the information which will feed into an algorithm that will refine and improve future searches. The result will be an increasing accuracy of the information and the business value of the portal news page.

Once information matches have been made, the intelligent agent brings it back to the news page and sends an e-mail alert to the user’s mailbox, pager or cell phone to let him/her know that the information is available, further reducing the time gap before the user can review and take advantage of the information.

To make the information as valuable as it can be, future enhancements to the portal will also incorporate the information from the Web into documents and spreadsheets that already exist. For example, a stock price from the Web might be brought back daily and incorporated into a spreadsheet that formats the information and displays different charts of stock performance trends over time. New analyst recommendations might be formatted into a preexisting document and automatically sent to a preset distribution list for review and actions.

Changing the Corporate Landscape

The impact of real-time, Web- accessed, relevant information on an organization is often dramatic. Remember, current portal functionality already brings information from internal company databases to user desktops as soon as it becomes available. In this way, a company becomes more responsive, but only in terms of "internal" information. When information from the Web is incorporated into corporate decision making in an actionable manner, the entire company focus can change.

Until now, companies were faced with precise internal information from company databases that were relatively carefully produced and managed and extremely vague external information from word of mouth, newspaper ads, piecemeal articles and an occasional library search. As a result, organizations have been designed to focus primarily on internally reported information with less time and attention given to externally targeted information. This is not the result of a strategic management decision, rather the net impact of limitations encountered in accessing external data sources. With more internal and little (if any) external data, companies become more inwardly focused, slowing competitive reaction, customer responsiveness and overall business growth.

Key to Growth

Intelligent agents in business portals now bring to fruition the promise of the Web – universal access to information – permitting targeting and filtering of the information to only bring the more important to the desktop and integrating that information into corporate decision support processes and systems. By integrating external information into systems that formerly focused exclusively on internal information, the business portal plays a key role in changing the focus of the organization, making it more responsive to customer needs and competitive requirements and, ultimately, growing the business faster and in more unexpected ways.

Mark Price is the director of Analytics and Marketing Solutions for Zamba Solutions, a leading customer relationship management (CRM) consulting and systems integration company. In this capacity, Price assists clients in developing data warehouse, analytics and marketing technology infrastructures and in reinventing the processes that support marketing and analytics. He is based in Minneapolis and can be reached at mprice@zambasolutions.com.