Managing Information: Infoglut

New tools can help tame an ocean of data

I nformation overload threatens to drown the American worker. The sheer volume of data--from E-mail, voicemail, the Internet and World Wide Web, fax, news feeds, commercial online services, and much more--keeps growing. It's growing faster, in fact, than many professionals can manage. "Users are becoming overwhelmed," sighs Ron Horton, director of information management at Transport Canada, a government agency in Ottawa.

Bu t isn't more information what everyone wanted? Paul Saffo, a director with the Institute for the Future, a nonprofit research organization in Menlo Park, Calif., says yes. "Information overload is not a function of the volume of information out there," he says. "It's a gap between the volume of information and the tools we have to assimilate that information into useful knowledge."

It's this gap that has technology managers scurrying for new tools to control information flow. They're in luck: A host of new technologies can help close the spigot on the data deluge. These tools include filtering mechanisms for groupware and E-mail, software agents that scour databases, and search engines that help tame the World Wide Web.

A variety of enterprise-class products may help close the gap between the overwhelming volume of information and users' ability to filter and manage it. The soon-to-be-released Lotus Notes 4.0 from IBM's Lotus Development division, for example, will feature software agents that will h elp users home in on critical data. Microsoft is preparing its next-generation messaging platform, Exchange Server, to include rules-based software that will offer filtering and routing capabilities beyond what's now available. News retrieval services such as Corporate Profound from Profound Inc. come with direct connections to corporate networks and easy-to-use interfaces. And on the Internet, software agents, search engines, and other tools promise to help users wade through and sort ever-increasing amounts of information.

All these technologies have one thing in common: They're designed to get the right information into the right hands, and block out unnecessary data. "Data that's irrelevant to your task is like white noise," says Alex Neihaus, senior marketing manager at Lotus. "You have to spend energy to ignore it."

Already, the field of companies offering business-information services is crowded. It includes Automatic Data Processing, Bloomberg, Desktop Data, Dow Jones, Individual, Knight-Ridd er, and Reuters. Online service providers America Online, CompuServe, and Prodigy also have news feeds and vast information resources with some level of individual customization and search capabilities.

How serious a problem is information overload? Consider Eric Sachs, VP of research and development at Wolf Communications in Houston, a provider of network services for Notes users. On a typical day, Sachs answers 20 telephone calls, replies to 60 E-mail messages, monitors six news services and filters 12 others, participates in several online discussion forums (three on an hourly basis), and gets two reports from a news retrieval service. In his spare time, Sachs scans 10 trade journals. It all translates into a 70- to 80-hour work week. "I would say I'm unusual," Sachs admits.

Unusual, yes. Alone, no. Greg Lobdell, Microsoft's director of product management for Exchange, accumulated a staggering 2,700 E-mail, voicemails, faxes, and other messages in his Exchange mailbox in only two weeks. "I get a l ot of FYI mail," he explains.

On the bright side, Lobdell adds, he's already read 800 of the messages--only 1,900 left to go.

Ironically, Microsoft's Exchange Server, now in beta tests, is causing Lobdell's information overload--and should help relieve it. Exchange Server is now being tested by companies that include Boeing, Dun & Bradstreet, Intergraph, Shell, and Texaco, and it's scheduled for release by year's end.

Exchange Server supports a universal inbox--a feature included with Microsoft's Windows 95 operating system--that delivers E-mail, voicemail, fax, and information services into a single, integrated mailbox. (Banyan offers a similar feature.) "One step [in managing overload] is consolidation of the incoming information into one place where you can deal with it," Lobdell says.

Exchange Server, which runs on Windows NT, will also contain an Inbox Assistant, rules-based technology that goes beyond the rudimentary "if, then" capabilities of some messaging systems.

If simple r systems, for instance, detect a message labeled "high priority," then they simply forward it to remote mail. The Inbox Assistant, however, can automatically apply a digital signature to a document that arrives in the inbox, then forward that document to another user. "It cuts down pretty dramatically on the kind of things I have to look at," says Lobdell.

Some details: Lobdell averages about 300 messages a day, or five hours of work, assuming he spends just one minute on each message. But filtering in Exchange, Lobdell explains, will cull those 300 messages down to only 50 that he needs to act on. Using that same minute-per-message assumption, that's less than an hour of work. "If I had to go back to [Microsoft Mail], I'd definitely be under water," he says.

Also, Exchange Server will contain a Delegate Mailbox that lets a colleague or assistant read and respond to mail in a busy executive's inbox. For example, Lobdell's administrative assistant has "delegate" access to Lobdell's mailbox, so she c an look at her boss' mail, and send replies that look as if they come from him.

Lotus has a different tool for reducing infoglut. Its Notes 4.0, expected by year's end, will include software agents, or applications logic, that developers and users can run to automate tasks. "The agents allow full-strength applications to be written that can go anywhere in a network to retrieve, update, or condense data for presentation to an end user," explains marketing manager Neihaus.

Also new in Notes 4.0 is LotusScript, an object-oriented programming language that will let developers add complicated business logic, representing the needs of one or multiple users, to a database on a server.

Users are increasingly integrating Notes and other messaging packages with news services, further simplifying the process of finding and filtering information. Individual Inc. in Burlington, Mass., delivers a daily summary of news events to corporate customers via E-mail, Lotus Notes, or corporate servers based on the Web model. "Today, most of our business is in the enterprise feed--delivery of a database of news [to a corporate server] that can be read by users at the desktop," says Paul Pinella, product manager of First!, Individual's enterprise service.

Individual was one of the first companies to build a business around the use of agents. Actually software code, agents are programmed to search databases for information that fits an organization's or single user's parameters.

Individual's Smart (System for the Manipulation And Retrieval of Text) also uses artificial intelligence to sort through 20,000 news articles daily, up from 15,000 per day just a year ago.

In September, Individual added a personal-profiling capability, including desktop software, that makes it easier for users to organize the data and share it with colleagues on the same network. "We've not considered ourselves in the software business until very recently," says Pinella.

Individual filters and delivers the news to corporations once a day at an annual cost of $8,000 to $100,000. Ponying up for that bill are customers that include Fidelity Investments and Avon Products.

Individual's ability to pair a news retrieval service with a groupware product such as Notes that also uses agents forms a powerful combination. "When someone in a workgroup can find data and share it, that tends to reduce the overload on everyone else," says Lotus' Neihaus.

Users agree. "I get an additional filter on the news," says Horton of Transport Canada, which tested Notes and First! together. "It went over extremely well." Despite Horton's satisfaction, Transport Canada has not widely rolled out the technology because the Canadian government is downsizing the agency and has tightened its spending.

The need for info-filtering has not escaped the notice of Microsoft. The Redmond, Wash., software giant revealed earlier this month that it is taking a minority stake in Individual. iNews, Individual's personal daily news service, will be offered on the Micr osoft Network online service. The partners say they are also assessing the potential for additional development projects that take advantage of Individual's news-filtering and distribution technologies.

Desktop Data in Waltham, Mass., is taking Individual's concept one step further. In September, the company began offering NBC-TV's business programming as an option with NewsEdge, Desktop Data's real-time news service. Marni Hoyle, the company's marketing director, says 40% of Desktop Data's customers already use Lotus Notes. Those customers, which currently include Merrill Lynch and Chase Bank, can now combine Notes, NewsEdge, and NBC broadcasts.

There are dozens of other services similar to First! and NewsEdge--and as many ways to get at their data. Reed Elsevier Plc., for instance, plans soon to deliver its Lexis/Nexis services via Novell's NetWare Connect Services. Dow Jones & Co., publisher of The Wall Street Journal, already has put the newspaper's "Money And Investing" section on the Web. D ow Jones plans to start posting the entire newspaper's content on the Web early next year. Other information service providers are experimenting with ways of presenting data. Their goal: To make data easier to absorb. "Good graphic design is a very powerful overload tool," says Institute for the Future's Saffo.

Using that idea, Profound in New York is trying to carve a niche for itself. The company's six-month-old Corporate Profound service culls information from 4,000 news sources and 40,000 research reports and comes with Adobe's Acrobat software, which lets users view reports in their original format, even if users don't have the applications software used to create the original report. "We bring to bear a very easy-to-use interface, but with all the deep data content," says John O'Brien, director of marketing at Profound.

Best Foods, the Englewood, N.J., maker of Skippy peanut butter and other packaged goods, uses Profound and other information services to track competitors, follow trends, and monitor its markets. "There's more information than we can ever really analyze," says Rich Beylon, director of market research. "There's just not enough time. Requests [for information] come with incredibly tight timelines." Also, Profound's search-and-retrieve service helps Best Foods' research department serve in-house customers. "You can screen information very quickly," says Beylon.

A data-filtering technique known as collaborative filtering could make it even easier for knowledge workers to find the information they need, says Patti Maes, associate professor at MIT's Media Lab in Cambridge, Mass., and co-founder of a startup company, Agents Inc. in Cambridge, that intends to develop commercial products based on collaborative filtering. Individual and similar service providers use what Maes calls feature-based filtering, in which a single software agent seeks and filters data according to a user profile. With collaborative filtering, "agents talk to one another," she adds.

Collaborative filtering reduces the amount of legwork needed to get information by matching the interests of an info-seeker with those of like-minded individuals. MIT's Web Hound service, for example, uses collaborative filtering to find and retrieve Web documents on any subject. And the school's HOMR (Helpful Online Music Recommendation) system uses the same technique to locate recordings that appeal to a specific musical taste. "The system figures out which people have similar information interests," explains Maes.

On the Internet, agents, filters, software "robots," and other technologies are becoming essential tools for finding and organizing information. A simple starting point for any Web user is to create a well-organized file of bookmarks to track frequently visited sites. Users can also subscribe to list servers, which automatically deliver articles of interest directly to users' electronic mailboxes.

Web search engines are becoming indispensable tools for navigating the Web. The user simply types in a few words th at describe the topic of interest, and within seconds (or minutes, depending on line speeds), the search engine delivers a list of relevant hot sites that the user can visit with a single mouse click.

Most are supported by advertising and are free to the user. Popular Web search engines include Worm , Web Crawler , and W3 Search Engines . Web directories that also can help in a search include Yahoo , Lycos , and the recently announced Excite .

Beyond Technology?
But computer technology can't solve every case of infoglut. Stock and bond traders, for example, must now deal with a global market. While the stock market closes in New York at 4:30 p.m., trading begins in Tokyo two hours later, at 6:30 p.m. Eastern time, and in London nearly e ight hours later, when it's 2 a.m. in New York.

"We're going to a global market, and the hours are being extended," says James Kenney, head of government bond trading for Prudential Securities in New York. "It's gotten to be 24 hours. I don't know that this type of system will be helped by computers."

Also, futurist Saffo warns technology managers against relying too heavily on technology to solve the problem of info overload. "The technical solutions are incomplete and partial," he says. "You've got to start with changing your behavior first, then pick the right tools for the job."

Still, changed behavior can lead down some unlikely--and, for some, undesirable--paths. Microsoft's Lobdell, for instance, likes to read and respond to E-mail outside of normal business hours. "I get up at 4:30 [in the morning] and do E-mail from home," he says.

But Saffo takes things to the opposite extreme. "I no longer carry a laptop on business trips, unless I absolutely must," he says. To cut his reading time and ensure that what he's reading is relevant, Saffo has come up with a completely nontechnical approach: "I read my magazines standing up."

See related sidebars "Avon Calling--And Calling And Calling" and "Fighting E-Mail Glut"

Comments on this story?

InformationWeek http://techweb.cmp.com/iwk