Features - Enterprise Data Insights:

E-MAIL ARCHIVING: A WHOLE NEW BALL GAME
By Kon Leong, CEO, ZipLip Inc

The SEC is on the phone. They're looking for a particular e-mail. You search your e-mail archive but come up empty. Then, in a meeting, the SEC throws on the table a copy of that e-mail, taken from the customer's records. No one speaks. Then they turn to look at you ...

Well, you may be surprised to know that there are holes and inaccuracies in most of the e-mail archiving solutions available today. Understandably, archive vendors are less than enthusiastic to talk about these and other shortcomings. To complicate things further, the state-of-the-art in e-mail archiving has taken a few leaps forward, while user requirements have escalated in the last year. It's a whole new ball game in archiving and you may have to educate yourself. Here, then, are some of the major archiving issues you need to consider:

Archival "Holes"

Many an e-mail slips by the archive unnoticed. They include: group lists (e-mail sent to group101@xyz.com contains no information about who got it); blind copies or bcc:s (some e-mail systems are, well, blind to these); e-mail blocked by the content filter (which typically doesn't bother to inform the archive about the e-mail it blocks); and e-mail generated by other systems such as CRM, SFA, ERP, etc. Compliance records are simply incomplete without them. Plugging these holes requires the archive to capture e-mail at two levels: at the server and at the gateway. Unfortunately, most vendors, offer just one or the other. Insist on both server and gateway capture.

Pre-review Inspection and Blocking

Typically, archiving systems provide post-review of e-mail (i.e., inspection of e-mail after they have already gone out). This is akin to locking the gate after the horses have bolted. Instead, the archive should inspect problem messages before they are sent, and route the problem e-mail through compliance officers who can hold or release them. This significantly lowers your liability exposure. Since this may involve a major change in architecture, don't accept brochure-ware; insist on a demo of pre-review.

Unified Archive

There are three major applications in e-mail archiving: 1) e-mail compliance; 2) storage offloading of large e-mail and attachments from overworked e-mail servers; and 3) e-mail content management, which allows individuals to search, retrieve and restore from the archive. It is much better to have one integrated solution than replicate three separate solutions with added systems costs and administration. Insist on a unified archive.

The Frankenstein Factor

E-mail archiving is very difficult to engineer, so vendors tend to license third party software for quick results. Common third party licenses may include: searching; mail parsing; queue management; the mail transfer agent (MTA); the storage vault; the rules engine; content filtering; attachment parsing; Lotus or Microsoft Exchange connectors; compliance tools; the SMTP server, etc. However, too much of third party software leads to serious loss of code control and problems ensue. Users get the first inkling when the vendor doesn't seem to respond to bugs, upgrades, and scaling issues. That's usually when the finger-pointing begins. You can minimize this risk by asking vendors to identify all third party components. The fewer, the better.

Standards-based Storage Formats

Your archive should be accessible for the long term, and access should be vendor-independent. This way, if the data is organized and stored based on open standards, it will always be accessible, with or without the help of the vendor or its software. Why is this important? Well, think of how you would retrieve old data stored on an 8.5-inch floppy diskette from an old IBM word processor. You need the right hardware (try the high-tech museum); the right IBM software (version 3.4-b, if you please); and the right manuals. If long-term access is important, ask for open-standards archive solutions which enable you to search and retrieve data without any proprietary hardware or software component.

Scaling

E-mail archival requirements have increased exponentially, with storage capacities now being measured in terabytes, not gigabytes. The list of "basic" features and capabilities has also expanded exponentially, in terms of technical difficulty, e.g., instantaneous searches of not just headers but also bodies and attachments of millions of e-mail. To illustrate, consider that a 22,000-employee firm seeking to archive e-mail for seven years must store, index and search 4.5 billion messages. By contrast, even Google, the world's largest search service only searched 4.3 billion web pages, as of February 2004. With e-mail archiving struggling to cope with such massive increases in both volume and functionality, be very cautious of vendor claims on scalability. You can avoid nasty surprises by insisting that the vendor conduct actual trials within your environment before the purchase decision.

Old vs. New Technology

Last, but not least, you should consider if the software is coded in old technology (C/C++) or new (Java/J2EE or .NET). Why? Because programs coded in new technology are up to 10 times easier to develop, debug, modify and integrate. They can also easily support foreign languages (Japanese, Chinese, etc.), while most solutions based on C/C++ struggle to adapt. (Note that regulations, such as Sarbanes-Oxley, extend to foreign shores.) Insist on new technology standards that are flexible and here to stay.

Conclusion

E-mail archiving has gone through wrenching changes and you need to be better informed and more cautious in evaluating vendor solutions. Maintain healthy skepticism when you hear "It's-on-our-roadmap" and "It's-in-beta" claims. Reduce your risk by insisting on a full stress test of the solution in your environment, before purchase. That way, you can feel a lot more confident the next time a phone call comes in from the SEC.

About Kon Leong

Kon Leong is president and CEO of ZipLip Inc, a vendor of management software for e-mail and files whose main product lines include e-mail archival, secure files and secure e-mail. He has extensive experience in storage and messaging, and is active in high tech trade organizations, having served on the boards of the High Performance Networking Forum (HNF) and the SCSI Trade Association (STA).