Quadrics

SEYMOUR & NSA REVIVE SIMD AS THINKING MACHINES BURIED

News Analysis by Norris Parker Smith, Editor at Large


08/26/94
  High-performance computing is still capable of a few surprises. The
recent termination of Thinking Machines Corp., which had long been
anticipated by industry observers, does not qualify as a real surprise.
  The same observers would not have been surprised if the next company to
declaim the hymns of Chapter 11 had been Cray Computer.
  After all, the company had yet to sell any of its products. Even though
additional money had been lured from various sources (including the pockets
of founder Seymour Cray), it was presumed that the cash would run out soon.
  Seymour Cray is, however, an expert stage-manager, and surprises are one
of his specialties. He has not quite sold a machine, but the National
Security Agency (NSA) will pay Cray Computer $4.2 million to develop an
unusual hybrid system -- half-vector, half-SIMD.
  It is anticipated that the new product, the CRAY-3/SSS (for Super
Scalable System) will be demonstrated during the first quarter of 1995.
  This particular surprise may not be sufficient to rescue Cray Computer
over the long term, but it certainly enhances the credibility of the
company -- along with the standing of its products and its founder.


STRIKE WHILE THE IRONY IS HOT An additional irony is provided by the coincidence between this announcement and the interment of Thinking Machines. From its earliest years, Thinking Machines and its technofounder, Danny Hillis, were tireless advocates of the virtues of systems based upon very large numbers of minimalist processors (about as smart as a primitive digital watch) with distributed memories, operating in a SIMD mode. In contrast, Seymour Cray achieved fame (and made his fortune) deploying small numbers of very powerful proprietary processors sharing a single memory through a complex, highly sophisticated switch. Hillis became a heretic in 1991. Thinking Machines revealed the abandonment of the SIMD/many-tiny-fingers philosophy. They joined the mainstream religion of somewhat smaller numbers (up to a few thousand) of powerful commodity microprocessors. Did this apostasy lead to Thinking Machines' recent doom? This writer is not licensed to consider cases of moral retribution. It seems clear, however, that Thinking Machines as a company found it difficult to manage a mid-stream switch between two very different architectural horses. Indeed, they had to re-shoe the horses in mid-stream because, among other things, the old and new products had different interconnect topologies. It was also difficult for customers who had to adjust their applications from the requirements of one product to quite a different one.

THE OLD ORIGINAL MPP Under Cray's new contract with NSA, a two-processor CRAY-3 vector system will be joined in computational matrimony with a system that emphatically deserves the almost-outdated term "massively parallel". The CRAY-3's partner is to be a SIMD system with 512,000 processors. This is eight times as many processors as the most massive parallel processor ever available for general purchase: the 64,000-processor CM-1 and CM-2 SIMD systems sold by Thinking Machines in the 1980s. It is somewhat reminiscent of the first Massively Parallel Processor (that was its real name) built by Goodyear Aerospace Corporation and installed at NASA's Goddard Space Flight Center in Greenbelt, Maryland in 1983. This was a specialized machine used primarily for image processing with 16,3845 processors. Each was supplied with 1024 bits of local memory. A Cray Computer announcement says that the microprocessor to be used, known as a Processor-in-Memory (PIM) chip, was developed by the Supercomputer Research Center, an affiliate of the NSA. As on the Thinking Machines CM-2, these are single-bit processors. 64 processors are accommodated on a single slice of silicon, along with 128 kilobits of memory. Thus the Cray/NSA hybrid of the 1990s will have only twice the memory per node of the 1983 Goddard/Goodyear machine. The Goddard/Goodyear MPP system was front-ended by a PDP and a VAX that loaded application programs, looked after I/O, and carried out the other functions that parallel systems of that era were unable to perform on their own. Will the CRAY-3 simply be doing the job that was once done by a humble PDP? Not quite. The Cray Computer announcement explains that the "scalable array will connect to the CRAY-3 memory interface and will be addressable as standard memory to facilitate use of the SIMD array with minimal delays for data transfer." The NSA project will utilize Seymour Cray's exceptional skills in packaging technology as well as the power of the CRAY-3 which, at a clock speed of 2.08 ns, is a tad swifter than its closest Japanese competitors. PIM chips will be packed by Cray using the company's MCM (multiple chip module) technology. Once the current Cray-NSA development project is completed, the PIM chip-based system is said to be able to scale up to one million processors.

SIMD REDUX? Does this signify a general rebirth of SIMD as a widely used, general purpose architecture for parallel systems? This question is answered by a sentence in the Cray Computer release: "The CRAY-3 memory interface bandwidth will allow the application-specific SIMD array to provide dramatic performance improvements over existing architectures for bit and image processing, pattern recognition, signal processing, and sophisticated graphics applications." That sounds good, but this is just about the complete list of applications that do well (sometimes spectacularly so) on SIMD architectures. MIMD architectures, deafened by the din of message-passing among their distributed-memory nodes, have their own problems. Nevertheless, MIMD systems (if supplied with good compilers and a great deal of memory) are able to perform somewhere on a range from adequate to excellent on a wide range of applications, including many of those that consume most of the cycles used in high-performance computing. Since the apostasy of Thinking Machines in 1991, MasPar has been the only company producing SIMD systems for the general market. (Somewhat like the NSA strategy, MasPar packages multiple small processors with associated memory on integrated chips.) After undergoing a financial health/business-focus/leadership crisis analogous to the problems that Thinking Machines seemed unable to overcome, MasPar has changed bosses and slimmed down its staff roster. In particular, MasPar has stopped trying to sell to everybody and has focused upon areas like gene research and statistical analysis where a SIMD architecture, properly equipped with an awful lot of I/O and fast disks to bounce around a great deal of data, offers distinct price/performance over conventional MIMD parallel systems or other solutions. Some of its greatest successes have been in the high-speed comparison and matching of the strings of letters that biologists use to identify the DNA makeup of each individual gene. Who else needs to make high-speed comparisons and matchings of strings of letters? NSA, for one. That is pretty close to a job description of a computer optimized for cryptographic analysis. NSA technologists have been grumping for some time about the present and anticipated shortcomings of current computers. (Indeed, the Supercomputing Research Center was formed in an attempt to cope more effectively with this trend.) Meanwhile, the agency's tasks grow more complicated. The cold war may be over, but other changes are taking place. Sophisticated cryptographic systems have become simpler, cheaper, and are more widely used -- by private persons from banks to cocaine wholesalers as well as governments. In addition, the volumes and speeds of data transmission are growing very rapidly, making it more difficult to identify significant items and pluck them out of the message stream. Along with the rest of the defense establishment, NSA has been under pressure to cut costs. Buying $25 million supercomputers in half-dozen lots is not as easy to justify as it once was. Development of the CRAY-3/SSS is to cost a relatively moderate $9.2 million, divided roughly equally between the NSA and Cray Computer. The government is also to provide software consulting services valued at about $400,000. A new era demands new solutions -- or, in this case, sweeping adaptation of an old solution. For Seymour Cray, from his early triumphs at Control Data through his years at Cray Research, the NSA has always been one of the earliest and best customers for his supercomputers. Now that both Cray Computer and the NSA need help, what would be more plausible than for the two old friends to work it out together? Right up to the end, Thinking Machines sought frantically to secure comparable help from one of its old friends, but nothing could be worked out in time. SIMD or MIMD, the line can be very fine between failure and managing to keep on getting by. These developments are not likely to stimulate vendors of MIMD systems to comb the lists of former SIMD specialists, cut in the most recent round of down-sizing and think about calling them back briefly for a consultation. The CRAY-3/SSS could be a good solution for NSA, but -- with the partial exception of image analysis -- the real growth in high-performance computing is taking place in applications categories where SIMD offers no advantages.