Vendor Spotlight:

HP REDUCES COMPLEXITY TO ADVANCE THE POWER OF CLUSTERS
by Tim Curns, Editor

Designing clusters that are simple to live with - while providing a base for the latest technologies and an extensive array of applications - is the goal of the new "Unified Cluster Portfolio" program recently launched by HP's High Performance Computing Division.

As part of its program announcement, HP has simplified its cluster offerings by introducing a unified, common implementation for cluster platforms, a move HP believes will give customers rapid access to the latest hardware and software technology without the heartburn.

HPCwire spoke with Alanna Dwyer, HP's Linux marketing manager for high- performance technical computing division at HP, to find out further information about this announcement.

HPCwire: What is this Unified Cluster Portfolio?

Alanna Dwyer: This high-performance cluster portfolio is a tested collection of modular offerings (hardware, software and support) that is delivered as one, unified cluster solution. We are providing a cluster solution that goes beyond computation to incorporate scalable storage and advanced visualization technologies. This level of integration, the combination of computation, data management, and visualization, is new for the HPC market and an HP first. This new portfolio provides a single, unified environment for multiple applications and tools from HP, our partners, and the open source community. We utilize a common implementation across servers and operating systems to ensure agile and confident deployment of the portfolio and new technologies.

HPCwire: What's new with the Unified Cluster Portfolio? Is this just a re- packaging of HP's cluster portfolio or is this something we haven't heard before?

AD: Our goal was simple. To simplify and strengthen current cluster technology and deployments. Clusters are not new - but the technologies and choices are incredibly dynamic and diverse. We wanted to give customers a way to capitalize on this change and exploit new technology. Currently, if you want an integrated solution from a cluster vendor, they exist (to some extent) but the offerings are limited and don't keep pace with technology. In other words, there's not much choice in terms of processors or interconnects. Additionally, you'll find that these systems feature one single, proprietary cluster management environment and have incomplete and uncertain application availability. If you want something different from the Model T you are offered, you're on you own - utilizing precious research and engineering time integrating and testing, tuning and maintaining a 'one-off' solution.

What's new with our Unified Cluster Portfolio is the modular technology framework that features:

• An unmatched, broad choice of hardware and software;
• The functionality to rapidly incorporate new technologies; and
• A strong partnership with the open source community enabling HP, our partners, and our customers to focus on advancing rather than replicating cluster capabilities.

The new HP Cluster Platforms, which include the HP Cluster Platform 3000, the HP Cluster Platform 4000, and the HP Cluster Platform 6000, are the foundation for the Unified Cluster Portfolio. The 3000 features the Intel Xeon with EM64 Technology. The 4000 features the Opteron processor; the 6000 features Itanium 2. These Cluster Platforms are modular, driven by customer specification including the number of nodes, operating systems, interconnects, storage configurations, and software. Designed for serviceability and reliability, these clusters are built in HP manufacturing integration centers with final integration at the customer site. By the time the clusters are powered up, the customer is assured of rapid deployment and robust reliability.

We moved in this Unified Cluster direction because there are issues, such as deployment complexity and applications availability, associated with cluster use that high-performance computing vendors, including HP, were not fixing. The industry has produced a potpourri of cluster products and components to meet diverse customer demands. This diversity contributes to cluster complexity, yet does not deliver the flexibility customers need.

Customers recognize cluster complexity as a serious issue. In an article HPCwire published in May (107690, http://www.tgc.com/hpcwire/hpcwireWWW/04/0521/107690.html), a survey of attendees at a cluster computing conference found there was a "critical need for supporting and enabling technologies to help organizations achieve optimal cluster performance. Of the respondents not planning to deploy a cluster in the near future, 67 percent cited the lack of supporting technologies as the primary reason."

HPCwire: You mention "advancing cluster technology" rather than replicating technology. Could you explain what you mean?

AD: By leveraging the work of open source community and our partners, we can focus on the strengthening a common core of technology, adding functionality and also making the complete solution 'production-ready'. So, we surveyed the capabilities for multiple cluster functions, such as job launch, resource scheduling, software installation, and monitoring, and selected technologies based on scalability, ease of administration/management, and the robustness of the software to work across different architectures. We chose not to focus on problem areas that are already being addressed. Instead, we focused on integrating and testing across multiple platforms and architectures, identifying/fixing the issues that arise as you scale clusters and run real- world applications and tools. Finally, once the cluster foundation was built, we added performance enhancements with HP-MPI, or included large scale I/O or high-performance visualization. It was rigorous activity. Integration is not something that most of our customers want to deal with - they want their clusters production-ready for immediate productivity.

HPCwire: Couldn't your competitors claim they are delivering fully integrated, ready-to-go solutions as well? What's your competition like in this domain?

AD: No other vendor delivers production-ready clusters offering the breadth of choice - operating systems, node counts, interconnects, storage configurations, and software - that that we do. Competitive choices are limited and proprietary, even when based on Linux. Our competitors also do not provide a reference platform for applications, especially in the Linux space. They also do not provide fully, integrated clusters but only supply the pieces to build one. I guess you could call that turnkey, but it's turnkey in monolithic way. Limiting a customer's flexibility, while increasing complexity, does not help solve cluster productivity issues.

HPCwire: What is the benefit for HPC customers? How do you know you are solving a critical business issue for high performance computing customers?

AD: We have 21 years of cluster expertise. We have, according to recent surveys, the market lead in cluster revenues. As a company, our mission is to remain agile and responsive to the accelerating pace of technology change while offering customers the service and comprehensive solutions to incorporate clusters into HPTC environments. We recognize that the issues around cluster complexity and application availability are historical issues. Offering a simple, modular, flexible framework to support rapid and confident deployment is one benefit of the Unified Cluster Portfolio. Other benefits include increased computing agility through the use of industry-standard technologies and increased value through the integration of computation, data management, and visualization. This level of integration is something no other vendor does or has even attempted to do. Integrating computation/data management/visualization greatly enhances the functionality, productivity and performance of clusters.

HPCwire: You've mentioned visualization and scalable storage a few times. Why the emphasis in the portfolio? What trend does this reflect?

AD: The capabilities for clusters for computation have grown at a dramatic pace and the ability to "feed the compute engine" with data is a gating factor for tackling bigger, more complex problems. Data consumed and/or generated is growing as clusters become more capable. Data management via our Scalable File Share has become an important capability for large-scale systems. Similarly, data interpretation and data analysis require better tools as data grows. The ability to see data visually allows customers to absorb information quickly. So, our new Scalable Visualization Cluster is a critical capability that our customers are quite excited about.

HPCwire: Why are your customers so excited?

AD: HP's visualization technology will do for high-performance visualization what clusters have done for supercomputing, which is make it affordable and accessible. Our solution distributes the rendering, using a parallel compositing subsystem that eliminates bottlenecks that impede visualization and under-utilize the rendering engines. The use of COTS components drives affordability. And, as part of the Unified Cluster Portfolio, we'll complement the visualization technology with tools, applications and support to ensure successful production deployment. Again, deliver comprehensive solution for computation, visualization and storage.

HPCwire: How important is the Unified Cluster Portfolio to HP's HPC business?

AD: Clusters are important segment of our market - forecasts show that clusters will, in less than 5 years, account for half the HPC business. The major vendors need to develop and deliver their cluster offerings with the same rigor and innovation that we apply to servers and other products. This is our intention with the HP Cluster Platforms and our Unified Cluster Portfolio.

HPCwire: How does this program differ from your Linux Cluster Blocks?

AD: The Linux Cluster Block program is the forerunner to this comprehensive program. We continue to enjoy a lot of success with Cluster Blocks. The scope for the Unified Cluster Portfolio is broader as the configurations include UNIX and Windows as well as Linux. The program is also worldwide with our manufacturing centers trained and practiced on assembling per specifications (power layouts, mechanicals, cabling), installing, and support.

HPCwire: You say HP takes technology from open source and other suppliers, hardens it and simplifies it. Please provide an example of this and describe how it is unique.

AD: HP has a full partnership with the open source development community. We are focusing our core effort on a common platform and electing to embrace standards and open source rather than compete with it. The open source technology and components are as good, if not better, than what is available from proprietary technologies. Our development activities are focused on creating and delivering new functionality, often in collaboration with the open source community. We push open source to do more, pushing it to achieve greater scalability and availability. One example of simplifying open source is our focus on an open source base for our XC Cluster, which provides a reference architecture for high performance clusters. This reference architecture simplifies cluster management by offering a single reference platform for testing and verification by HP and our software partners.

HPCwire: Please describe the new HP Cluster Platforms.

AD: The HP Cluster Platforms are core building blocks for the choices offered in our Unified Cluster Platform. Here are some specifics: The HP Cluster Platform 3000 features ProLiant servers with Intel Xeon EM64T processors, and a choice of Myrinet, IB or Gigabit Ethernet networks, and support for either Linux or Microsoft Windows environments. The HP Cluster Platform 4000 features ProLiant servers with Opteron processors, and a choice of Quadrics, Myrinet, IB or Gigabit Ethernet networks, and support for either Linux or Microsoft Windows environments, with operating system and middleware options. The HP Cluster Platform 6000 features Integrity servers with Itanium 2 processors, and choice of GigE, Quadrics, and InfiniBand interconnects, and support for either Linux or HP-UX environments.

The platforms are available in configurations from 5 nodes to 512 nodes - and more by request - with packaging options for density or expandability. The clusters are built to uniform, worldwide specifications, and fully integrated with HP warranty and support.

HPCwire: How are you simplifying cluster management? Please provide specific examples.

AD: First, there is a single point of control, a single user interface, that customers can deploy to support multiple users, mixed workloads, and different applications. We offer tested and validated software, both applications and middleware, which simplifies life for the system manager. HP-MPI provides a common tool or layer that is above the interconnect so customers don't have to get into lower level libraries. A modular platform allows for easier expansion, which means customers do not need to re-architect the enterprise to expand and add new systems. We designed the systems for serviceability and quick deployment.

HPCwire: What else are you doing to advance the power of clusters? Again, please be specific.

AD: There are a couple of examples that emphasize increased scalability. The first is our work with Lustre. Lustre is an open, standards-based software technology that is well funded and backed by the U. S. Department of Energy, the greater open source Linux community, Clustered File Systems, Inc. (CFS) and HP. Lustre's elegant open technology is prominent in HP's new file system, the StorageWorks Scalable File Share (HP SFS). Lustre is a major breakthrough for fast I/O on Linux clusters, delivering up to 100 times more bandwidth than typical clusters. HP SFS is the first commercial research product to use Lustre. A second example is our scalable visualization for clusters that addresses the needs of scientific, research, and engineering organizations. Such organizations simulate a variety of physical systems and produce increasingly large data sets. Examples of such demanding environments include government and scientific research, manufacturing industries such as automotive and aerospace, and the oil and gas industry. HP combines standard components, unique Sepia technology in the form of compositing hardware, and an integrated software system optimized for the Sepia technology. The result is a visualization system that scales from small node counts (4) to clusters capable of displaying 100 million pixels or more.

HPCwire: Thanks for clarifying things for us, Alanna.

HP also announced that The University of Kentucky's (UK) Center for Computational Sciences recently purchased a 128-node HP Cluster Platform 3000, with XC System Software and a 6-TB HP StorageWorks Scalable File Share cluster.

More information on the HP Unified Cluster Portfolio is available at http://www.hp.com/go/clusterportfolio.