Jump to content

Roadrunner (supercomputer)

From Wikipedia, the free encyclopedia

This is an old revision of this page, as edited by 207.241.247.150 (talk) at 14:57, 23 December 2008 (→‎PowerXCell). The present address (URL) is a permanent link to this revision, which may differ significantly from the current revision.

Roadrunner
Activeoperational 2008, final completion 2009
SponsorsIBM,  United States
OperatorsNational Nuclear Security Administration,  United States
LocationLos Alamos National Laboratory,  United States
Architecture12,960 IBM PowerXCell 8i CPUs, 6,480 AMD Opteron dual-core processors, Infiniband, Linux
Power2.35 MW
Space296 racks, 6,000 sq ft (560 m2)
Memory103.6 TiB
Speed1.7 petaflops (peak)
CostUS$133M
RankingTOP500: 1, June 2008
PurposeModeling the decay of the U.S. nuclear arsenal.
LegacyFirst TOP500 Linpack sustained 1.0 petaflops, May 25, 2008
Websitehttp://www.lanl.gov/roadrunner/

Roadrunner is a supercomputer built by IBM at the Los Alamos National Laboratory in New Mexico, USA. Currently the world's fastest computer, the US$133-million Roadrunner is designed for a peak performance of 1.7 petaflops, achieving 1.026 on May 25, 2008,[1][2][3] and to be the world's first TOP500 Linpack sustained 1.0 petaflops system. It is a one-of-a-kind supercomputer, built from commodity parts, with many novel design features.

Overview

IBM built the computer for the U.S. Department of Energy's (DOE) National Nuclear Security Administration.[4][5] It is a hybrid design with 12,960 IBM PowerXCell[6] 8i CPUs and 6,480 AMD Opteron dual-core processors[7] in specially designed server blades connected by Infiniband. The Roadrunner uses Red Hat Enterprise Linux along with Fedora as its operating systems and is managed with xCAT distributed computing software. It also uses the Open MPI Message Passing Interface implementation.[8]

Roadrunner occupies approximately 6,000 square feet (560 square metres)[9] and became operational in 2008.

The DOE plans to use the computer for simulating how nuclear materials age in order to predict whether the USA's aging arsenal of nuclear weapons is safe and reliable. Other uses for the Roadrunner include the sciences, financial, automotive and aerospace industries.

Hybrid design

Roadrunner differs from many contemporary supercomputers in that it is a hybrid system, using two different processor architectures. Usually supercomputers only use one, since such a design is easier to design and program for. To realise the full potential of Roadrunner, all software will have to be written specially for this hybrid architecture. The hybrid design consists of dual-core Opteron server processors manufactured by AMD utilizing the standard AMD64 architecture. Attached to each Opteron core is a Cell processor manufactured by IBM using Power Architecture technology. As a supercomputer, the Roadrunner is considered an Opteron cluster with Cell accelerators, as each node consists of a Cell attached to an Opteron core and the Opterons to each other.[10]

Development

Roadrunner has been in development since 2002, and went online in 2006. Due to its novel design and complexity it is constructed in three phases and became fully operational in 2008.

Phase 1

The first phase of the Roadrunner was building a standard (albeit quite large) Opteron based cluster, while evaluating the feasibility to further construct and program the future hybrid version. This Phase 1 Roadrunner reached 71 teraflops and has been in full operation at Los Alamos National Laboratory doing advanced weapons simulations since 2006. Even if Roadrunner had not been greenlit for Phase 2, the Phase 1 form would still be a formidable supercomputer and would have ranked, at its time, in the top 10 list of the world's fastest computers.

Phase 2

Phase 2 known as “AAIS” (Advanced Architecture Initial System) included building a small hybrid version of the finished system using an older version of the Cell processor. This phase was used to build prototype applications for the hybrid architecture. It went online in January 2007.

Phase 3

The goal of Phase 3 was to reach sustained performance in excess of 1 petaflops. Additional Opteron nodes and new PowerXCell processors were added to the design. These PowerXCell processors are five times as powerful as the Cell processors used in Phase 2. It was built to full scale at IBM’s Poughkeepsie, New York facility, where it broke the 1 petaflops barrier during its fourth attempt on May 25th, 2008. The complete system will be moving to its permanent location in New Mexico in the summer of 2008, where fine tuning of the applications will continue until final completion in 2009.

Technical specification

Processors

The Roadrunner uses many processors in the system, some for computation (solving the problem at hand) and some for operation (helping the humans, storing data, monitoring, passing data around, etc). Roadrunner is unusual in the respect that it uses two different processors for solving the problems. Usually a supercomputer only uses one processor kind for the complete system or one for operations and another for the problem solving.

Opteron

AMD Opteron 2210, running at 1.8 GHz. These are processors with two general purpose cores each. Opterons are used both in the computational nodes feeding the Cells with useful data and in the system operations and communication nodes passing data between computing nodes and helping the operators running the system. Roadrunner has a total of 6912 Opteron processors (6480 computation, 432 operation), for a total of (12960+864) 13824 cores.

PowerXCell

IBM PowerXCell 8i, running at 3.2 GHz. These processors have one general purpose core (PPE), and eight special performance cores (SPE) for floating point operations. Roadrunner has a total of 12960 PowerXCell processors, with 12960 PPE cores and 103680 SPE cores, for a total of 116640 cores. 

 My Pc is better than this by a long shot....

Number of cores

On the Top500 list, Roadrunner is said to have 122400 cores. It is important to know which core is counted.

  • 13824 Opteron cores + 116640 Cell cores = 130464 cores for both the computing nodes and the operation nodes.

This is a number larger than the one mentioned on Top500. It turns out that the Roadrunner only used 17 Connected Units while doing the LINPACK benchmark, and it was not counting the cores in the operations and communication nodes (they didn't run the benchmark).[11]

  • 6120 Opteron (2 cores) + 12240 PowerXCell 8i (9 cores) = 122400 cores

TriBlade

A schematic description of the TriBlade module.

Logically, a TriBlade consists of two dual-core Opterons with 16 GB RAM and four PowerXCell 8i CPUs with 16 GB Cell RAM.[7]

Physically, a TriBlade consists of one LS21 Opteron blade, an expansion blade, and two QS22 Cell blades. The LS21 has two 1.8 GHz dual-core Opterons with 16 GB memory for the whole blade, providing 8GB for each CPU. Each QS22 has two PowerXCell 8i CPUs, running at 3.2 GHz and 8GB memory, which makes 4 GB for each CPU. The expansion blade connects the two QS22 via four PCIe x8 links to the LS21, two links for each QS22. It also provides outside connectivity via an Infiniband 4x DDR adapter. This makes a total width of four slots for a single TriBlade. Three TriBlades fit into one BladeCenter H chassis.

Connected Unit (CU)

A Connected Unit is 60 BladeCenter H full of TriBlades, that is 180 TriBlades. All TriBlades are connected to a 288-port Voltaire ISR2012 Infiniband switch. Each CU also has access to the Panasas file system through twelve System x3755 servers.[7].

CU system information:[7].

  • 360 dual-core Opterons with 2.88 TiB RAM.
  • 720 PowerXCell 8i cores with 2.88 TiB RAM.
  • 12 System x3755 with dual 10-GBit Ethernet each.
  • 288-port Voltaire ISR2012 switch with 192 Infiniband 4x DDR links (180 TriBlades and twelve I/O nodes).

Roadrunner cluster

A schematic overview of the tiered composition of the Roadrunner supercomputer cluster.

The final cluster is made up of 18 connected units, which are connected via eight additional (second-stage) Infiniband ISR2012 switches. Each CU is connected through twelve uplinks for each second-stage switch, that makes a total of 96 uplink connections.[7]

Overall system information:[7]

  • 6,480 Opteron processors with 51.8 TiB RAM (in 3,240 LS21 blades)
  • 12,960 Cell processors with 51.8 TiB RAM (in 6,480 QS22 blades)
  • 216 System x3755 I/O nodes
  • 26 288-port ISR2012 Infiniband 4x DDR switches
  • 296 racks
  • 2.35 MW power[12]

See also

References

  1. ^ Sharon Gaudin (2008-06-09). "IBM's Roadrunner smashes 4-minute mile of supercomputing". Computerworld. Retrieved 2008-06-10.
  2. ^ "Military supercomputer sets record - CNET News.com".
  3. ^ "Supercomputer sets petaflop pace". BBC. 2008-06-09. Retrieved 2008-06-09.
  4. ^ "IBM to Build World's First Cell Broadband Engine Based Supercomputer". IBM. 2006-09-06. Retrieved 2008-05-31.
  5. ^ "IBM Selected to Build New DOE Supercomputer". NNSA. 2006-09-06. Retrieved 2008-05-31.
  6. ^ International Supercomputing Conference to Host First Panel Discussion on Breaking the Petaflop/s Barrier
  7. ^ a b c d e f "RR Seminar - System Overview" (PDF). Los Alamos National Laboratory. 2008-03-13. Retrieved 2008-05-31.
  8. ^ Jeff Squyres. "Open MPI: 10^15 Flops Can't Be Wrong" (PDF). Open MPI. Retrieved 2008-11-22.
  9. ^ "Los Alamos computer breaks petaflop barrier". IBM. 2008-06-09. Retrieved 2008-06-12.
  10. ^ Kevin J. Barker, Kei Davis, Adolfy Hoisie, Darren J. Kerbyson, Mike Lang, Scott Pakin, Jose C. Sancho. "Entering the Petaflop Era: The Architecture and Performance of Roadrunner" (PDF). Retrieved 2008-11-22.{{cite web}}: CS1 maint: multiple names: authors list (link)
  11. ^ "Roadrunner: Science, Cell and a Petaflop/s" (PDF). Los Alamos National Laboratory and IBM. 2008-06-18. Retrieved 2008-09-22.
  12. ^ "TOP500 List - June 2008". TOP500 Supercomputer Sites. 2008-06-15. Retrieved 2008-08-12.
Retrieved from "https://en.wikipedia.org/w/index.php?title=Roadrunner_(supercomputer)&oldid=259726374"