Specifications
From Wiki
Contents |
Summary
A total of 452 Opteron cores configured as follows:
- bass-himem: A single 16-CPU (8 dual-core 2.8GHz Opterons), 128GB RAM (16 cores).
- bass-comp: Sixteen 16-CPU (8 dual-core 2.8GHz Opterons), 32GB RAM each (256 cores).
- bass-gpu: Forty-five 4-CPU (2 dual-core 2.6GHz Opterons, 8GB RAM each (180 cores).
The bass-comp0 node is the compile and submit node, and the main login node. It is not configured as part of the compute queue. The CPUs on the GPU nodes are slightly slower than on the compute node; they are primarily configured to manage the GPUs.
Compute Nodes
The compute nodes are Sun Fire X4600 servers. Each contains 8 dual-core 2.8GHz Opteron processors and 32GB RAM. The high memory node (bass-himem) has 128GB RAM.
GPU Nodes
The GPU nodes are Sun Ultra 40 workstations.
Each of the bass-gpu nodes has at least one NVidia QuadroPlex Model IV, which contain two Quadro FX 5600 cards (128 floating-point units and 1.5GB RAM each).
Six of the nodes have an additional Quadroplex (bass-gpu29, 28, 27, 26, 25, and 20), bringing the total number of FX5600s to 102.
The other 39 nodes are being outfitted by a donation from nVidia of a GeForce 9800 GX2, each of which looks like two GPUs (128 floating-point units and 512M RAM each) to the system. This will bring each node up to the point of having 4 available GPUs.
There are 2 PCXe 16x slots on the GPU nodes. One of the slots shares its PCI bandwidth with the GigE card. There are two GPUs per 16x slot (either the two FX5600's in the Quadroplex or the two GPUs on the 9800 GX 2s).
Storage Nodes
bass-files
The bass-files storage node is an 8-core Opteron processor connected via Fibre Channel to a total of 33TB of disk space (unformatted capacity). The drives are in a variety RAID-5 configurations; total formatted space is on the order of 24TB. This is split between research storage space, scratch space, home directory space and backup mirror space.
bass-thor
The bass-thor storage node is a Sun X4540 (Thor). It contains 2 4-core 2.7GHz AMD Opteron 2384 CPUs with 32GB of RAM connected to 48 500GB disks yielding an unformatted capacity of 24TB. This disk space is split between scratch space and dedicated project space and is shared over InfiniBand to each of the nodes in the cluster.
Communication
There is an InfiniBand 4X double-data-rate network connecting all of the nodes. This is where the MPI traffic goes. There are 2 single port cards in bass-comp0-15, bass-himem and bass-files. There is 1 single port card in bass-gpu0-45.
There is a 1Gbit/second switched Ethernet network that is used for file traffic and for starting jobs on the nodes.
