About TNet

TNet is a Network System (Hard- & Software) developed by Supercomputing Systems, Zurich - Switzerland for the Tx-Project at EPFL (see Links).

Background

TNet is a complete network including hardware and software for High Performance Computing that has been developed by Supercomputing Systems (SCS) Zurich, Switzerland. The primary application of this product was the Swiss-Tx cluster located at the Swiss Federal Institute of Technology in Lausanne (EPFL) where its expected high bandwidth and low latency was proved in practice.

The TNet hardware consists of 32bit as well as 64bit PCI network interface cards and intelligent, destination routed switches interconnected by Fiber channel technology. A NIC firmware, a TNet driver and the Fast Communication Interface (FCI) together form the basic software to run message passing oriented applications. On the Swiss-Tx cluster, job management is done by Gridware’s Codine that is interacting with Cosmos, a process control service for TNet by SCS. The concepts of TNet go back to SCS’s Remote Store Architecture technology, a technique that enables a direct store from one processor into the memory of another processor. It is the long research and experience of SCS on parallel computing that characterizes the high quality of this product.

Fundamentals

“It’s all about address mapping” could be a technical saying to describe the fundamentals of TNet. As the idea is to transfer (store) data from one node to another, the most direct way is to let the running process send the data down to the NIC (bypassing the OS) and out on to the network. The arriving packet on the remote NIC will then be copied into the appropriate memory area of the destination process.

Address translation is done on multiple levels (PCI-Bus, Network, etc.) for a remote store. The important part is, that the network card can map the host memory to the network in form of Virtual Communication Addresses. The example above describes the situation after the target has sent a request to get a message to the initiator. This mechanism allows the target’s NIC to handle the incoming data and place it at the right memory spot in the host.

TNet has been designed for large-scale clusters. Each NIC is addressed by a 16bit destination ID which gives a total of 65536 nodes possible. The actual network bandwidth is 1Gbit/s and will be multiplied on future TNet releases. Switches are remotely configurable and allow any network topology by preventing deadlocks. The 12 port full crossbar switches, which in contrast to Myrinet use destination routing, also include monitoring of the network performance. Variable size micropackets of maximal 128 bytes permit a fair access even on a highly loaded network. Reliability is improved by a hardware implemented retransmission protocol.
© 2000 | Adrian Riedo | University of New Mexico ..