High-performance ethernet-based communications for future multi-core processors

Schlansker, Michael; Chitlur, Nagabhushan; Oertli, Erwin; Stillwell, Paul M.; Rankin, L.; Bradford, Dennis; Carter, Richard J.; Mudigonda, Jayaram; Binkert, Nathan; Jouppi, Norman P.

doi:10.1145/1362622.1362672

Cited by 17 publications

(9 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several authors [22], [23], [24], [25] have studied tightlycoupled NIC architectures and on-load software on Ethernet. For example, the JNIC project replaces one of the four sockets of a server multiprocessor server with a NIC [25].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Tyche: An efficient Ethernet-based protocol for converged networked storage

González-Férez

Bilas

2014

2014 30th Symposium on Mass Storage Systems and Technologies (MSST)

View full text Add to dashboard Cite

Current technology trends for efficient use of infrastructures dictate that storage converges with computation by placing storage devices, such as NVM-based cards and drives, in the servers themselves. With converged storage the role of the interconnect among servers becomes more important for achieving high I/O throughput. Given that Ethernet is emerging as the dominant technology for datacenters, it becomes imperative to examine how to reduce protocol overheads for accessing remote storage over Ethernet interconnects.In this paper we propose Tyche, a network storage protocol directly on top of Ethernet, which does not require any hardware support from the network interface. Therefore, Tyche can be deployed in existing infrastructures and to co-exist with other Ethernet-based protocols. Tyche presents remote storage as a local block device and can support any existing filesystem. At the heart of our approach, there are two main axis: reduction of host-level overheads and scaling with the number of cores and network interfaces in a server. Both target at achieving high I/O throughput in future servers. We reduce overheads via a copy-reduction technique, storage-specific packet processing, preallocation of memory, and using RDMA-like operations without requiring hardware support. We transparently handle multiple NICs and offer improved scaling with the number of links and cores via reduced synchronization, proper packet queue design, and NUMA affinity management.Our results show that Tyche achieves scalable I/O throughput, up to 6.4 GB/s for reads and 6.8 GB/s for writes with 6 x 10 GigE NICs. Our analysis shows that although multiple aspects of the protocol play a role for performance, NUMA affinity is particularly important. When comparing to NBD, Tyche performs better by up to one order of magnitude.

show abstract

Section: Related Workmentioning

confidence: 99%

“…For example, the JNIC project replaces one of the four sockets of a server multiprocessor server with a NIC [25]. By closely attaching the NIC to CPU and memory, the NIC can be accessed using coherent memory as opposed to PCI transactions, which reduces latency of accessing the NIC from the processor.…”

Section: Related Workmentioning

confidence: 99%

Tyche: An efficient Ethernet-based protocol for converged networked storage

González-Férez

Bilas

2014

2014 30th Symposium on Mass Storage Systems and Technologies (MSST)

View full text Add to dashboard Cite

show abstract

“…This is implemented in Blue Gene/L [11] and was also proposed for the JNIC Ethernet controller [36]. In the M-Machine, a small receiver FIFO is directly mapped to processor registers [19].…”

Section: Total Communication Timementioning

confidence: 99%

Exploiting 162-Nanosecond End-to-End Communication Latency on Anton

Dror

Grossman

MacKenzie

et al. 2010

2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis

View full text Add to dashboard Cite

Strong scaling of scientific applications on parallel architectures is increasingly limited by communication latency. This paper describes the techniques used to mitigate latency in Anton, a massively parallel special-purpose machine that accelerates molecular dynamics (MD) simulations by orders of magnitude compared with the previous state of the art. Achieving this speedup required a combination of hardware mechanisms and software constructs to reduce network latency, sender and receiver overhead, and synchronization costs. Key elements of Anton's approach, in addition to tightly integrated communication hardware, include formulating data transfer in terms of counted remote writes, leveraging fine-grained communication, and establishing fixed, optimized communication patterns. Anton delivers software-to-software inter-node latency significantly lower than any other large-scale parallel machine, and the total critical-path communication time for an Anton MD simulation is less than 4% that of the next fastest MD platform.

show abstract

“…The front-side bus of the Pentium-III processor runs with 66 MHz, which does not pose technological challenges. In Schlansker et al [2007] an FPGA is plugged into socket 604 and participates in the cache-coherent FSB of a more recent 3.0 GHz Xeon processor, acting as a Gigabit Ethernet device. MemoNet [Tanabe et al 2000] uses a different approach to overcome the performance gap of I/O buses: The NIC is connected to the DIMM slot of the memory controller.…”

Section: Related Workmentioning

confidence: 99%

An open-source HyperTransport core

Slogsnat

Giese

Nüssle

et al. 2008

ACM Trans. Reconfigurable Technol. Syst.

View full text Add to dashboard Cite

This article presents the design of a generic HyperTransport (HT) core. HyperTransport is a packetbased interconnect technology for low-latency, high-bandwidth point-to-point connections. It is specially optimized to achieve a very low latency. The core has been verified in system using an FPGA. This exhaustive verification and the generic design allow the mapping to both ASICs and FPGAs. The implementation described in this work supports a 16-bit link width, as used by Opteron processors. On a Xilinx Virtex-4 FX60, the core supports a link frequency of 400 MHz DDR and offers a maximum bidirectional bandwidth of 3.2GB/s. The in-system verification has been performed using a custom FPGA board that has been plugged into a HyperTransport extension connector (HTX) of a standard Opteron-based motherboard. HTX slots in Opteron-based motherboards allow very high-bandwidth, low-latency communication, since the HTX device is directly connected to one of the HyperTransport links of the processor. Performance analysis shows a unidirectional payload bandwidth of 1.4GB/s and a read latency of 180 ns. The HT core in combination with the HTX board is an ideal base for prototyping systems and implementing FPGA coprocessors. The HT core is available as open source.

show abstract

High-performance ethernet-based communications for future multi-core processors

Cited by 17 publications

References 21 publications

Tyche: An efficient Ethernet-based protocol for converged networked storage

Tyche: An efficient Ethernet-based protocol for converged networked storage

Exploiting 162-Nanosecond End-to-End Communication Latency on Anton

An open-source HyperTransport core

Contact Info

Product

Resources

About