The allreduce operation is one of the most commonly used communication routines in distributed applications. To improve its bandwidth and to reduce network traffic, this operation can be accelerated by offloading it to network switches, that aggregate the data received from the hosts, and send them back the aggregated result. However, existing solutions provide limited customization opportunities and might provide suboptimal performance when dealing with custom operators and data types, with sparse data, or when reproducibility of the aggregation is a concern. To deal with these problems, in this work we design a flexible programmable switch by using as a building block PsPIN, a RISC-V architecture implementing the sPIN programming model. We then design, model, and analyze different algorithms for executing the aggregation on this architecture, showing performance improvements compared to state-of-the-art approaches.

Flare: Flexible in-Network Allreduce / De Sensi, D; Di Girolamo, S; Ashkboos, S; Li, S; Hoefler, T. - (2021). (Intervento presentato al convegno International Conference for High Performance Computing, Networking, Storage and Analysis (was Supercomputing Conference) tenutosi a St. Louis, Missouri) [10.1145/3458817.3476178].

Flare: Flexible in-Network Allreduce

De Sensi D
Primo
;
2021

Abstract

The allreduce operation is one of the most commonly used communication routines in distributed applications. To improve its bandwidth and to reduce network traffic, this operation can be accelerated by offloading it to network switches, that aggregate the data received from the hosts, and send them back the aggregated result. However, existing solutions provide limited customization opportunities and might provide suboptimal performance when dealing with custom operators and data types, with sparse data, or when reproducibility of the aggregation is a concern. To deal with these problems, in this work we design a flexible programmable switch by using as a building block PsPIN, a RISC-V architecture implementing the sPIN programming model. We then design, model, and analyze different algorithms for executing the aggregation on this architecture, showing performance improvements compared to state-of-the-art approaches.
2021
International Conference for High Performance Computing, Networking, Storage and Analysis (was Supercomputing Conference)
in-network processing; networking hardware; distributed architectures;
04 Pubblicazione in atti di convegno::04b Atto di convegno in volume
Flare: Flexible in-Network Allreduce / De Sensi, D; Di Girolamo, S; Ashkboos, S; Li, S; Hoefler, T. - (2021). (Intervento presentato al convegno International Conference for High Performance Computing, Networking, Storage and Analysis (was Supercomputing Conference) tenutosi a St. Louis, Missouri) [10.1145/3458817.3476178].
File allegati a questo prodotto
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/1656252
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 23
  • ???jsp.display-item.citation.isi??? 13
social impact