Today we are witnessing a dramatic shift toward a data-driven economy, where the ability to efficiently and timely analyze huge amounts of data marks the difference between industrial success stories and catastrophic failures. In this scenario Storm, an open source distributed realtime computation system, represents a disruptive technology that is quickly gaining the favor of big players like Twitter and Groupon. A Storm application is modeled as a topology, i.e. a graph where nodes are operators and edges represent data flows among such operators. A key aspect in tuning Storm performance lies in the strategy used to deploy a topology, i.e. how Storm schedules the execution of each topology component on the available computing infrastructure. In this paper we propose two advanced generic schedulers for Storm that provide improved performance for a wide range of application topologies. The first scheduler works offline by analyzing the topology structure and adapting the deployment to it; the second scheduler enhance the previous approach by continuously monitoring system performance and rescheduling the deployment at run-time to improve overall performance. Experimental results show that these algorithms can produce schedules that achieve significantly better performances compared to those produced by Storm's default scheduler. Copyright © 2013 ACM.

Adaptive online scheduling in storm / Aniello, Leonardo; Baldoni, Roberto; Querzoni, Leonardo. - (2013), pp. 207-218. (Intervento presentato al convegno 7th ACM International Conference on Distributed Event-Based Systems, DEBS 2013 tenutosi a Arlington, TX nel 29 June 2013 through 3 July 2013) [10.1145/2488222.2488267].

Adaptive online scheduling in storm

ANIELLO, LEONARDO;BALDONI, Roberto;QUERZONI, Leonardo
2013

Abstract

Today we are witnessing a dramatic shift toward a data-driven economy, where the ability to efficiently and timely analyze huge amounts of data marks the difference between industrial success stories and catastrophic failures. In this scenario Storm, an open source distributed realtime computation system, represents a disruptive technology that is quickly gaining the favor of big players like Twitter and Groupon. A Storm application is modeled as a topology, i.e. a graph where nodes are operators and edges represent data flows among such operators. A key aspect in tuning Storm performance lies in the strategy used to deploy a topology, i.e. how Storm schedules the execution of each topology component on the available computing infrastructure. In this paper we propose two advanced generic schedulers for Storm that provide improved performance for a wide range of application topologies. The first scheduler works offline by analyzing the topology structure and adapting the deployment to it; the second scheduler enhance the previous approach by continuously monitoring system performance and rescheduling the deployment at run-time to improve overall performance. Experimental results show that these algorithms can produce schedules that achieve significantly better performances compared to those produced by Storm's default scheduler. Copyright © 2013 ACM.
2013
7th ACM International Conference on Distributed Event-Based Systems, DEBS 2013
scheduling; cep; storm; distributed event processing
04 Pubblicazione in atti di convegno::04b Atto di convegno in volume
Adaptive online scheduling in storm / Aniello, Leonardo; Baldoni, Roberto; Querzoni, Leonardo. - (2013), pp. 207-218. (Intervento presentato al convegno 7th ACM International Conference on Distributed Event-Based Systems, DEBS 2013 tenutosi a Arlington, TX nel 29 June 2013 through 3 July 2013) [10.1145/2488222.2488267].
File allegati a questo prodotto
File Dimensione Formato  
VE_2013_11573-526001.pdf

solo gestori archivio

Tipologia: Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 1.12 MB
Formato Adobe PDF
1.12 MB Adobe PDF   Contatta l'autore

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/526001
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 210
  • ???jsp.display-item.citation.isi??? ND
social impact