Anomalies are rare and anomaly detection is often therefore framed as One-Class Classification (OCC), ie trained solely on normalcy. Leading OCC techniques constrain the latent representations of normal motions to limited volumes and detect as abnormal anything outside, which accounts satisfactorily for the openset'ness of anomalies. But normalcy shares the same openset'ness property, since humans can perform the same action in several ways, which the leading techniques neglect. We propose a novel generative model for video anomaly detection (VAD), which assumes that both normality and abnormality are multimodal. We consider skeletal representations and leverage state-of-the-art diffusion probabilistic models to generate multimodal future human poses. We contribute a novel conditioning on the past motion of people and exploit the improved mode coverage capabilities of diffusion processes to generate different-but-plausible future motions. Upon the statistical aggregation of future modes, an anomaly is detected when the generated set of motions is not pertinent to the actual future. We validate our model on 4 established benchmarks: UBnormal, HR-UBnormal, HR-STC, and HR-Avenue, with extensive experiments surpassing state-of-the-art results.

Multimodal Motion Conditioned Diffusion Model for Skeleton-based Video Anomaly Detection / Flaborea, Alessandro; Collorone, Luca; D'AMELY DI MELENDUGNO, GUIDO MARIA; D'Arrigo, Stefano; Prenkaj, Bardh; Galasso, Fabio. - (2023), pp. 10284-10295. (Intervento presentato al convegno IEEE/CVF International Conference on Computer Vision 2023 tenutosi a Paris, France) [10.1109/ICCV51070.2023.00947].

Multimodal Motion Conditioned Diffusion Model for Skeleton-based Video Anomaly Detection

Alessandro Flaborea
Co-primo
;
Luca Collorone
Co-primo
;
Guido Maria D'Amely di Melendugno
Co-primo
;
Stefano D'Arrigo
Co-primo
;
Bardh Prenkaj;Fabio Galasso
Ultimo
2023

Abstract

Anomalies are rare and anomaly detection is often therefore framed as One-Class Classification (OCC), ie trained solely on normalcy. Leading OCC techniques constrain the latent representations of normal motions to limited volumes and detect as abnormal anything outside, which accounts satisfactorily for the openset'ness of anomalies. But normalcy shares the same openset'ness property, since humans can perform the same action in several ways, which the leading techniques neglect. We propose a novel generative model for video anomaly detection (VAD), which assumes that both normality and abnormality are multimodal. We consider skeletal representations and leverage state-of-the-art diffusion probabilistic models to generate multimodal future human poses. We contribute a novel conditioning on the past motion of people and exploit the improved mode coverage capabilities of diffusion processes to generate different-but-plausible future motions. Upon the statistical aggregation of future modes, an anomaly is detected when the generated set of motions is not pertinent to the actual future. We validate our model on 4 established benchmarks: UBnormal, HR-UBnormal, HR-STC, and HR-Avenue, with extensive experiments surpassing state-of-the-art results.
2023
IEEE/CVF International Conference on Computer Vision 2023
anomaly detection; diffusion models; computer vision
04 Pubblicazione in atti di convegno::04b Atto di convegno in volume
Multimodal Motion Conditioned Diffusion Model for Skeleton-based Video Anomaly Detection / Flaborea, Alessandro; Collorone, Luca; D'AMELY DI MELENDUGNO, GUIDO MARIA; D'Arrigo, Stefano; Prenkaj, Bardh; Galasso, Fabio. - (2023), pp. 10284-10295. (Intervento presentato al convegno IEEE/CVF International Conference on Computer Vision 2023 tenutosi a Paris, France) [10.1109/ICCV51070.2023.00947].
File allegati a questo prodotto
File Dimensione Formato  
Flaborea_Multimodal-Motion_2023.pdf

solo gestori archivio

Tipologia: Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 9.33 MB
Formato Adobe PDF
9.33 MB Adobe PDF   Contatta l'autore
Flaborea_Multimodal-Motion_postprint_2023.pdf

accesso aperto

Tipologia: Documento in Post-print (versione successiva alla peer review e accettata per la pubblicazione)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 8.78 MB
Formato Adobe PDF
8.78 MB Adobe PDF

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/1699647
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 19
  • ???jsp.display-item.citation.isi??? 17
social impact