Human adaptability relies crucially on the ability to learn and merge knowledge both from supervised and unsupervised learning: the parents point out few important concepts, but then the children fill in the gaps on their own. This is particularly effective, because supervised learning can never be exhaustive and thus learning autonomously allows to discover invariances and regularities that help to generalize. In this paper we propose to apply a similar approach to the task of object recognition across domains: our model learns the semantic labels in a supervised fashion, and broadens its understanding of the data by learning from self-supervised signals how to solve a jigsaw puzzle on the same images. This secondary task helps the network to learn the concepts of spatial correlation while acting as a regularizer for the classification task. Multiple experiments on the PACS, VLCS, Office-Home and digits datasets confirm our intuition and show that this simple method outperforms previous domain generalization and adaptation solutions. An ablation study further illustrates the inner workings of our approach.

Domain generalization by solving jigsaw puzzles / Carlucci, Fabio Maria; D'Innocente, Antonio; Bucci, Silvia; Tommasi, Tatiana; Caputo, Barbara. - (2019), pp. 2224-2233. (Intervento presentato al convegno 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) tenutosi a Long Beach; United States) [10.1109/CVPR.2019.00233].

Domain generalization by solving jigsaw puzzles

Carlucci, Fabio Maria
Primo
Software
;
D'Innocente, Antonio
Secondo
Validation
;
Tommasi, Tatiana
Penultimo
Methodology
;
Caputo, Barbara
Ultimo
Supervision
2019

Abstract

Human adaptability relies crucially on the ability to learn and merge knowledge both from supervised and unsupervised learning: the parents point out few important concepts, but then the children fill in the gaps on their own. This is particularly effective, because supervised learning can never be exhaustive and thus learning autonomously allows to discover invariances and regularities that help to generalize. In this paper we propose to apply a similar approach to the task of object recognition across domains: our model learns the semantic labels in a supervised fashion, and broadens its understanding of the data by learning from self-supervised signals how to solve a jigsaw puzzle on the same images. This secondary task helps the network to learn the concepts of spatial correlation while acting as a regularizer for the classification task. Multiple experiments on the PACS, VLCS, Office-Home and digits datasets confirm our intuition and show that this simple method outperforms previous domain generalization and adaptation solutions. An ablation study further illustrates the inner workings of our approach.
2019
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
deep learning; image classification; convolutional neural networks; domain generalization; self supervision; domain adaptation
04 Pubblicazione in atti di convegno::04b Atto di convegno in volume
Domain generalization by solving jigsaw puzzles / Carlucci, Fabio Maria; D'Innocente, Antonio; Bucci, Silvia; Tommasi, Tatiana; Caputo, Barbara. - (2019), pp. 2224-2233. (Intervento presentato al convegno 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) tenutosi a Long Beach; United States) [10.1109/CVPR.2019.00233].
File allegati a questo prodotto
File Dimensione Formato  
Carlucci_Domain-Generalization_2019.pdf

solo gestori archivio

Tipologia: Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 1.26 MB
Formato Adobe PDF
1.26 MB Adobe PDF   Contatta l'autore

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/1334322
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 422
  • ???jsp.display-item.citation.isi??? 260
social impact