An effective perception system is a fundamental component for farming robots, as it enables them to properly perceive the surrounding environment and to carry out targeted operations. The most recent methods make use of state-of-the-art machine learning techniques to learn a valid model for the target task. However, those techniques need a large amount of labeled data for training. A recent approach to deal with this issue is data augmentation through Generative Adversarial Networks (GANs), where entire synthetic scenes are added to the training data, thus enlarging and diversifying their informative content. In this work, we propose an alternative solution with respect to the common data augmentation methods, applying it to the fundamental problem of crop/weed segmentation in precision farming. Starting from real images, we create semi-artificial samples by replacing the most relevant object classes (i.e., crop and weeds) with their synthesized counterparts. To do that, we employ a conditional GAN (cGAN), where the generative model is trained by conditioning the shape of the generated object. Moreover, in addition to RGB data, we take into account also near-infrared (NIR) information, generating four channel multi-spectral synthetic images. Quantitative experiments, carried out on three publicly available datasets, show that (i) our model is capable of generating realistic multi-spectral images of plants and (ii) the usage of such synthetic images in the training process improves the segmentation performance of state-of-the-art semantic segmentation convolutional networks.

Multi-Spectral Image Synthesis for Crop/Weed Segmentation in Precision Farming / Fawakherji, M.; Potena, C.; Pretto, A.; Bloisi, D. D.; Nardi, D.. - In: ROBOTICS AND AUTONOMOUS SYSTEMS. - ISSN 0921-8890. - 146:(2021). [10.1016/j.robot.2021.103861]

Multi-Spectral Image Synthesis for Crop/Weed Segmentation in Precision Farming

Fawakherji M.;Potena C.;Pretto A.
;
Bloisi D. D.;Nardi D.
2021

Abstract

An effective perception system is a fundamental component for farming robots, as it enables them to properly perceive the surrounding environment and to carry out targeted operations. The most recent methods make use of state-of-the-art machine learning techniques to learn a valid model for the target task. However, those techniques need a large amount of labeled data for training. A recent approach to deal with this issue is data augmentation through Generative Adversarial Networks (GANs), where entire synthetic scenes are added to the training data, thus enlarging and diversifying their informative content. In this work, we propose an alternative solution with respect to the common data augmentation methods, applying it to the fundamental problem of crop/weed segmentation in precision farming. Starting from real images, we create semi-artificial samples by replacing the most relevant object classes (i.e., crop and weeds) with their synthesized counterparts. To do that, we employ a conditional GAN (cGAN), where the generative model is trained by conditioning the shape of the generated object. Moreover, in addition to RGB data, we take into account also near-infrared (NIR) information, generating four channel multi-spectral synthetic images. Quantitative experiments, carried out on three publicly available datasets, show that (i) our model is capable of generating realistic multi-spectral images of plants and (ii) the usage of such synthetic images in the training process improves the segmentation performance of state-of-the-art semantic segmentation convolutional networks.
2021
agricultural robotics; cGANs; crop/weed detection; semantic segmentation
01 Pubblicazione su rivista::01a Articolo in rivista
Multi-Spectral Image Synthesis for Crop/Weed Segmentation in Precision Farming / Fawakherji, M.; Potena, C.; Pretto, A.; Bloisi, D. D.; Nardi, D.. - In: ROBOTICS AND AUTONOMOUS SYSTEMS. - ISSN 0921-8890. - 146:(2021). [10.1016/j.robot.2021.103861]
File allegati a questo prodotto
File Dimensione Formato  
Fawakherji_Multi-Spectral_2021.pdf

solo gestori archivio

Tipologia: Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 2.62 MB
Formato Adobe PDF
2.62 MB Adobe PDF   Contatta l'autore
Fawakherji_preprint_Multi-Spectral_2021.pdf.pdf

accesso aperto

Tipologia: Documento in Pre-print (manoscritto inviato all'editore, precedente alla peer review)
Licenza: Creative commons
Dimensione 4.73 MB
Formato Adobe PDF
4.73 MB Adobe PDF

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/1678148
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 33
  • ???jsp.display-item.citation.isi??? 24
social impact