Detection, segmentation and tracking of fruits and vegetables are three fundamental tasks for precision agriculture, enabling robotic harvesting and yield estimation applications. However, modern algorithms are data hungry and it is not always possible to gather enough data to apply the best performing supervised approaches. Since data collection is an expensive and cumbersome task, the enabling technologies for using computer vision in agriculture are often out of reach for small businesses. Following previous work in this context (Ciarfuglia et al., 2022), where we proposed an initial weakly supervised solution to reduce the data needed to get state-of-the-art detection and segmentation in precision agriculture applications, here we improve that system and explore the problem of tracking fruits in orchards. We present the case of vineyards of table grapes in southern Lazio (Italy) since grapes are a difficult fruit to segment due to occlusion, colour and general illumination conditions. We consider the case in which there is some initial labelled data that could work as source data (e.g. wine grape data), but it is considerably different from the target data (e.g. table grape data). To improve detection and segmentation on the target data, we propose to train the segmentation algorithm with a weak bounding box label, while for tracking we leverage 3D Structure from Motion algorithms to generate new labels from already labelled samples. Finally, the two systems are combined in a full semi-supervised approach. Comparisons with state-of-the-art supervised solutions show how our methods are able to train new models that achieve high performances with few labelled images and with very simple labelling.

Weakly and semi-supervised detection, segmentation and tracking of table grapes with limited and noisy data / Ciarfuglia, Thomas A.; Motoi, Ionut M.; Saraceni, Leonardo; Fawakherji, Mulham; Sanfeliu, Alberto; Nardi, Daniele. - In: COMPUTERS AND ELECTRONICS IN AGRICULTURE. - ISSN 0168-1699. - 205:(2023). [10.1016/j.compag.2023.107624]

Weakly and semi-supervised detection, segmentation and tracking of table grapes with limited and noisy data

Thomas A. Ciarfuglia
;
Ionut M. Motoi;Leonardo Saraceni;Mulham Fawakherji;Daniele Nardi
2023

Abstract

Detection, segmentation and tracking of fruits and vegetables are three fundamental tasks for precision agriculture, enabling robotic harvesting and yield estimation applications. However, modern algorithms are data hungry and it is not always possible to gather enough data to apply the best performing supervised approaches. Since data collection is an expensive and cumbersome task, the enabling technologies for using computer vision in agriculture are often out of reach for small businesses. Following previous work in this context (Ciarfuglia et al., 2022), where we proposed an initial weakly supervised solution to reduce the data needed to get state-of-the-art detection and segmentation in precision agriculture applications, here we improve that system and explore the problem of tracking fruits in orchards. We present the case of vineyards of table grapes in southern Lazio (Italy) since grapes are a difficult fruit to segment due to occlusion, colour and general illumination conditions. We consider the case in which there is some initial labelled data that could work as source data (e.g. wine grape data), but it is considerably different from the target data (e.g. table grape data). To improve detection and segmentation on the target data, we propose to train the segmentation algorithm with a weak bounding box label, while for tracking we leverage 3D Structure from Motion algorithms to generate new labels from already labelled samples. Finally, the two systems are combined in a full semi-supervised approach. Comparisons with state-of-the-art supervised solutions show how our methods are able to train new models that achieve high performances with few labelled images and with very simple labelling.
2023
Fruit detection and segmentation; Yield prediction; Computer vision; Deep learning; Self-supervised learning;
01 Pubblicazione su rivista::01a Articolo in rivista
Weakly and semi-supervised detection, segmentation and tracking of table grapes with limited and noisy data / Ciarfuglia, Thomas A.; Motoi, Ionut M.; Saraceni, Leonardo; Fawakherji, Mulham; Sanfeliu, Alberto; Nardi, Daniele. - In: COMPUTERS AND ELECTRONICS IN AGRICULTURE. - ISSN 0168-1699. - 205:(2023). [10.1016/j.compag.2023.107624]
File allegati a questo prodotto
File Dimensione Formato  
Ciarfuglia_Weakly_2023.pdf

Open Access dal 19/01/2024

Tipologia: Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza: Creative commons
Dimensione 7.49 MB
Formato Adobe PDF
7.49 MB Adobe PDF

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/1665050
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 11
  • ???jsp.display-item.citation.isi??? 6
social impact