Taking medicines is a fundamental aspect to cure illnesses. However, studies have shown that it can be hard for patients to remember the correct posology. More aggravating, a wrong dosage generally causes the disease to worsen. Although, all relevant instructions for a medicine are summarized in the corresponding patient information leaflet, the latter is generally difficult to navigate and understand. To address this problem and help patients with their medication, in this paper we introduce an augmented reality mobile application that can present to the user important details on the framed medicine. In particular, the app implements an inference engine based on a deep neural network, i.e., a densenet, fine-tuned to recognize a medicinal from its package. Subsequently, relevant information, such as posology or a simplified leaflet, is overlaid on the camera feed to help a patient when taking a medicine. Extensive experiments to select the best hyperparameters were performed on a dataset specifically collected to address this task; ultimately obtaining up to 91.30% accuracy as well as real-time capabilities.

Medicinal Boxes Recognition on a Deep Transfer Learning Augmented Reality Mobile Application / Avola, D.; Cinque, L.; Fagioli, A.; Foresti, G. L.; Marini, M. R.; Mecca, A.; Pannone, D.. - 13231:(2022), pp. 489-499. (Intervento presentato al convegno 21st International Conference on Image Analysis and Processing, ICIAP 2022 tenutosi a Lecce, Italia) [10.1007/978-3-031-06427-2_41].

Medicinal Boxes Recognition on a Deep Transfer Learning Augmented Reality Mobile Application

Avola D.
Primo
;
Cinque L.;Fagioli A.;Foresti G. L.;Marini M. R.;Mecca A.;Pannone D.
2022

Abstract

Taking medicines is a fundamental aspect to cure illnesses. However, studies have shown that it can be hard for patients to remember the correct posology. More aggravating, a wrong dosage generally causes the disease to worsen. Although, all relevant instructions for a medicine are summarized in the corresponding patient information leaflet, the latter is generally difficult to navigate and understand. To address this problem and help patients with their medication, in this paper we introduce an augmented reality mobile application that can present to the user important details on the framed medicine. In particular, the app implements an inference engine based on a deep neural network, i.e., a densenet, fine-tuned to recognize a medicinal from its package. Subsequently, relevant information, such as posology or a simplified leaflet, is overlaid on the camera feed to help a patient when taking a medicine. Extensive experiments to select the best hyperparameters were performed on a dataset specifically collected to address this task; ultimately obtaining up to 91.30% accuracy as well as real-time capabilities.
2022
21st International Conference on Image Analysis and Processing, ICIAP 2022
Augmented reality; Convolutional neural network; Deep learning
04 Pubblicazione in atti di convegno::04b Atto di convegno in volume
Medicinal Boxes Recognition on a Deep Transfer Learning Augmented Reality Mobile Application / Avola, D.; Cinque, L.; Fagioli, A.; Foresti, G. L.; Marini, M. R.; Mecca, A.; Pannone, D.. - 13231:(2022), pp. 489-499. (Intervento presentato al convegno 21st International Conference on Image Analysis and Processing, ICIAP 2022 tenutosi a Lecce, Italia) [10.1007/978-3-031-06427-2_41].
File allegati a questo prodotto
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/1639441
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 6
  • ???jsp.display-item.citation.isi??? 4
social impact