Accurate classification of liver nuclei based on AI models for diagnostic support of hepatocellular carcinoma

Hepatocellular carcinoma (HCC) is the most common primary liver cancer worldwide. Early diagnosis is crucial to improve management and provide survival benefits to patients with HCC. Liver biopsy coupled to histopathology remains the “gold standard”. Visual inspection of biopsy images is a crucial examination to identify tissue and cellular alterations related to HCC, but it is limited by long timelines and by the variability in interpretation. Therefore, recent advances in digital microscopy and automatic image analysis have been developed for HCC classification, thus facilitating the final diagnosis. We pointed to develop and validate Artificial Intelligence (AI) models for the recognition between normal and HCC nuclei. Materials and methods. We designed a Machine Learning (ML) and a Deep Learning (DL) for nuclei classification by using features extracted by immunofluorescence images. Results. The ML approach seeks to reduce false negatives focusing on the differences between normal and pathological conditions. It provides two parallel branches: a Support Vector Machine (SVM) trained on images that highlight healthy nuclei compared to diseased ones, and an unsupervised clustering that takes as input morphological/functional features of nuclei; their outputs are combined to increase the sensitivity of the model. The DL approach performs the classification of nuclei with a neural network whose inputs are the statistical functions extracted from the images downstream of two simultaneous pre-processing operations: wavelet transform followed by the extraction of the co-occurrence matrices, and line detection filtering followed by the extraction of run-length matrices. Both models are evaluated on new images reaching specificity, sensitivity and F1 score values equal to 98.13%, 99.8%, 98.78% in the ML model, and 99%, 82%, 88.87% in the DL. Conclusions. We demonstrated the ability of the proposed models to classify liver nuclei segmented by immunofluorescence images, thus suggesting a potential useful tool for pathologists in HCC diagnosis.