Methodological approach for the automatic discrimination of pictorial materials using fused hyperspectral imaging data from the visible to mid-infrared range coupled with machine learning methods

Hyperspectral imaging represents a powerful tool for the study of artwork’s materials since it permits to obtain simultaneously information about the spectral behavior of the materials and their spatial distribution. By combining hyperspectral images performed on several spectral intervals (visible, near infrared and mid-infrared ranges) through chemometric methods it is possible to clearly identify most of the materials used in painting (i.e., pigments, dyes, varnishes, and binders). Moreover, in the last decade, the development of machine learning algorithms coupled with comprehensive and continuously updated databases opens new perspective on the automatic recognition of pictorial materials. In this work, we propose a novel procedure to support the automatic discrimination of pictorial materials consisting in a mid-level data fusion on imaging datasets coming from two commercial hyperspectral cameras, in the 400–1000 nm and 1000–2500 nm spectral ranges, respectively, and a MAcroscopic Fourier Transform InfRared scanning in reflection mode (MA-rFTIR), in the 7000 to 350 cm-1 (1428 nm − 28 μm) spectral range. The automatic recognition of 102 pictorial mock-ups from the fused data is performed by testing the performance of ECOC-SVM (error-correcting output coding and support vector machine) model obtaining a good predictive result with only few pixels that are confused with other classes. The methodology described in this paper demonstrates that an accurate paint layer multiclass recognition is feasible, and the use of chemometric approaches solves some challenges involving the study of materials.