Multi-source statistics on employment status in Italy, a machine learning approach

In recent decades, National Statistical Institutes have started to produce official statistics by exploiting multiple sources of information (multi-source statistics) rather than a single source, usually a statistical survey. In this context, one of the research projects addressed by the Italian National Statistical Institute (Istat) concerned methods for producing estimates on employment in Italy using survey data and administrative sources. The former are drawn from the Labour Force survey conducted by Istat, the latter from several administrative sources that Istat regularly acquires from external bodies. We use machine learning methods to predict the individual employment status. This approach is based on the application of decision tree and random forest techniques, that are frequently used to classify large amounts of data. We show how to construct a “new” response variable denoting agreement of the data sources: this approach is shown to maximise the information we may derive by machine learning approach in some problematic cases. The methods have been applied using the R software.