Kernelized Support Vector Machines (SVM) have gained the status of off-the-shelf classifiers, able to deliver state of the art performance on almost any problem. Still, their practical use is constrained by their computational and memory complexity, which grows super-linearly with the number of training samples. In order to retain the low training and testing complexity of linear classifiers and the exibility of non linear ones, a growing, promising alternative is represented by methods that learn non-linear classifiers through local combinations of linear ones. In this paper we propose a new multi class local classifier, based on a latent SVM formulation. The proposed classifier makes use of a set of linear models that are linearly combined using sample and class specific weights. Thanks to the latent formulation, the combination coefficients are modeled as latent variables. We allow soft combinations and we provide a closed-form solution for their estimation, resulting in an efficient prediction rule. This novel formulation allows to learn in a principled way the sample specific weights and the linear classifiers, in a unique optimization problem, using a CCCP optimization procedure. Extensive experiments on ten standard UCI machine learning datasets, one large binary dataset, three character and digit recognition databases, and a visual place categorization dataset show the power of the proposed approach.

Multiclass latent locally linear support vector machines / Fornoni, Marco; Caputo, Barbara; Orabona, Francesco. - In: JOURNAL OF MACHINE LEARNING RESEARCH. - ISSN 1532-4435. - 29:(2013), pp. 229-244. (Intervento presentato al convegno 5th Asian Conference on Machine Learning, ACML 2013 tenutosi a Canberra; Australia).

Multiclass latent locally linear support vector machines

CAPUTO, BARBARA;
2013

Abstract

Kernelized Support Vector Machines (SVM) have gained the status of off-the-shelf classifiers, able to deliver state of the art performance on almost any problem. Still, their practical use is constrained by their computational and memory complexity, which grows super-linearly with the number of training samples. In order to retain the low training and testing complexity of linear classifiers and the exibility of non linear ones, a growing, promising alternative is represented by methods that learn non-linear classifiers through local combinations of linear ones. In this paper we propose a new multi class local classifier, based on a latent SVM formulation. The proposed classifier makes use of a set of linear models that are linearly combined using sample and class specific weights. Thanks to the latent formulation, the combination coefficients are modeled as latent variables. We allow soft combinations and we provide a closed-form solution for their estimation, resulting in an efficient prediction rule. This novel formulation allows to learn in a principled way the sample specific weights and the linear classifiers, in a unique optimization problem, using a CCCP optimization procedure. Extensive experiments on ten standard UCI machine learning datasets, one large binary dataset, three character and digit recognition databases, and a visual place categorization dataset show the power of the proposed approach.
2013
5th Asian Conference on Machine Learning, ACML 2013
artificial intelligence; software; control and systems engineering; statistics and probability
04 Pubblicazione in atti di convegno::04c Atto di convegno in rivista
Multiclass latent locally linear support vector machines / Fornoni, Marco; Caputo, Barbara; Orabona, Francesco. - In: JOURNAL OF MACHINE LEARNING RESEARCH. - ISSN 1532-4435. - 29:(2013), pp. 229-244. (Intervento presentato al convegno 5th Asian Conference on Machine Learning, ACML 2013 tenutosi a Canberra; Australia).
File allegati a questo prodotto
File Dimensione Formato  
Fornoni_Multiclass_2013.pdf

accesso aperto

Tipologia: Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 610.42 kB
Formato Adobe PDF
610.42 kB Adobe PDF

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/910711
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 12
  • ???jsp.display-item.citation.isi??? ND
social impact