UMR CNRS 7253

Antoine Bordes
Antoine Bordes
Antoine Bordes
Antoine Bordes
Antoine Bordes

Site Tools


en:sgdqn

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

en:sgdqn [2013/04/16 11:41]
bordesan created
en:sgdqn [2013/04/16 11:43] (current)
bordesan
Line 3: Line 3:
 LibSGDQN proposes an implementation of ''​SGD-QN'',​ a carefully designed quasi-Newton stochastic gradient descent solver for linear SVMs. The ''​SGD-QN''​ algorithm is a stochastic gradient descent algorithm that makes careful use of second order information and splits the parameter update into independently scheduled components. Thanks to this design, SGD-QN iterates nearly as fast as a first order stochastic gradient descent but requires less iterations to achieve the same accuracy. LibSGDQN proposes an implementation of ''​SGD-QN'',​ a carefully designed quasi-Newton stochastic gradient descent solver for linear SVMs. The ''​SGD-QN''​ algorithm is a stochastic gradient descent algorithm that makes careful use of second order information and splits the parameter update into independently scheduled components. Thanks to this design, SGD-QN iterates nearly as fast as a first order stochastic gradient descent but requires less iterations to achieve the same accuracy.
  
-This algorithm is extensively described in the paper: "​SGD-QN:​ Careful Quasi-Newton Stochastic Gradient Descent"​ by A. Bordes, L. Bottou and P. Gallinari. See the paper [[abstract_sgdqn|[bordes et al., 09] ]] for more details.+This algorithm is extensively described in the paper: "​SGD-QN:​ Careful Quasi-Newton Stochastic Gradient Descent"​ by A. Bordes, L. Bottou and P. Gallinari. ​
  
 Along with ''​SGD-QN'',​ this library proposes the implementation of two other online solvers for linear SVMs: ''​SGD2'',​ a standard first-order stochastic gradient descent (implemented by [[http://​leon.bottou.org/​projects/​sgd|Leon Bottou]]) and ''​oLBFGS''​ (see the paper [[http://​scholar.google.fr/​scholar?​hl=fr&​lr=&​cluster=4237024506956348633|[Schraudolph et al., 07]]]). A script to re-run the experiments of the JMLR paper is also provided. ​ Along with ''​SGD-QN'',​ this library proposes the implementation of two other online solvers for linear SVMs: ''​SGD2'',​ a standard first-order stochastic gradient descent (implemented by [[http://​leon.bottou.org/​projects/​sgd|Leon Bottou]]) and ''​oLBFGS''​ (see the paper [[http://​scholar.google.fr/​scholar?​hl=fr&​lr=&​cluster=4237024506956348633|[Schraudolph et al., 07]]]). A script to re-run the experiments of the JMLR paper is also provided. ​
Line 12: Line 12:
 We provide an implementation of ''​SGD-QN''​ under the [[http://​www.fsf.org/​licensing/​licenses/​gpl.html|GNU Public License]]. We provide an implementation of ''​SGD-QN''​ under the [[http://​www.fsf.org/​licensing/​licenses/​gpl.html|GNU Public License]].
  
-** Description **: The source code can be downloaded here: {{libsgdqn-1.1.tar.gz|}}. All three algorithms are implemented in C++ and does not require any additional library. ​ Their codes are in the '​libsgdqn/​svm/'​ directory.+** Description **: The source code can be downloaded here: {{en:libsgdqn-1.1.tar.gz|libsgdqn-1.1.tar.gz}}. All three algorithms are implemented in C++ and does not require any additional library. ​ Their codes are in the '​libsgdqn/​svm/'​ directory.
  
 ** Compilation **: (1) cd to '​libsgdqn/​svm/'​ and (2) simply type make.  ​ ** Compilation **: (1) cd to '​libsgdqn/​svm/'​ and (2) simply type make.  ​
Line 24: Line 24:
  
 ** Input file format **: training and testing sets must be in LibSVM/​SVMLight format and gziped. ** Input file format **: training and testing sets must be in LibSVM/​SVMLight format and gziped.
- 

User Tools