UMR CNRS 7253

Antoine Bordes
Antoine Bordes
Antoine Bordes
Antoine Bordes
Antoine Bordes

Site Tools


en:sgdqn

Introduction

LibSGDQN proposes an implementation of SGD-QN, a carefully designed quasi-Newton stochastic gradient descent solver for linear SVMs. The SGD-QN algorithm is a stochastic gradient descent algorithm that makes careful use of second order information and splits the parameter update into independently scheduled components. Thanks to this design, SGD-QN iterates nearly as fast as a first order stochastic gradient descent but requires less iterations to achieve the same accuracy.

This algorithm is extensively described in the paper: “SGD-QN: Careful Quasi-Newton Stochastic Gradient Descent” by A. Bordes, L. Bottou and P. Gallinari.

Along with SGD-QN, this library proposes the implementation of two other online solvers for linear SVMs: SGD2, a standard first-order stochastic gradient descent (implemented by Leon Bottou) and oLBFGS (see the paper [Schraudolph et al., 07]). A script to re-run the experiments of the JMLR paper is also provided.

How-to

We provide an implementation of SGD-QN under the GNU Public License.

Description : The source code can be downloaded here: libsgdqn-1.1.tar.gz. All three algorithms are implemented in C++ and does not require any additional library. Their codes are in the 'libsgdqn/svm/' directory.

Compilation : (1) cd to 'libsgdqn/svm/' and (2) simply type make. Binaries: the compilation produces 2 binaries for each algorithm. A version for dense data and a version handling sparse data (with the prefix 'sparse_').

Command line : typing the command with no argument print a short help. The binaries can take up to 3 files as arguments:

  • trainfile (required): the training set in LibSVM/SVMlight format.
  • testfile (optional): the test set in the same format.
  • logfile (optional): a file in which are written intermediate values of train time, train cost, train error, test cost and test error.

Input file format : training and testing sets must be in LibSVM/SVMLight format and gziped.


User Tools