cmu.arktweetnlp.impl

Class OWLQN

java.lang.Object

cmu.arktweetnlp.impl.OWLQN

public class OWLQN
extends java.lang.Object

Class implementing the Orthant-Wise Limited-memory Quasi-Newton algorithm (OWL-QN). OWN-QN is a numerical optimization procedure for finding the optimum of an objective of the form smooth function plus L1-norm of the parameters. It has been used for training log-linear models (such as logistic regression) with L1-regularization. The algorithm is described in "Scalable training of L1-regularized log-linear models" by Galen Andrew and Jianfeng Gao. This implementation includes built-in capacity to train logistic regression or least-squares models with L1 regularization. It is also possible to use OWL-QN to optimize any arbitrary smooth convex loss plus L1 regularization by defining the function and its gradient using the supplied "DifferentiableFunction" class, and passing an instance of the function to the OWLQN object. For more information, please read the included file README.txt. Also included in the distribution are the ICML paper and slide presentation. Significant portions of this code are taken from Galen Andew's implementation

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	OWLQN.WeightsPrinter

Field Summary

Fields

Modifier and Type	Field and Description
static java.util.Set<java.lang.Integer>	biasParameters

Constructor Summary

Constructors

Constructor and Description
OWLQN()
OWLQN(boolean quiet)
OWLQN(cmu.arktweetnlp.impl.OWLQN.TerminationCriterion termCrit, boolean quiet)

Method Summary

Methods

Modifier and Type	Method and Description
int	getMaxIters()
static boolean	isConstrained()
double[]	minimize(edu.stanford.nlp.optimization.DiffFunction function, double[] initial, double l1weight, double tol, int m)
static double[]	project(double[] c) Implementation of the (Michelot 1986) technique for projecting a real vector into the feasible region K defined by the following constraints: - components must sum to 1, and - they must be nonnegative.
protected static double[]	projectWeights(double[] c) Projects the full weight vector into the feasible region where constrains are operative only on a subset of weights.
static void	setConstrained(boolean constrained) Sets constrained
static void	setConstrained(int numUnconstrainedWeights) Specify that each parameter vector will start with numUnconstrainedWeights; only the remaining weights are subject to the constraints.
void	setMaxIters(int maxIters)
void	setQuiet(boolean q)
void	setWeightsPrinting(OWLQN.WeightsPrinter printer) Specify a callback for printing the weights after each iteration of optimization.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

biasParameters

public static java.util.Set<java.lang.Integer> biasParameters

Constructor Detail

OWLQN

public OWLQN(boolean quiet)

OWLQN

public OWLQN()

OWLQN

public OWLQN(cmu.arktweetnlp.impl.OWLQN.TerminationCriterion termCrit,
     boolean quiet)

Method Detail

setQuiet

public void setQuiet(boolean q)

minimize

public double[] minimize(edu.stanford.nlp.optimization.DiffFunction function,
                double[] initial,
                double l1weight,
                double tol,
                int m)

setMaxIters

public void setMaxIters(int maxIters)

getMaxIters

public int getMaxIters()

setWeightsPrinting

public void setWeightsPrinting(OWLQN.WeightsPrinter printer)

Specify a callback for printing the weights after each iteration of optimization.

setConstrained

public static void setConstrained(boolean constrained)

Sets constrained

setConstrained

public static void setConstrained(int numUnconstrainedWeights)

Specify that each parameter vector will start with numUnconstrainedWeights; only the remaining weights are subject to the constraints. A negative value indicates that no constraints should be applied.

isConstrained

public static boolean isConstrained()

projectWeights

protected static double[] projectWeights(double[] c)

Projects the full weight vector into the feasible region where constrains are operative only on a subset of weights. The first numUnconstrainedWeights weights are preserved.

project

public static double[] project(double[] c)

Implementation of the (Michelot 1986) technique for projecting a real vector into the feasible region K defined by the following constraints: - components must sum to 1, and - they must be nonnegative. PROJECTION ALGORITHM

           Given: a real vector c to be projected into a linearly-constrained subspace
                        I ← ∅   // index set
                        x ← c
                        do
                                Compute x' = P_V_I(x): x'_i =
                                        0 if i ∈ I,
                                        x_i - (1/(dim(c)-dim(I))) * (Σ_j { x_j } - 1) otherwise
                                if x' ≥ 0: return x'
                                I ← I ∪ {i | x'_i < 0}
                                x ← P_X_I(x'): i.e. x' but with any negative components replaced by 0
                        repeat
                Output: projection of c into constrained subspace K, i.e. P_K(c)