SLEP: A Sparse Learning Package

[Home] [Overview] [Feature] [Download] [Manual] [Citation]

ℓ₁-Regularized (Constrained) Sparse Learning

The ℓ₁-regularized sparse learning problem has the following general form:

min_x f(x) + λ||x||₁

Here f(.) is a convex function, x is a vector of length n, and λ>0 is a regularization parameter.

ℓ ₁/ ℓ _q-Regularized Sparse Learning (q>1)

The ℓ₁/ ℓ_q-regularized sparse learning problem has the following general form:

min_x f(x) + λ ||x||_q,1

Here f(.) is a convex function, λ>0 is a regularization parameter, and the (q,1)-norm of x is based on a (predefined) partitioning of x into a set of non-overlapping groups.

Fused Lasso

The fused Lasso problem has the following general form:

min_x f(x) + λ₁ ||x||₁ + λ₂ ∑_i |x_i-x_i+1|

Here f(.) is a convex function, λ₁, λ₂>0 are two regularization parameters, and x_i is the i-th entry of x.

Sparse Inverse Covariance Estimation

The sparse inverse covariance estimation solves the following problem:

min_Θ>0 <S,Θ> - log |Θ| + λ ||Θ||₁

Here the inverse covariance matrix Θ to be estimated is positive definite, S is the sample covariance matrix, <S,Θ> is the inner product of S and Θ, log |Θ| is the log-determinant of Θ, and λ>0 is a regularization parameter.

Sparse Group Lasso

The sparse group Lasso problem has the following general form:

min_x f(x) + λ₁ ||x||₁ + λ₂ ||x||_2,1

Here f(.) is a convex function, λ₁, λ₂>0 are two regularization parameters, and the (2,1)-norm of x is based on a (predefined) partitioning of x into a set of non-overlapping groups.

Tree Structured Group Lasso

The tree structured group Lasso problem has the following general form:

min_x f(x) + λ ||x||_tree

Here f(.) is a convex function, λ>0 is a regularization parameter, and ||x||_tree is based on a (predefined) partitioning of x into a hierarchical tree.

Overlapping Group Lasso

The overlapping group Lasso problem has the following general form:

min_x f(x) + λ ||x||_2,1

Here f(.) is a convex function, λ>0 is a regularization parameter, and the (2,1)-norm of x is based on a (predefined) partitioning of x into a set of possibly overlapping groups.

Trace Norm Regularized Learning

The trace norm regularized learning problem has the following general form:

min_x f(X) + λ ||X||_*

Here f(.) is a convex function, X is a matrix of size n by k, λ>0 is a regularization parameter, and the trace norm of X denoted as ||X||_* is defined as the summation of its singular values.

Loss Function

In the current version, we implement the following two loss functions: (1) the least squares loss and (2) the logistic loss.