Home  |  About Us  |  Link To Us  |  FAQ  |  Contact

# Fit GLM with quadratic penalty 1.0

Date Added: April 22, 2013  |  Visits: 233

Generalized linear models (GLMs) are a natural extension of linear regression models in which eta = X*w is related to y by a fixed nonlinearity and a possibly non-Gaussian noise source. Standard linear regression, logistic regression and Poisson regression are all special types of GLMs.This package fits GLMs with quadratic penalties. That is, if the negative log likelihood of the data with respoect to the model parameters is given by -log(p(y|w)), then glmfitqp solves the problem:arg min_w (-log(p(y|w)) + .5*w'*qf'w)This form of penalty naturally arises by assuming a prior on w, p(w) = N(0,qf^-1). Quadratic penalties can be used to impose that the weights are small (qf = lambda*I) or that the weights are smooth (qf = lambda*D).In general qf is only known up to a multiplicative constant lambda that determines the strength of the regularization and must be determined empirically. The function cvglmfitqp finds this optimal lambda through k-fold cross-validation. The cross-validation can be parallelized through parfor (requires parallel computing toolbox).Example use:---%%%Figure out optimal strength of prior through cross validation%Assume smoothness of the model parametersqf = blkdiag(qfsmooth1D(16),.01);rg = (-7.5:7.5)';%Simulate a model with w = Gabor functionw = exp(-rg.^2/3^2).*sin(rg*2*pi/6);nobs = 150;X = [randn(nobs,length(w)),ones(nobs,1)];r = 3*X*[w;.01];%output is binary -> logistic regressionr = binornd(1,1./(1+exp(-r)));%Set up 5-fold CVfolds = getcvfolds(length(r),5,1001);%Fit the dataclear optsopts.family = 'binomlogit';opts.lambda0 = 1;results = cvglmfitqp(r,X,qf,folds,opts);plot(results.w(1:end-1))

 Requirements: No special requirements Platforms: Matlab Keyword: Crossvalidation,  Figure,  Kfold,  Parallel,  Parallelized,  Parfor,  Requires,  Toolbox Users rating: 0/10