Author: Steve Sneller

Principal Components Regression: Part 3 – The NIPALS Algorithm

Principal Components Regression: Recap of Part 2 Recall that the least squares solution to the multiple linear problem is given by(1) $latex \hat{\beta} = (X^T X)^{-1} X^T y $ And that problems occurred finding when the matrix(2) was close to being singular. The Principal Components Regression approach to addressing the problem is to replace in equation (1) with a better conditioned...
Read More

Principal Components Regression: Part 2 – The Problem With Linear Regression

This is the second part in a three part series on PCR, the first article on the topic can be found here. The Linear Regression Model Multiple Linear Regression (MLR) is a common approach to modeling the relationship between one or two or more explanatory variables and a response variable by fitting a linear equation to observed data. First let’s set up some notation. I will be rather brief, ass...
Read More

Principal Component Regression: Part 1 – The Magic of the SVD

Introduction This is the first part of a multi-part series on Principal Component Regression, or PCR for short. We will eventually end up with a computational algorithm for PCR and code it up using C# using the NMath libraries. PCR is a method for constructing a linear regression model in the case that we have a large number of predictor variables which are highly correlated. Of course, we don't ...
Read More

Contracts in .NET

Back in the early 90's I went to work for a small, progressive company that was creating and selling C++ class libraries. At this time Object-Oriented programming was still quite new and the developers at the company were a talented group of people who approached programming with a balance of academic rigor and humble pragmatism. Selling code other programmers used in their applications meant tha...
Read More

Non-negative Matrix Factorization in NMath, Part 1

A couple of years ago, we were asked by a customer to provide an implementation of an algorithm called Non-negative Matrix Factorization (NMF). We did a basic implementation, which we later included in our NMath Stats library. I kind of forgot about it until we recently heard from a prospective NMath customer who wanted to use NMF for grouping, or clustering. Talking with this customer rekindled m...
Read More