Author: Steve Sneller

Principal Components Regression: Part 3 – The NIPALS Algorithm

Principal Components Regression: Recap of Part 2 Recall that the least squares solution to the multiple linear problem is given by (1) And that problems occurred finding when the matrix (2) was close to being singular. The Principal Components Regression approach to addressing the problem is to replace in equation (1) with a better conditioned approximation. This approximatio...
Read More

Principal Components Regression: Part 2 – The Problem With Linear Regression

This is the second part in a three part series on PCR, the first article on the topic can be found here. The Linear Regression Model Multiple Linear Regression (MLR) is a common approach to modeling the relationship between one or two or more explanatory variables and a response variable by fitting a linear equation to observed data. First let’s set up some notation. I will be rather brief, ass...
Read More

Principal Component Regression: Part 1 – The Magic of the SVD

Introduction This is the first part of a multi-part series on Principal Component Regression, or PCR for short. We will eventually end up with a computational algorithm for PCR and code it up using C# using the NMath libraries. PCR is a method for constructing a linear regression model in the case that we have a large number of predictor variables which are highly correlated. Of course, we don't ...
Read More

Contracts in .NET

Back in the early 90's I went to work for a small, progressive company that was creating and selling C++ class libraries. At this time Object-Oriented programming was still quite new and the developers at the company were a talented group of people who approached programming with a balance of academic rigor and humble pragmatism. Selling code other programmers used in their applications meant tha...
Read More

Non-negative Matrix Factorization in NMath, Part 1

A couple of years ago, we were asked by a customer to provide an implementation of an algorithm called Non-negative Matrix Factorization (NMF). We did a basic implementation, which we later included in our NMath Stats library. I kind of forgot about it until we recently heard from a prospective NMath customer who wanted to use NMF for grouping, or clustering. Talking with this customer rekindled m...
Read More