NMath User's Guide

TOC | Previous | Next | Index

22.4 Weighted Least Squares (.NET, C#, CSharp, VB, Visual Basic, F#)

NMath provides class DoubleCOWeightedLeastSq for solving weighted least squares (WLS) problems. WLS can modulate the importance of each observation in the final solution to correct for violations of the homoscedasticity assumption in ordinary least squares, to give less weight to outliers, or to give less weight to observations thought to be less reliable.

DoubleCOWeightedLeastSq uses a complete orthogonal decomposition technique.1 The computed solution minimizes the 2-norm of the weighted residual vector

 

 

where D is a diagonal weight matrix whose diagonal consists of the weights.

Prerequisites on the matrix A are that it has more rows than columns, and is of full rank. Note that the algorithm satisfies an accuracy bound that is not affected by ill conditioning in the weight matrix D.

Instances of DoubleCOWeightedLeastSq are constructed from a matrix of observations and a vector of weights. For example:

Code Example – C# weighted least squares

var A = new DoubleMatrix( "5x2[1 2  1 3  1 6  1 10  1 7]" );
var weights = new DoubleVector( A.Rows, .2, .2 );
var wls = new DoubleCOWeightedLeastSq( A, weights );

Code Example – VB weighted least squares

Dim A As New DoubleMatrix("5x2[1 2  1 3  1 6  1 10  1 7]")
Dim Weights As New DoubleVector(A.Rows, 0.2, 0.2)
Dim WLS As New DoubleCOWeightedLeastSq(A, Weights)

In this case, the weights are arbitrary—observations are simply given increasingly higher weights.

DoubleCOWeightedLeastSq provides a Solve() method that accepts a vector y, and computes the solution:

Code Example – C# weighted least squares

var y = new DoubleVector( "[3 6 8 10 11]" );
DoubleVector solution = wls.Solve( y );

Code Example – VB weighted least squares

Dim Y As New DoubleVector("[3 6 8 10 11]")
Dim Solution As DoubleVector = WLS.Solve(Y

Other properties and methods on DoubleCOWeightedLeastSq include:

Property A gets the original matrix of observations.

ResidualVector() returns the residual vector .

ResidualNormSqr() computes the 2-norm squared of the residual vector.

Factor() factors other matrices.

Reweight() updates the weights.

NMath provides a selection of weighting functions for use in iteratively reweighted least squares (IRLS; Section 22.5). These functions can also be used to create weights for WLS. Typical weighting functions used in IRLS are a function of the adjusted residuals from the previous iteration. For example, this code computes an ordinary least squares solution, then uses the resulting residuals to solve the same problem using WLS, downweighting the outliers:

Code Example – C# weighted least squares

// compute ordinary least squares solution
var ols = new DoubleQRLeastSq( A );
DoubleVector olsSolution = ols.Solve( y );
DoubleVector olsResiduals = ols.ResidualVector( y );

// compute weights from residuals using bisquare function
var weights = new DoubleVector( residuals.Length );
IDoubleLeastSqWeightingFunction weightingFunction =
  new DoubleBisquareWeightingFunction( A );
weightingFunction.GetWeights( olsResiduals, ref weights );

// compute weighted least squares solution
DoubleCOWeightedLeastSq wls = 
  new DoubleCOWeightedLeastSq( A, weights );
DoubleVector wlsSolution = wls.Solve( y );

Code Example – VB weighted least squares

' compute ordinary least squares solution
Dim OLS As New DoubleQRLeastSq(A)
Dim OLSSolution As DoubleVector = OLS.Solve(Y)
Dim OLSResiduals As DoubleVector = OLS.ResidualVector(Y)

' compute weights from residuals using bisquare function
Dim Weights As New DoubleVector(Residuals.Length)
Dim WeightingFunction As IDoubleLeastSqWeightingFunction =
  New DoubleBisquareWeightingFunction(A)
WeightingFunction.GetWeights(OLSResiduals, Weights)

' compute weighted least squares solution
Dim WLS As New DoubleCOWeightedLeastSq(A, Weights)
Dim WLSSolution As DoubleVector = WLS.Solve(Y)



  1. Patricia D. Hough and Stephen A. Vavasis, "Complete Orthogonal Decomposition For Weighted Least Squares", SIAM J. Matrix Anal. Appl. 18, no. 2 (April 1997): 369-392

Top

Top