**22.4****
****Weighted Least Squares** (.NET, C#, CSharp, VB, Visual Basic, F#)

**NMath**
provides class **DoubleCOWeightedLeastSq**
for solving weighted least squares (WLS)
problems. WLS can modulate the importance of each observation in the
final solution to correct for violations of the homoscedasticity assumption
in ordinary least squares, to give less weight to outliers, or to give
less weight to observations thought to be less reliable.

**DoubleCOWeightedLeastSq**
uses a complete orthogonal decomposition technique.^{1} The computed solution minimizes the 2-norm of the weighted residual
vector

where *D* is a diagonal
weight matrix whose diagonal consists of the weights.

Prerequisites on the matrix *A*
are that it has more rows than columns, and is of full rank. Note that
the algorithm satisfies an accuracy bound that is not affected by ill
conditioning in the weight matrix *D*.

Instances of **DoubleCOWeightedLeastSq**
are constructed from a matrix of observations and a vector of weights.
For example:

Code Example – C# weighted least squares

var A = new DoubleMatrix( "5x2[1 2 1 3 1 6 1 10 1 7]" ); var weights = new DoubleVector( A.Rows, .2, .2 ); var wls = new DoubleCOWeightedLeastSq( A, weights );

Code Example – VB weighted least squares

Dim A As New DoubleMatrix("5x2[1 2 1 3 1 6 1 10 1 7]") Dim Weights As New DoubleVector(A.Rows, 0.2, 0.2) Dim WLS As New DoubleCOWeightedLeastSq(A, Weights)

In this case, the weights are arbitrary—observations are simply given increasingly higher weights.

**DoubleCOWeightedLeastSq**
provides a Solve() method that accepts a
vector y, and computes the solution:

Code Example – C# weighted least squares

var y = new DoubleVector( "[3 6 8 10 11]" ); DoubleVector solution = wls.Solve( y );

Code Example – VB weighted least squares

Dim Y As New DoubleVector("[3 6 8 10 11]") Dim Solution As DoubleVector = WLS.Solve(Y

Other properties and methods on **DoubleCOWeightedLeastSq**
include:

● Property A gets the original matrix of observations.

● ResidualVector() returns the residual vector .

● ResidualNormSqr() computes the 2-norm squared of the residual vector.

● Factor() factors other matrices.

● Reweight() updates the weights.

**NMath**
provides a selection of weighting functions for use
in iteratively reweighted least squares (IRLS; Section 22.5). These functions can also be used
to create weights for WLS. Typical weighting functions used in IRLS are
a function of the adjusted residuals from the previous iteration. For
example, this code computes an ordinary least squares solution, then
uses the resulting residuals to solve the same problem using WLS, downweighting
the outliers:

Code Example – C# weighted least squares

// compute ordinary least squares solution var ols = new DoubleQRLeastSq( A ); DoubleVector olsSolution = ols.Solve( y ); DoubleVector olsResiduals = ols.ResidualVector( y ); // compute weights from residuals using bisquare function var weights = new DoubleVector( residuals.Length ); IDoubleLeastSqWeightingFunction weightingFunction = new DoubleBisquareWeightingFunction( A ); weightingFunction.GetWeights( olsResiduals, ref weights ); // compute weighted least squares solution DoubleCOWeightedLeastSq wls = new DoubleCOWeightedLeastSq( A, weights ); DoubleVector wlsSolution = wls.Solve( y );

Code Example – VB weighted least squares

' compute ordinary least squares solution Dim OLS As New DoubleQRLeastSq(A) Dim OLSSolution As DoubleVector = OLS.Solve(Y) Dim OLSResiduals As DoubleVector = OLS.ResidualVector(Y) ' compute weights from residuals using bisquare function Dim Weights As New DoubleVector(Residuals.Length) Dim WeightingFunction As IDoubleLeastSqWeightingFunction = New DoubleBisquareWeightingFunction(A) WeightingFunction.GetWeights(OLSResiduals, Weights) ' compute weighted least squares solution Dim WLS As New DoubleCOWeightedLeastSq(A, Weights) Dim WLSSolution As DoubleVector = WLS.Solve(Y)

- Patricia D. Hough and Stephen A. Vavasis, "Complete Orthogonal Decomposition For Weighted Least Squares", SIAM J. Matrix Anal. Appl. 18, no. 2 (April 1997): 369-392