NMath User's Guide

TOC | Previous | Next | Index

32.1 Nonlinear Least Squares Interfaces (.NET, C#, CSharp, VB, Visual Basic, F#)

In NMath, classes which solve nonlinear least squares problems implement either the INonlinearLeastSqMinimizer interface or the IBoundedNonlinearLeastSqMinimizer interface.


The INonlinearLeastSqMinimizer interface provides the Minimize() method for minimizing a given function encapsulated as a DoubleMultiVariableFunction, an abstract class for representing a multivariable function. Instances override the Evaluate() method and, optionally, the Jacobian() method. If the Jacobian() method is not overriden, a central differences approximation is used to calculate the Jacobian.

For example, this code encapsulates a function that has four input variables and twenty output variables:

Code Example – C# nonlinear least squares

public class MyFunction : DoubleMultiVariableFunction
  DoubleVector yi = new DoubleVector( 20 );
  DoubleVector ti = new DoubleVector( 20 );
  DoubleVector p = new DoubleVector( 4 );

  public MyFunction() : base(4, 20)
    p[0] = -4;
    p[1] = -5;
    p[2] = 4;
    p[3] = -4;

    for ( int i = 0; i < yi.Length; i++ )
      ti[i] = i;
      yi[i] = p[2]*Math.Exp( p[0]*i ) + p[3]*Math.Exp( p[1]*i );

  public override void Evaluate(DoubleVector x, ref DoubleVector y)
    if ( x.Length != 4 || y.Length != 20 ) throw
      new InvalidArgumentException( "bad length" );

    for ( int i = 0; i < ti.Length; i++ )
      y[i] = yi[i] - x[2] * Math.Exp( x[0] * ti[i] )
                   - x[3] * Math.Exp( x[1] * ti[i] );

Code Example – VB nonlinear least squares

Public Class MyFunction
  Inherits DoubleMultiVariableFunction

  Private YI As As New DoubleVector( 20 )
  Private TI As New DoubleVector(20)
  Private P As New DoubleVector(4)

  Public Sub New()
    MyBase.New(4, 20)

    P(0) = -4
    P(1) = -5
    P(2) = 4
    P(3) = -4

    For I As Integer = 0 To YI.Length - 1
      TI(I) = I
      yi(I) = P(2) * Math.Exp(P(0) * I) + P(3) * Math.Exp(P(1) * I)

  End Sub

  Public Overrides Sub Evaluate(X As DoubleVector,
    ByRef Y As DoubleVector)

    If X.Length <> 4 Or Y.Length <> 20 Then
      Throw New InvalidArgumentException("bad length")
    End If

    For I As Integer = 0 To TI.Length - 1
      Y(I) = yi(I) - X(2) * Math.Exp(X(0) * TI(I)) - X(3) * 
        Math.Exp(X(1) * TI(I))

  End Sub

End Class

The Minimize() method takes:

the function to minimize, encapsulated as a DoubleMultiVariableFunction

the starting point

The IBoundedNonlinearLeastSqMinimizer interface extends INonlinearLeastSqMinimizer to provide an overload of the Minimize() method which also accepts lower and upper linear bounds on the solution.

Minimization Results

The Minimize() method returns the solution found by the minimization:

Code Example – C# nonlinear least squares

DoubleVector solution = minimizer.Minimize( f, start );

Code Example – VB nonlinear least squares

Dim Solution As DoubleVectorn = Minimizer.Minimize(F, Start)

Additional information about the last performed fit is available from properties in the INonlinearLeastSqMinimizer interface:

InitialResidual gets the residual associated with the starting point.

FinalResidual gets the residual associated with the last computed solution.

Iterations gets the number of iterations used in the last computed solution.

MaxIterations gets and sets the maximum number of iterations used in computing minima estimates.

MaxIterationsMet returns true if the minimum just computed stopped because the maximum number of iterations was reached; otherwise, false.

For example:

Code Example – C# nonlinear least squares

double initialResidual = minimizer.InitialResidual;
double finalResidual = minimizer.FinalResidual;
int iterations = minimizer.Iterations;

Code Example – VB nonlinear least squares

Dim InitialResidual As Double = Minimizer.InitialResidual
Dim FinalResidual As Double = Minimizer.FinalResidual
Dim Iterations As Integer = Minimizer.Iterations


NMath provides two implementations of the nonlinear least squares interfaces:

Class TrustRegionMinimizer (Section 32.2) solves both constrained and unconstrained nonlinear least squares problems using the Trust-Region method, and implements the IBoundedNonlinearLeastSqMinimizer interface.

Class LevenbergMarquardtMinimizer (Section 32.3) solves nonlinear least squares problems using the Levenberg-Marquardt method, and implements the INonlinearLeastSqMinimizer interface.