NMath User's Guide

TOC | Previous | Next | Index

32.5 Nonlinear Least Squares Surface Fitting (.NET, C#, CSharp, VB, Visual Basic, F#)

NMath provides classes MultiVariableFunctionFitter and BoundedMultiVariableFunctionFitter for fitting generalized multivariable functions to a set of points. The interface is analogous to OneVariableFunctionFitter and BoundedOneVariableFunctionFitter (Section 32.4), with only a couple changes to accommodate multivariate data. Again, you must supply at least as many data points to fit as your function has parameters.

Generalized Multivariable Functions

A multivariable function takes a vector of x values, and returns a double y:

 

A generalized multivariable function additionally takes a set of parameters, p, which may appear in the function expression in arbitrary ways:

 

For example, this code computes :

Code Example – C# nonlinear least squares surface fit

public double MyFunction( DoubleVector p, DoubleVector x )
{
  return p[0] * Math.Pow( x[0], 2.0 ) * x[1] + 
         p[1] * Math.Sin( x[0] ) + 
         p[2] * Math.Pow( x[1], 3.0 );
};

Code Example – VB nonlinear least squares surface fit

Public Function MyFunction(P As DoubleVector, X As DoubleVector) As 
  Double
  Return P(0) * Math.Pow(X(0), 2.0) * X(1) +
         P(1) * Math.Sin(X(0)) +
         P(2) * Math.Pow(X(1), 3.0)
End Function

Encapsulating Generalized Multivariable Functions

In NMath, generalized multivariable functions can be encapsulated in two ways:

By extending the abstract class DoubleParameterizedFunctional, and implementing the Evaluate() method. The GradientWithRespectToParams() can also be implemented to compute the gradient with respect to the parameters; otherwise, a numerical approximation is used.

By wrapping a Func<DoubleVector, DoubleVector, double> delegate in a DoubleVectorParameterizedDelegate. An Action<DoubleVector, DoubleVector, DoubleVector> delegate can also be provided for computing the gradient with respect to the parameters; otherwise a numerical approximation is used.

For example, this code encapsulates a multivariable function using a DoubleParameterizedFunctional:

Code Example – C# nonlinear least squares surface fit

public class MyFunction : DoubleParameterizedFunctional
{
  public MyFunction()
    : base (2)
  {}

  public override double Evaluate( DoubleVector p, DoubleVector x )
  {
    // z = ayx^2 + bsin(x) + cy^3
    return p[0] * x[0] * Math.Pow( x[1], 2.0 ) + 
           p[1] * Math.Sin( x[0] ) + 
           p[2] * Math.Pow( x[1], 3.0 );
  }
}

DoubleParameterizedFunctional f = new MyFunction();

Code Example – VB nonlinear least squares surface fit

Public Class MyFunction
  Inherits DoubleParameterizedFunctional

  Sub New()
    MyBase.New(2)
  End Sub

  Public Overrides Function Evaluate(P As DoubleVector, X As 
    DoubleVector) As Double
    ' z = ayx^2 + bsin(x) + cy^3
    Return P(0) * X(0) * Math.Pow(X(1), 2.0) +
           P(1) * Math.Sin(X(0)) +
           P(2) * Math.Pow(X(1), 3.0)
  End Function

End Class

DoubleParameterizedFunctional F As New MyFunction()

This code encapsulates the same function using a DoubleVectorParameterizedDelegate:

Code Example – C# nonlinear least squares surface fit

public double MyFunction( DoubleVector p, DoubleVector x )
{
    // z = ayx^2 + bsin(x) + cy^3
    return p[0] * x[0] * Math.Pow( x[1], 2.0 ) + 
           p[1] * Math.Sin( x[0] ) + 
           p[2] * Math.Pow( x[1], 3.0 );
}

var f = new DoubleVectorParameterizedDelegate( MyFunction );

Code Example – VB nonlinear least squares surface fit

Public Function MyFunction(P As DoubleVector, X As DoubleVector) As 
  Double
  Return P(0) * Math.Pow(X(0), 2.0) * X(1) +
         P(1) * Math.Sin(X(0)) +
         P(2) * Math.Pow(X(1), 3.0)
End Function

Dim F As New DoubleVectorParameterizedDelegate( MyFunction )

Constructing a MultiVariableFunctionFitter

Class MultiVariableFunctionFitter is templatized on INonlinearLeastSqMinimizer, and class BoundedMultiVariableFunction is templatized on IBoundedNonlinearLeastSqMinimizer (Section 32.1). Instances are constructed from an encapsulated generalized multivariable function. For example:

Code Example – C# nonlinear least squares surface fit

Func<DoubleVector, DoubleVector, double> myDelegate = 
  delegate( DoubleVector p, DoubleVector x )
  {
    // z = ayx^2 + bsin(x) + cy^3
    return p[0] * x[0] * Math.Pow( x[1], 2.0 ) + 
           p[1] * Math.Sin( x[0] ) + 
           p[2] * Math.Pow( x[1], 3.0 );
  };

DoubleVectorParameterizedDelegate f =
  new DoubleVectorParameterizedDelegate( myDelegate );
 
MultiVariableFunctionFitter<TrustRegionMinimizer> fitter =
  new MultiVariableFunctionFitter<TrustRegionMinimizer>( f );

Again, an existing minimizer instance can also be passed to the constructor:

Code Example – C# nonlinear least squares surface fit

var minimizer = new LevenbergMarquardtMinimizer();
minimizer.DefaultTolerance = 1e-6;

var fitter =
  new MultiVariableFunctionFitter<LevenbergMarquardtMinimizer>( 
    f, minimizer );

Code Example – VB nonlinear least squares surface fit

Dim Minimizer As New LevenbergMarquardtMinimizer()
Minimizer.GradientTolerance = "1e-6"

Dim Fitter As New MultiVariableFunctionFitter(
  Of LevenbergMarquardtMinimizer)(F, Minimizer)

Fitting Data

Once you've constructed an instance of MultiVariableFunctionFitter or BoundedMultiVariableFunctionFitter containing a function, you can fit that function to a set of points using the Fit() method.

The Fit() method on MultiVariableFunctionFitter takes a DoubleMatrix of x values, where each row in the matrix represents a point, a DoubleVector of y values representing the data points, and a starting position in the function parameter space. For instance:

Code Example – C# nonlinear least squares surface fit

var x = new DoubleMatrix(10, 2);
x[Slice.All, 0] = new DoubleVector("3.6 7.7 9.3 4.1 8.6
                                    2.8 1.3 7.9 10.0 5.4");
x[Slice.All, 1] = new DoubleVector("16.5 150.6 263.1 24.7 208.5 
                                    9.9 2.7 163.9 325.0 54.3");

var y = new DoubleVector("95.09 23.11 60.63 48.59 89.12 
                          76.97 45.68 1.84 82.17 44.47");

var start = new DoubleVector("10 10 10");

DoubleVector solution = fitter.Fit( x, y, start );

Code Example – VB nonlinear least squares surface fit

Dim X As New DoubleMatrix(10, 2)
X(Slice.All, 0) = New DoubleVector("3.6 7.7 9.3 4.1 8.6" & _
                                    "2.8 1.3 7.9 10.0 5.4")
X(Slice.All, 1) = New DoubleVector("16.5 150.6 263.1 24.7 208.5" & 
_
                                        "9.9 2.7 163.9 325.0 54.3")

Dim Y As New DoubleVector("95.09 23.11 60.63 48.59 89.12" & _
                          "76.97 45.68 1.84 82.17 44.47")

Dim Start As New DoubleVector("10 10 10")

Dim Solution As DoubleVector = Fitter.Fit(X, Y, Start)

In the space of the function parameters, beginning at a specified start point, Fit() finds a minimum (possibly local) in the sum of the squared residuals with respect to the given x and y values.

NOTE—You must supply at least as many data points to fit as your function has parameters.

 

The Fit() method on BoundedMultiVariableFunctionFitter additionally accepts linear bounds on the solution:

Code Example – C# nonlinear least squares surface fit

var lowerBounds = new DoubleVector( "[0 -18 0]" );
var upperBounds = new DoubleVector( "[.007 -3 1]" );
DoubleVector solution =
  fitter.Fit( x, y, start, lowerBounds, upperBounds );

Code Example – VB nonlinear least squares surface fit

Dim LowerBounds As New DoubleVector("[0 -18 0]")
Dim UpperBounds As New DoubleVector("[.007 -3 1]")
Dim Solution As DoubleVector =
  Fitter.Fit(X, Y, Start, LowerBounds, UpperBounds)

Trying different initial starting points is recommended for better solutions. If possible, use starting points based on a priori information about the curve shape and the data being fit. Otherwise, random value close to zero are usually a good choice.

Fit Results

The Fit() method returns the solution found by the minimization. To compute the residuals relative to the data points at the solution, use the ResidualVector() method:

Code Example – C# nonlinear least squares surface fit

DoubleVector residuals = fitter.ResidualVector( x, y, solution );

Code Example – VB nonlinear least squares surface fit

Dim Residuals As DoubleVector =
  Fitter.ResidualVector(X, Y, solution)

Additional information about the last performed fit is available from the underlying minimizer instance, accessible using the Minimizer property. For example, this code gets the sum of the squared residuals at the starting point and at the solution, the number of iterations performed, and the stop criterion:

Code Example – C# nonlinear least squares surface fit

INonlinearLeastSqMinimizer minimizer = fitter.Minimizer;

double initialResidual = minimizer.InitialResidual;
double finalResidual = minimizer.FinalResidual;
int iterations = minimizer.Iterations;

Code Example – VB nonlinear least squares surface fit

Dim Minimizer As INonlinearLeastSqMinimizer = Fitter.Minimizer

Dim InitialResidual As Double = Minimizer.InitialResidual
Dim FinalResidual As Double = Minimizer.FinalResidual
Dim Iterations As Integer = Minimizer.Iterations


Top

Top