As previously described, NMath 5.1 and NMath Stats 3.4 include classes for plotting NMath types using the Microsoft Chart Controls for .NET. (Free adapter code is also available for using NMath with Syncfusion Essential Chart.) Let’s use Anscombe’s data to explore how NMath’s new visualization capabilities can be used to reveal the differences in the data sets.

First, we’ll load the data into a DoubleMatrix.

DoubleMatrix A = new DoubleMatrix( @"11x8 [
  10.0 8.04  10.0 9.14 10.0 7.46  8.0  6.58
  8.0  6.95  8.0  8.14 8.0  6.77  8.0  5.76
  13.0 7.58  13.0 8.74 13.0 12.74 8.0  7.71
  9.0  8.81  9.0  8.77 9.0  7.11  8.0  8.84
  11.0 8.33  11.0 9.26 11.0 7.81  8.0  8.47
  14.0 9.96  14.0 8.10 14.0 8.84  8.0  7.04
  6.0  7.24  6.0  6.13 6.0  6.08  8.0  5.25
  4.0  4.26  4.0  3.10 4.0  5.39  19.0 12.50
  12.0 10.84 12.0 9.13 12.0 8.15  8.0  5.56
  7.0  4.82  7.0  7.26 7.0  6.42  8.0  7.91
  5.0  5.68  5.0  4.74 5.0  5.73  8.0  6.89 ]" );

Now let’s perform some simple descriptive statistics.

 int groups = 4;
 Slice rows = Slice.All;
 Slice xCols = new Slice( 0, groups, 2 );
 Slice yCols = new Slice( 1, groups, 2 );
 double unbiased = (double)A.Rows / ( A.Rows - 1 );
 
 Console.WriteLine( "Mean of x: {0}",
   NMathFunctions.Mean( A[ rows, xCols ] ) );
 Console.WriteLine( "Variance of x: {0}",
   NMathFunctions.Variance( A[rows, xCols] ) * unbiased );
 Console.WriteLine( "Mean of y: {0}",
   NMathFunctions.Round( NMathFunctions.Mean( A[rows, yCols] ), 2 ) );
 Console.WriteLine( "Variance of y: {0}",
   NMathFunctions.Round(
     NMathFunctions.Variance( A[rows, yCols] ) * unbiased, 3 ) );
 
 Console.Write( "Correlation of x-y: " );
 for (int i = 0; i < A.Cols; i += 2 )
 {
   Console.Write( NMathFunctions.Round(
    StatsFunctions.Correlation( A.Col(i), A.Col(i + 1) ), 3 ) + " " );
 }
 Console.WriteLine();

You can see from the output that the statistics are nearly identical for all four data sets:

Mean of x: [ 9 9 9 9 ]
Variance of x: [ 11 11 11 11 ]
Mean of y: [ 7.5 7.5 7.5 7.5 ]
Variance of y: [ 4.127 4.128 4.123 4.123 ]
Correlation of x-y: 0.816 0.816 0.816 0.817

Now let’s fit a linear model to each data set.

 LinearRegression[] lrs = new LinearRegression[groups];
 
 for( int i = 0; i < groups; i ++ )
 {
   Console.WriteLine( "Group {0}", i + 1 );
 
   bool addIntercept = true;
   lrs[i] = new LinearRegression( new DoubleMatrix( A.Col( 2 * i ) ),
     A.Col( 2 * i + 1 ), addIntercept );
   Console.WriteLine( "equation of regression line: Y = {0} + {1}X",
     Math.Round( lrs[i].Parameters[0], 2 ),
     Math.Round( lrs[i].Parameters[1], 3 ) );
 
   LinearRegressionParameter param =
     new LinearRegressionParameter( lrs[i], 1 );
   Console.WriteLine( "standard error of estimate of slope: {0}",
     Math.Round( param.StandardError, 3 ) );
   Console.WriteLine( "t-statistic: {0}",
     Math.Round( param.TStatistic( 0 ), 2 ) );
 
   LinearRegressionAnova anova = new LinearRegressionAnova( lrs[i] );
   Console.WriteLine( "regression sum of squares: {0}",
     Math.Round( anova.RegressionSumOfSquares, 2 ) );
   Console.WriteLine( "residual Sum of squares: {0}",
     Math.Round( anova.ResidualSumOfSquares, 2 ) );
   Console.WriteLine( "r2: {0}", Math.Round( anova.RSquared, 2 ) );
 
   Console.WriteLine();
 }

Again, the output is nearly identical for each data set:

Group 1
equation of regression line: Y = 3 + 0.5X
standard error of estimate of slope: 0.118
t-statistic: 4.24
regression sum of squares: 27.51
residual Sum of squares: 13.76
r2: 0.67

Group 2
equation of regression line: Y = 3 + 0.5X
standard error of estimate of slope: 0.118
t-statistic: 4.24
regression sum of squares: 27.5
residual Sum of squares: 13.78
r2: 0.67

Group 3
equation of regression line: Y = 3 + 0.5X
standard error of estimate of slope: 0.118
t-statistic: 4.24
regression sum of squares: 27.47
residual Sum of squares: 13.76
r2: 0.67

Group 4
equation of regression line: Y = 3 + 0.5X
standard error of estimate of slope: 0.118
t-statistic: 4.24
regression sum of squares: 27.49
residual Sum of squares: 13.74
r2: 0.67

Finally, let’s use the new NMath charting functionality to plot each linear regression fit. Note that we make use of the Compose() method to combine multiple charts into a single composite Chart control.

 List<Chart> charts = new List<Chart>();
 for( int i = 0; i < lrs.Length; i++ )
 {
   charts.Add( NMathStatsChart.ToChart( lrs[i], 0 ) );
 }
 Chart all = NMathStatsChart.Compose( charts, 2, 2,
   NMathChart.AreaLayoutOrder.RowMajor );
 for( int i = 0; i < groups; i++ )
 {
   all.ChartAreas[i].AxisX.Title = "x" + ( i + 1 );
   all.ChartAreas[i].AxisX.Minimum = 2;
   all.ChartAreas[i].AxisX.Maximum = 22;
   all.ChartAreas[i].AxisX.Interval = 4;
 
   all.ChartAreas[i].AxisY.Title = "y" + ( i + 1 );
   all.ChartAreas[i].AxisY.Minimum = 2;
   all.ChartAreas[i].AxisY.Maximum = 14;
   all.ChartAreas[i].AxisY.Interval = 4;
 
   all.Series[2 * i].Color = Color.DarkOrange;
   all.Series[2 * i + 1].Color = Color.SteelBlue;
 }
 NMathStatsChart.Show( all );

The charts reveal  dramatic differences between the data sets, despite the identical fitted models. Group 1 shows a linear relationship, while in Group 2 the releationship is clearly non-linear.  Groups 3 and 4 demonstrate how a single outlier can have a large effect on simple statistics.

Ken

References

Anscombe, F. J. (1973). “Graphs in Statistical Analysis”. American Statistician 27 (1): 17–21. JSTOR 2682899.
Tufte, Edward R. (2001). The Visual Display of Quantitative Information (2nd ed.). Cheshire, CT: Graphics Press

These were performed on a DoubleVector, v,  of length 10,000,000.

1) Create a new vector. This isn’t really clearing out an existing vector but we thought we should include it for completeness.

 DoubleVector v2 = new DoubleVector( v.Length, 0.0 );

The big drawback here is that you’re creating new memory. Time: 419.5ms

2) Probably the first thing to come to mind is to simply iterate through the vector and set everything to zero.

for ( int i = 0; i < v.Length; i++ )
{
  v[i] = 0.0;
}

We have to do some checking in the index operator. No new memory created. Time: 578.5ms

3) In some cases, you could iterate through the underlying array of data inside the DoubleVector.

for ( int i = 0; i < v.DataBlock.Data.Length; i++ )
{
  v.DataBlock.Data[i] = 0.0;
}

This is a little less intuitive. And, very importantly, it will not work with many views into other data structures. For example, a row slice of a matrix. However, it’s easier for the CLR to optimize this loop. Time: 173.5ms

4) We can use the power of Intel’s MKL to multiply the vector by zero.

 v.Scale( 0.0 );

Scale() does this in-place. No new memory is created. In this example, we assume that MKL has already been loaded and is ready to go which is true if another MKL-based NMath call was already made or if NMath was initialized. This method will work on all views of other data structures. Time: 170ms

5) This surprised us a bit but the best method we could find was to clear out the underlying array using Array.Clear() in .NET

 Array.Clear( v.DataBlock.Data, 0, v.DataBlock.Data.Length );

This creates no new memory. However, this will not work with non-contiguous views. However, this method is very fast. Time: 85.8ms

To make this much simpler, we have created a Clear() method and a Clear( Slice) method on vectors and matrices. It will do the right thing in the right circumstance. It will be released in NMath 5.2 in 2012.

And, here is the code we used for this test:

using System;
using CenterSpace.NMath.Core;

namespace Test
{
  class ClearVector
  {
    static int size = 100000000;
    static int runs = 10;
    static int methods = 5;

    static void Main( string[] args )
    {
      System.Diagnostics.Stopwatch sw = new System.Diagnostics.Stopwatch();
      DoubleMatrix times = new DoubleMatrix( runs, methods );
      NMathKernel.Init();

      for ( int run = 0; run < runs; run++ )
      {
        Console.WriteLine( "Run {0}...", run );
        DoubleVector v = null;

        // Create a new one
        v = new DoubleVector( size, 1.0, 2.0 );
        sw.Start();
        DoubleVector v2 = new DoubleVector( v.Length, 0.0 );
        sw.Stop();
        times[run, 0] = sw.ElapsedMilliseconds;
        Console.WriteLine( Assert( v2 ) );

        // iterate through vector
        v = new DoubleVector( size, 1.0, 2.0 );
        sw.Reset();
        sw.Start();
        for ( int i = 0; i < v.Length; i++ )
        {
          v[i] = 0.0;
        }
        sw.Stop();
        times[run, 1] = sw.ElapsedMilliseconds;
        Console.WriteLine( Assert( v ) );

        // iterate through array
        v = new DoubleVector( size, 1.0, 2.0 );
        sw.Reset();
        sw.Start();
        for ( int i = 0; i < v.DataBlock.Data.Length; i++ )
        {
          v.DataBlock.Data[i] = 0.0;
        }
        sw.Stop();
        times[run, 2] = sw.ElapsedMilliseconds;
        Console.WriteLine( Assert( v ) );

        // scale
        v = new DoubleVector( size, 1.0, 2.0 );
        sw.Reset();
        sw.Start();
        v.Scale( 0.0 );
        sw.Stop();
        times[run, 3] = sw.ElapsedMilliseconds;
        Console.WriteLine( Assert( v ) );

        // Array Clear
        v = new DoubleVector( size, 1.0, 2.0 );
        sw.Reset();
        sw.Start();
        Array.Clear( v.DataBlock.Data, 0, v.DataBlock.Data.Length );
        sw.Stop();
        times[run, 4] = sw.ElapsedMilliseconds;
        Console.WriteLine( Assert( v ) );
        Console.WriteLine( times.Row( run ) );
      }
      Console.WriteLine( "Means: " + NMathFunctions.Mean( times ) );
    }

    private static bool Assert( DoubleVector v )
    {
      if ( v.Length != size )
      {
        return false;
      }
      for ( int i = 0; i < v.Length; ++i )
      {
        if ( v[i] != 0.0 )
        {
          return false;
        }
      }
      return true;
    }
  }
}

- Trevor

There is a one-time delay when the appropriate x86 or x64 native code is loaded. This cost can be easily controlled by the developer by using the NMathKernel.Init() method. Please see Initializing NMath for more details.

- Trevor

Dr. Fabian M. Uriarte, Research Associate at UT-CEM, recently had this to say about NMath:

“If performance, documentation, and reliable technical support are important to you, then NMath is what you need. In our application, we parallelize the solution of large, sparse Ax=b systems. Using NMath, we saw significant performance gains over Math.NET. Additionally, NMath has a very large set of classes that suit our needs in other numerical aspects as well. We are extremely pleased with the performance of NMath and highly recommend it to others. This library is a ‘must have.’"

Thanks for the kind words, Fabian!  You can check out the CEM Solver project at http://www.utexas.edu/research/cem/projects/CEM_solver.html, or check out the demonstration video below.

In an earlier post, we described how NMath types can be plotted using the Microsoft Chart Controls for .NET, but programmatically constructing Chart objects with data series bound to NMath objects. The latest release of the NMath libraries makes this process much simpler by providing adapter classes which easily construct basic charts for you from NMath types:

• NMath 5.1 includes assembly NMathChartMicrosoft.dll containing class NMathChart, which provides convenience methods for plotting NMath types using the Microsoft Chart Controls
• NMath Stats 3.4 includes assembly NMathStatsChartMicrosoft.dll containing NMathStatsChart which provides convenience methods for plotting NMath Stats types using the Microsoft Chart Controls

The generated charts can then be customized to meet your needs.

Using the Chart Adapters

The Microsoft Chart Controls for .NET are available as a separate download for .NET 3.5. Beginning in .NET 4.0, the Chart controls are part of the .NET Framework. To use the Chart controls, add a reference to System.Windows.Forms.DataVisualization.
To use the adapters, add a reference to NMathChartMicrosoft.dll and/or NMathStatsChartMicrosoft.dll, and using statement:

using CenterSpace.NMath.Charting.Microsoft;

A previous blog article provides a simple app.config solution for deploying your chart dependent applications to target both .NET 3.5 or .NET 4.0. Without this app.config modification, your application will only target .NET 3.5.

The Adapter API

Both NMathChart and NMathStatsChart classes share a common API. Overloads of the ToChart() function are provided for our common math and stats types. ToChart() returns an instance of System.Windows.Forms.DataVisualization.Charting.Chart, which can be customized as desired. For example

Polynomial poly = new Polynomial( new DoubleVector( 4, 2, 5, -2, 3 ) );
Chart chart = NMathChart.ToChart( poly, -1, 1 );
chart.Titles.Add("Hello World");

For prototyping and debugging console applications, the Show() function shows a given chart in a default form.

NMathChart.Show( chart );

Note that when the window is closed, the chart is disposed.

If you do not need to customize the chart, overloads of Show() are also provided for common NMath types.

NMathChart.Show( poly );

This is equivalent to calling:

NMathChart.Show( NMathChart.ToChart( poly ) );

The Save() function saves a chart to a file or stream.

NMathChart.Save( chart, "chart.png", ChartImageFormat.Png );

If you are developing a Windows Forms application using the Designer, add a Microsoft Chart control to your form, then update it with an NMath object using the appropriate Update() function after initialization.

public Form1()
{
  InitializeComponent();
 
  Polynomial poly =
    new Polynomial( new DoubleVector( 4, 2, 5, -2, 3 ) );
  NMathChart.Update( ref this.chart1, poly, -1, 1 );
}

This has the following effect on your existing Chart object:

• a new, default ChartArea is added if one does not exist, otherwise chart.ChartAreas[0] is used
• axis titles, and DefaultAxisTitleFont and DefaultMajorGridLineColor, only have an effect if a new ChartArea is added
• titles are added only if the given Chart does not already contain any titles
• chart.Series[0] is replaced, or added if necessary

Charting Examples

Here are a few examples of usage in C# (Of course, as with all NMath functionality, these examples will also work, given the necessary syntactic changes, from VB.NET or F#, or any other .NET language.) :

double xmin = -1;
double xmax = 1;
int numInterpolatedValues = 100;
Polynomial poly = new Polynomial( new DoubleVector( 4, 2, 5, -2, 3 ) );
NMathChart.Show( poly, xmin, xmax, numInterpolatedValues );

DoubleVector x = new DoubleVector( 100, 0, 0.1 );
DoubleVector cos = x.Apply( NMathFunctions.CosFunction );
DoubleVector sin = x.Apply( NMathFunctions.SinFunction );
 
DoubleVector[] data = new DoubleVector[] { cos, sin };
NMathChart.Unit unit = new NMathChart.Unit( 0, 0.1, "x" );
Chart chart = NMathChart.ToChart( data, unit );
 
chart.Series[0].Name = "cos(x)";
chart.Series[1].Name = "sin(x)";
NMathChart.Show( chart );

double lambda = 4.0;
PoissonDistribution poisson = new PoissonDistribution( lambda );
 
NMathStatsChart.Show( poisson,
  NMathStatsChart.DistributionFunction.PDF );

DoubleMatrix predictors =
  new DoubleMatrix( 100, 3, new RandGenUniform() ); ;
DoubleVector observations = 2 * predictors.Col(0) +
  -0.75 * predictors.Col(1) + 1.25 * predictors.Col(2) + 0.5;
bool addIntercept = true;
 
LinearRegression lr =
  new LinearRegression( predictors, observations, addIntercept );
NMathStatsChart.Show( lr, 2 );

More Information

For more information, and many more examples, see:

• Whitepaper: NMath Visualization Using the Microsoft Chart Controls
• Whitepaper: NMath Stats Visualization Using the Microsoft Chart Controls
• the API docs for the CenterSpace.NMath.Charting.Microsoft namespace

<configuration>
   ...
   <runtime>
      ...
      <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
         ...
         <dependentAssembly>
            <assemblyIdentity name="System.Windows.Forms.DataVisualization"
              publicKeyToken="31bf3856ad364e35"/>
            <bindingRedirect oldVersion="3.5.0.0-3.5.0.0" newVersion="4.0.0.0"/>
         </dependentAssembly>
         ...
      </assemblyBinding>
      ...
   </runtime>
   ...
</configuration>

In this way, your application with charts can be deployed to both .NET 3.5 and .NET 4.0. Please note that all other NMath assemblies (with non-visual functionality) can be used in .NET 3.5 and .NET 4.0 without this step.

See this tutorial for more information.

Version 5.1 of NMath adds:

• Assembly NMathChartMicrosoft.dll containing class NMathChart, which provides static methods for plotting NMath types using the Microsoft Chart Controls for .NET. [More]

• Classes for solving first order initial value differential equations by the Runge-Kutta method. Class FirstOrderInitialValueProblem encapsulates a first order initial value differential equation, and class RungeKuttaSolver solves them. [More]
• Exponential moving average filter weights to class MovingWindowFilter.

A complete changelog is located here.

Version 3.4 of NMath Stats adds:

• Assembly NMathStatsChartMicrosoft.dll containing class NMathStatsChart, which provides static methods for plotting NMath Stats types using the Microsoft Chart Controls for .NET. [More]

• Class TwoSampleUnpairedUnequalTTest. Unlike TwoSampleUnpairedTTest, a pooled estimate of the variance is not used.

A complete changelog is located here.

The new release is build-compatible with the prior release. Upgrades are provided free of charge to customers with current annual maintenance contracts. Maintenance contracts are available through our webstore.

This release features simplified NMath and NMath Stats installers. Updated license keys will be issued at upgrade time. A license file must now be deployed with your NMath applications. [More]

For instance, let’s try fitting a circle to a set of points. A circle is defined by three parameters: a center (a,b), and a radius r. The circle is all points such that (x-a)^2 + (y – b)^2 = r^2. To setup the minimization problem, we first need to define a function which given a set of circle parameters, returns the residuals with respect to a cloud of 2D points. There are several methods for doing this in NMath. In the following C# code, we extend the abstract base class DoubleMultiVariableFunction, and override the Evaluate() method:

public class CircleFitFunction : DoubleMultiVariableFunction
{
 
  public CircleFitFunction( DoubleVector x, DoubleVector y )
    : base( 3, x.Length )
  {
    if( x.Length != y.Length )
      throw new Exception( "Unequal number of x,y values." );
 
    X = x;
    Y = y;
  }
 
  public DoubleVector X { get; internal set; }
  public DoubleVector Y { get; internal set; }
 
  public override void Evaluate( DoubleVector parameters, ref DoubleVector residuals )
  {
    // parameters of circle with center (a,b) and radius r
    double a = parameters[0];
    double b = parameters[1];
    double r = parameters[2];
 
    for( int i = 0; i < X.Length; i++ )
    {
      // distance of point from circle center
      double d = Math.Sqrt( Math.Pow( X[i] - a, 2.0 ) + Math.Pow( Y[i] - b, 2.0 ) );
 
      residuals[i] = d - r;
    }
  }
}

The constructor takes the set of x,y points to fit, which are stored on the class. Note that we call the base constructor with the dimensions of the domain and range of the function:

: base( 3, x.Length )

In this case, the function maps the 3-dimensional space of the circle parameters to the x.Length-dimension space of the residuals.

Next we override the Evaluate() method, which takes an input vector and output vector passed by reference. Our implementation computes the residual for each point with respect to a circle defined by the given parameters. We calculate the distance, d, of each point from the circle center; the residual is then equal to d – r. The nonlinear least squares method will minimize the L2 norm of this function.

To demonstrate the fitting process, let’s first start with a set of points generated from a circle of known parameters. For example, this C# code creates 20 x,y points evenly spaced around a circle with center (0.5, 0.25) and radius 2, plus some added noise:

int n = 20;
DoubleVector x = new DoubleVector( n );
DoubleVector y = new DoubleVector( n );
 
double a = 0.5;
double b = 0.25;
double r = 2;
 
RandGenUniform noise = new RandGenUniform( -.75, .75 );
for( int i = 0; i < n; i++ )
{
  double t = i * 2 * Math.PI / n;
  x[i] = a + r * Math.Cos( t ) + noise.Next();
  y[i] = b + r * Math.Sin( t ) + noise.Next();
}

Now that we have our function defined and some points to fit, performing the minimization takes only a few lines of code. This C# code finds the minimum and prints the solution and final residual:

CircleFitFunction f = new CircleFitFunction( x, y );
TrustRegionMinimizer minimizer = new TrustRegionMinimizer();
DoubleVector start = new DoubleVector( "0.1 0.1 0.1" );
DoubleVector solution = minimizer.Minimize( f, start );
 
Console.WriteLine( "solution = " + solution );
Console.WriteLine( "final residual = " + minimizer.FinalResidual );

Sample output:

solution = [ 0.446122611523468 0.175483486509012 1.93511389538286 ]
final residual = 1.62414259527024

Starting from point (0.1, 0.1, 0.1) in the parameter space of the circle, we minimized the sum of the squared residuals with respect to the (x,y) points. In the run above, the minimum found was center (0.45, 0.18) and radius 1.9, close to the actual parameters which generated our (noisy) data. To visually inspect the fit, we can use NMath with the Microsoft Chart Controls for .NET.

Now that we’re sure the minimization is working as desired, let’s try it again with some random points:

int n = 10;
RandGenUniform rng = new RandGenUniform( -1, 2 );
DoubleVector x = new DoubleVector( n, rng );
DoubleVector y = new DoubleVector( n, rng );
 
CircleFitFunction f = new CircleFitFunction( x, y );
TrustRegionMinimizer minimizer = new TrustRegionMinimizer();
DoubleVector start = new DoubleVector( "0.1 0.1 0.1" );
DoubleVector solution = minimizer.Minimize( f, start );

Again, plotting the solution:

In some runs, the fit may seem counter-intuitive. For example:

But if you think about it, this sort of fit makes perfect sense, given how we defined our fit function. We’re asking the minimizer to minimize the residuals with respect to the circle perimeter, and allowing it vary the circle center and radius however it can to achieve that goal. In an extreme case, if our points fall approximately along a line, the best fit will be a circle with a very large radius, so that the circle perimeter is nearly linear as it passes through the points.

In this case, the fitted circle has center (428.0, -757.8) and radius 870.6.  If we wish to avoid solutions such as these, we could define our fit function differently. For example, we might constrain the circle center to be the center of mass of the points, and vary only the radius.

Obviously, a similar technique can be used to fit other geometric primitives, though as the complexity of the shape increases, so does the complexity of the fit function.

Ken

There are a myriad of ways SQL Server can be setup and for this post, we took whatever shortcut we could find. Our goal was to get this running and worry about doing it “right” later. We found ourselves on a fairly steep learning curve without a lot of examples to work off of. Consider this more of a “proof of concept” post than the definitive “how to”.

As mentioned above there are lot of different approaches to this problem. In our minds, we wanted to see if we could couple our library at the server side and access results via Reporting Services and Report Builder. We believed that the computations would be more efficient to be handled at the server while the result could take advantage of the graphing packages in Report Services.

Our approach was to build a standalone system to develop and test this work on. We choose a standard PC platform running Windows 7. The first task at hand was to decide which version of Microsoft SQL Server to work with. We chose SQL Server 2008R2 full version as we needed the Report Services support. Microsoft supplies a six month trial version that has all services available which was perfect for our project. We installed the full version with support for the Management Studio, Report Services, and Analysis services. In addition, we downloaded Microsoft’s AdventureWorks sample databases for our testing environment.

We got this all running and ran some sample scripts to make sure we had everything running correctly before moving on. Since we were going to be developing code we installed Visual Studio 2010 next and made sure we included the necessary development templates for the SQL Server environment.

After installing CenterSpace’s NMath 5.0 library, we were ready to write some code.

We will start by creating a new project in Visual Studio called NMathSQL.

Creating a new SQL CLR project in Visual Studio

We then asked to pick a server and identify the database we will be using.

Connecting to the Database

In the next screen we will want to enable SQL/CLR debugging.

Enabling SQL/CLR debugging

At this point we need to start running SQL scripts as part of the overall setup. We could exit VS and perform these tasks in the Management Studio, but VS can do the job. By selecting from the VS menu bar Data>Transact-SQL Editor>New Query Connection, we get a new window to enter scripts to be run on our database.

Accessing the Transact SQL Editor in Visual Studio

We found it easier to copy and paste our scripts into this windows as we needed to run SQL statements.

Our first task at hand is to load the assembly NMath.dll into the database we have selected to work in. It is at this point we hit our first real obstacle. It turns out that in our shipping NMath.dll we are using some instructions to automatically load x86 or x64. These are illegal in the SQL CLR. We were forced to remove these instructions and build an entirely separate version of NMath.dll for SQL Server. Stepping back and looking at the big picture, we decided that this was an opportunity to also include some of the “glue” code we knew we wanted to write.

We had been looking at writing some User Defined Types (UDT) and User Defined Aggregate (UDA) functions to handle the calls into our library. It seemed to make sense to extend our libraries type definitions to be T-SQL types. The result of this approach would enable T-SQL to use NMath types such as DoubleVector and DoubleMatrix. We needed to have some way to load our new data types so we built UDAs LoadDoubleVector, LoadDoubleMatrixByRow, and LoadDoubleMatrixByColumn. After building our new, custom NMath.dll, we can then run a script to load our assembly into the database.

Before we could do this, though, we discovered that we needed some basic assemblies that SQL server had left out of their CLR. The following SQL script loads those assemblies.

Loading Supporting Assemblies to a SQL Database

We are now ready to run the following SQL script to load NMath for SQL.

Loading NMath assembly in the SQL Database

The next step is to add the reference to our project. From the main toolbar launch the Solution Explorer and right click on Reference then select Add Reference. Add the new assemblies we have loaded so far.

Adding NMath reference to the SQL/CLR project

We will now have to tell SQL about the new types and functions we have added. We will start by adding the new UDTs DoubleVector and DoubleMatrix.

SQL commands to add UDTs for LoadDoubleVector

Now that SQL knows about our new types we can add the UDA functions with the following SQL commands.

SQL Commands to add NMaths UDAs to the Database

At this point we are ready to test our library. We have chosen a problem that represents the work expected to be perform. We will create our sample datapoint tables rather than use tables from the AdventureWorks database. We will then load the data from the tables into DoubleVectors and call our library using a User Defined Function(UDF). Our UDF will be a call to library to find the function values for a curve that fits our datapoints. From our function solution we will create a table with datapoints that represent our curve function to compare against our original datapoints.

We will start by creating the UDF call to our library. From the Solution Explorer, we will add a UDF template called NMathFunction.cs.

Here is the C# code for this function call:

using System;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using Microsoft.SqlServer.Server;
using CenterSpace.NMath.Core;
using CenterSpace.NMath.Analysis;
 
 
public partial class UserDefinedFunctions
{
    [Microsoft.SqlServer.Server.SqlFunction]
    public static DoubleVector NMathFunction(DoubleVector xValues, DoubleVector yValues, DoubleVector start)
    {
        NMathFunctions.GeneralizedDoubleUnaryFunction f = AnalysisFunctions.FourParameterLogistic;
        OneVariableFunctionFitter fitter = new OneVariableFunctionFitter(f);
        return new DoubleVector(fitter.Fit(xValues, yValues, start));
    }
}

We can now tell VS to build the assembly NmathSQL.dll and use SQL to load the assembly to our database.

Loading our custom NMathFunctions to the database

In order for our UDF NmathFunction to be recognized we need to run a SQL create function command. Note that this command must be the only command in the script.

SQL Commands to add our UDF to the database

We have now loaded our NMath function call in the database that can be called by scripts. We are ready to write a script to solve the problem we described above.

We will start by using the same data from the example we did with our excel post.

Sample Data for our example

We can now build a script that uses the functionality we have built in with our library to find the solution.

-------------------------------------------------------
Declare @xv DoubleVector
Declare @yv DoubleVector
 
select @xv = dbo.LoadDoubleVector(xcol) from xvalues
select @yv = dbo.LoadDoubleVector(ycol) from yvalues
 
Declare @sPar DoubleVector
Declare @solution DoubleVector
 
set @sPar = '[ 0.1 0.1 0.1 0.1]'
 
select @solution = dbo.NMathFunction(@xv, @yv, @sPar)
 
select @solution.ToString()
 
declare @newxval float
declare @newyval float
declare @inc float
declare @a float
declare @b float
declare @c float
declare @d float
 
set @a = dbo.DVItem(@solution, 0)
set @b = dbo.DVItem(@solution, 1)
set @c = dbo.DVItem(@solution, 2)
set @d = dbo.DVItem(@solution, 3)
 
create table SolutionTBL (newxval float, newyval float)
	set @newxval = 0
 
	while @newxval < 35.10
	begin
 
	    set @newyval = @d+((@a-@d)/(1+POWER((@newxval/@c), @b)))
 
	    insert into SolutionTBL values(@newxval, @newyval)
 
	  set @newxval = (@newxval + 0.33)
	end
	select * from SolutionTBL
	go
---------------------------------------------------------------------

Here is the output from this script

Returned Solution from NMath library call

At this point we can pull data from tables, call our Math libraries, and put the solution in a table to be displayed. In production, this last script would be a stored procedure that would be run from the Management Studio. All of this work would reside on the SQL Server.

We can now move on to the Reporting Services to see how this solution could be displayed.

After launching Report Builder and establishing connection to the database. The data is accessed by setting up a dataset with a query into the table for the necessary data.

Selecting the chart data in Report Builder

From there it is merely Report Builder to build the necessary graphs. Unfortunately, the chart wizard doesn’t include scatterplots. It is best to select a chart type of Line and then run through the wizard. After your chart is set up you can change its properties to scatterplot.

Report Builder ScatterPlot

We can include our solution in the same report and generate a smooth line as we did in excel. The Report Builder is a very powerful tool which we are still learning.

Solution Fitted Line

Our results are equivilent to our previous excel post and demonstrate that the approach on the SQL Server is as accurate.

In conclusion, we have demonstrated that we can call the CenterSpace NMath libraries from SQL Server and display the results using Report Services powerful charting capabilities. We plan to work on more examples and compare how we might improve on what Analysis Services can produce. We certainly will entertain feedback on useful approaches that should be examined. As a reminder this interface is not possible with our current release of NMath 5.0 and require a “special” version of our assembly. Depending on customer interest and feedback we may or may not decide to develop a product for this interface.

Curve fitting is one of the most practical applications of mathematics as we are often asked to explain or make predictions based on a collection of data points. For example, if we collect a series of changing data values over time, we look to explain the relationship that time effects the generated values or in mathematical terms y=f(x). Here we are using x to represent time values and f(x) to represent the relationship or function that generates the resulting values or y. So if we can find a function f(x) that represents a good fit of the data we should be able to predict the result at any moment in time.

With this in mind, let us get started with some data points (x, y) that we have collected. I have entered the following values in an Excel spreadsheet.

We can now use Excel’s charting function to create the following XY scatterplot with our data:

At this point, we can add an Excel trendline to get the best fit it can provide. By right clicking on a data value in our chart we will get the option to add a trendline. Choose a 2nd order polynomial and these options.

Name the trendline “2nd Order Polynomial” and check “Display equation on chart” and “Display R-squared value on chart”.

Excel calculates and plots the line while returning the equation and the R2 value. If our R2 equals 1 we would have found a perfect fit. For a second order polynomial Excel returned a value of 0.8944 which means that roughly 10.6 percent (1-.894) are not on this line.

If we continue increasing our polynomial orders up to the maximum of six we can achieve the best R2 value of 0.9792, but look at the curve we have fitted to these points.

A better fit might be an exponential function so let us try Excel’s trendline option using exponentials.

Clearly the results are visually a better fit but the R2 value tells us that over 15% of the data points are not on this line. This pretty much represents the best we can do with Excel’s trendline functions for our data.

Looking to CenterSpace’s NMath and NStat libraries to give us more robust analysis, we can utilize more powerful curve fitting tools quickly and with little effort.

Using the linkage provided by ExcelDNA that we examined in our previous posts, we can create the following C# code for our ExcelDNA text file.

<![CDATA[
 
using System;
using ExcelDna.Integration;
using CenterSpace.NMath.Core;
using CenterSpace.NMath.Matrix;
using CenterSpace.NMath.Analysis;
using CenterSpace.NMath.Stats;
 
public class NMathExcelCurveFit
{
 
 [ExcelFunction(Description="Four parameterized Fit")]
 public static double[] NOneVarFunctFitFour(double[] xValues, double[] yValues, double[] start)
 {
  DoubleVector TempVx = new DoubleVector(xValues);
  DoubleVector TempVy = new DoubleVector(yValues);
  DoubleVector TempVs = new DoubleVector(start);
  OneVariableFunctionFitter
<TrustRegionMinimizer> fitter = new  OneVariableFunctionFitter
<TrustRegionMinimizer>( AnalysisFunctions.FourParameterLogistic );
  DoubleVector solution = fitter.Fit(TempVx, TempVy, TempVs);
  return solution.ToArray();
 }
 
 [ExcelFunction(Description="Four parameterized R2")]
 public static double NOneVarFunctFitFourR2(double[] xValues, double[] yValues, double[] start)
 {
  DoubleVector TempVx = new DoubleVector(xValues);
  DoubleVector TempVy = new DoubleVector(yValues);
  DoubleVector TempVs = new DoubleVector(start);
  OneVariableFunctionFitter
<TrustRegionMinimizer> fitter = new  OneVariableFunctionFitter
<TrustRegionMinimizer>( AnalysisFunctions.FourParameterLogistic );
  DoubleVector solution = fitter.Fit(TempVx, TempVy, TempVs);
  GoodnessOfFit gof = new GoodnessOfFit(fitter, TempVx, TempVy, solution);
  return gof.RSquared;
 }
}
]]>

As you can see by the code, I have chosen to use NMath’s OneVariableFunctionFitter with the FourParameterLogistic predefined generalized function.

From CenterSpace’s documentation, we get the above equation and need to solve for the model parameters a, b, c, d .

The OneVariableFunction call will require us to provide a starting guess along with the ranges containing the xValues and yValues. The following screen shows us making the function call from Excel with the necessary parameters.

After computing these values, we can call for the R2 value and generate some points to be plotted.

Note that I choose to hide the rfows from r17 to r107 for display purposes. You will need to have them unhid for the chart to look right. As you can see we returned the best R2 value so far at 0.9923.

In the next screen, we will have added the series to our chart and drawn a line through our calculated points.

This example illustrates the ease that Excel can use the power of NMath curve fitting routines to compute accurate fits to a collection of data.

Mike Magee