In signal processing, wavelets have been widely investigated for use in filtering bio-electric signals, among many other applications. Bio-electric signals are good candidates for multi-resolution wavelet filtering over standard Fourier analysis due to their non-stationary character. In this article we’ll discuss the filtering of electrocardiograms or ECGs and demonstrate with code examples how to filter an ECG waveform using NMath‘s new wavelet classes; keeping in mind that the techniques and code shown here apply to a wide class of time series measurements. If wavelets and their applications to filtering are unfamiliar to the reader, read a gentle and brief introduction to the subject in Wavelets for Kids: A Tutorial Introduction [5].

Filtering a Time Series with Wavelets

PhysioNet provides free access to a large collections of recorded physiologic signals, including many ECG’s. The ECG signal we will filter here, named aami-ec13 on PhysioNet, is shown below.

Our goal will be to remove the high frequency noise while preserving the character of the wave form, including the high frequency transitions at the signal peaks. Fourier based filter methods are ill suited for filtering this type of signal due to both it’s non-stationarity, as mentioned, but also the need to preserve the peak locations (phase) and shape.

A Wavelet Filter

As with Fourier analysis there are three basic steps to filtering signals using wavelets.

1. Decompose the signal using the DWT.
2. Filter the signal in the wavelet space using thresholding.
3. Invert the filtered signal to reconstruct the original, now filtered signal, using the inverse DWT.

Briefly, the filtering of signals using wavelets is based on the idea that as the DWT decomposes the signal into details and approximation parts, at some scale the details contain mostly the insignificant noise and can be removed or zeroed out using thresholding without affecting the signal. This idea is discussed in more detail in the introductory paper [5]. To implement this DWT filtering scheme there are two basic filter design parameters: the wavelet type and a threshold. Typically the shape and form of the signal to be filtered is qualitatively matched to the general shape of the wavelet. In this example we will use the Daubechies forth order wavelet.

The general shape of this wavelet roughly matches, at various scales, the morphology of the ECG signal. Currently NMath supports the following wavelet families: Harr, Daubechies, Symlet, Best Localized, and Coiflet, 27 in all. Additionally, any custom wavelet of your invention can be created by passing in the wavelet’s low & high pass decimation filter values. The wavelet class then imposes the wavelet’s symmetry properties to compute the reconstruction filters.

   // Build a Coiflet wavelet.
   var wavelet = new FloatWavelet( Wavelet.Wavelets.C4 );

   // Build a custom reverse bi-orthogonal wavelet.
   var wavelet = new DoubleWavelet( new double[] {0.0, 0.0, 0.7071068, 0.7071068, 0.0, 0.0}, new double[] {0.0883883, 0.0883883, -0.7071068, 0.7071068, -0.0883883, -0.0883883} );

The FloatDWT class provides four different thresholding strategies: Universal, UniversalMAD, Sure, and Hybrid (a.k.a SureShrink). We’ll use the Universal threshold strategy here. This is a good starting point but this strategy can over smooth the signal. Typically some empirical experimentation is done here to find the best threshold for the data (see [1], also see [4] for a good overview of common thresholding strategies.)

Wavelet Filtering Code

The three steps outlined above are easily coded using two classes in the NMath library: the FloatDWT class and the FloatWavelet class. As always in NMath, the library offers both a float precision and a double precision version of each of these classes. Let’s look at a code snippet that implements a DWT based filter with NMath.

   // Choose wavelet, the Daubechies 4 wavelet
   var wavelet = new FloatWavelet( Wavelet.Wavelets.D4 );

   // Build DWT object using our wavelet & data
   var dwt = new FloatDWT( data, wavelet );

   // Decompose signal with DWT to level 5
   dwt.Decompose( 5 );

   // Find Universal threshold & threshold all detail levels
   double lambdaU = dwt.ComputeThreshold( FloatDWT.ThresholdMethod.Universal, 1 );
   dwt.ThresholdAllLevels( FloatDWT.ThresholdPolicy.Soft, new double[] { lambdaU, 
       lambdaU, lambdaU, lambdaU, lambdaU } );

   // Rebuild the filtered signal.
   float[] reconstructedData = dwt.Reconstruct();

The first two lines of code build the wavelet object and the DWT object using both the input data signal and the abbreviated Daubechies wavelet name Wavelet.Wavelets.D4. The third line of code executes the wavelet decomposition at five consecutive scales. Both the signal’s details and approximations are stored in the DWT object at each step in the decomposition. Next, the Universal threshold is computed and the wavelet details are thresholded using the same threshold with a Soft policy (see [1], pg. 63). Lastly, the now filtered signal is reconstructed.

Below, the chart on the left shows the unfiltered ECG signal and the chart on the right shows the wavelet filtered ECG signal. It’s clear that this filter very effectively removed the noise while preserving the signal.

These two charts below show a detail from the chart above from indices 500 to 1000. Note how well the signal shape, phase, and amplitude has been preserved in this non-stationary wavelet-filtered signal.

It is this ability to preserve phase, form, and amplitude in DWT based filters all while having a O(n log n) runtime that Fourier-based filters enjoy that has made wavelets such an important part of signal processing today. The complete code for this example along with a link to the ECG data is provided below.

Paul

References

Test Data

To copy the data file provided by PhysioNet for this example click: ECG_AAMIEC13.data
This ECG data was taken from the ANSI EC13 test data set waveforms.

Complete Test Code

    public void BlogECGExample()
    {
      // Define your own dataDir
      var dataDir = "................";

      // Load ECG wave from physionet.org data file.
      string filename = Path.Combine( dataDir, "ECG_AAMIEC13.data.txt" );
      string line;
      int cnt = 0;
      FloatVector ecgMeasurement = new FloatVector( 3000 );
      var fileStrm = new System.IO.StreamReader( filename );
      fileStrm.ReadLine(); fileStrm.ReadLine();
      while ( ( line = fileStrm.ReadLine() ) != null && cnt < 3000 )
      {
        ecgMeasurement[cnt] = Single.Parse( line.Split( ',' )[1] );
        cnt++;
      }

      // Choose wavelet
      var wavelet = new FloatWavelet( Wavelet.Wavelets.D4 );

      // Build DWT object
      var dwt = new FloatDWT( ecgMeasurement.DataBlock.Data, wavelet );

      // Decompose signal with DWT to level 5
      dwt.Decompose( 5 );

      // Find Universal threshold & threshold all detail levels with lambdaU
      double lambdaU = dwt.ComputeThreshold( FloatDWT.ThresholdMethod.Universal, 1 );
      dwt.ThresholdAllLevels( FloatDWT.ThresholdPolicy.Soft, new double[] { lambdaU, lambdaU, lambdaU, lambdaU, lambdaU } );

      // Rebuild the signal to level 1 - the original (filtered) signal.
      float[] reconstructedData = dwt.Reconstruct();

      // Display DWT results.
      BlogECGExampleBuildCharts( dwt, ecgMeasurement, reconstructedData );

    }

    public void BlogECGExampleBuildCharts( FloatDWT dwt, FloatVector ECGMeasurement, float[] ReconstructedData )
    {

      // Plot out approximations at various levels of decomposition.
      var approxAllLevels = new FloatVector();
      for ( int n = 5; n > 0; n-- )
      {
        var approx = new FloatVector( dwt.WaveletCoefficients( DiscreteWaveletTransform.WaveletCoefficientType.Approximation, n ) );
        approxAllLevels.Append( new FloatVector( approx ) );
      }

      var detailsAllLevels = new FloatVector();
      for ( int n = 5; n > 0; n-- )
      {
        var approx = new FloatVector( dwt.WaveletCoefficients( DiscreteWaveletTransform.WaveletCoefficientType.Details, n ) );
        detailsAllLevels.Append( new FloatVector( approx ) );
      }

      // Create and display charts.
      Chart chart0 = NMathChart.ToChart( detailsAllLevels );
      chart0.Titles.Add( "Concatenated DWT Details to Level 5" );
      chart0.ChartAreas[0].AxisY.Title = "DWT Details";
      chart0.Height = 270;
      NMathChart.Show( chart0 );

      Chart chart1 = NMathChart.ToChart( approxAllLevels );
      chart1.Titles.Add("Concatenated DWT Approximations to Level 5");
      chart1.ChartAreas[0].AxisY.Title = "DWT Approximations";
      chart1.Height = 270;
      NMathChart.Show( chart1 );

      Chart chart2 = NMathChart.ToChart( (new FloatVector( ReconstructedData ))[new Slice(500,500)] );
      chart2.Titles[0].Text = "Thresholded & Reconstructed ECG Signal";
      chart2.ChartAreas[0].AxisY.Title = "mV";
      chart2.Height= 270;
      NMathChart.Show( chart2 );

      Chart chart3 = NMathChart.ToChart( (new FloatVector( ECGMeasurement ))[new Slice(500,500)] );
      chart3.Titles[0].Text = "Raw ECG Signal";
      chart3.ChartAreas[0].AxisY.Title = "mV";
      chart3.Height = 270;
      NMathChart.Show( chart3 );

    }

The post Filtering with Wavelet Transforms appeared first on CenterSpace.

You can download it here: MP4

Please let us know which topics you want us to cover. Email support@centerspace.net

Cheers,
Trevor

The post NMath Tutorial videos appeared first on CenterSpace.

Let’s start by creating a data set: 100 values drawn from a normal distribution with known parameters (mean = 0.5, variance = 2.0).

int n = 100;
double mean = .5;
double variance = 2.0;
var data = new DoubleVector( n, new RandGenNormal( mean, variance ) );

Now, compute y values based on the empirical cumulative distribution function (CDF), which returns the probability that a random variable X will have a value less than or equal to x–that is, f(x) = P(X <= x). Here’s an easy way to do, although not necessarily the most efficient for larger data sets:

var cdfY = new DoubleVector( data.Length );
var sorted = NMathFunctions.Sort( data );
for ( int i = 0; i < data.Length; i++ )
{
  int j = 0;
  while ( j < sorted.Length && sorted[j] <= data[i] ) j++;
  cdfY[i] = j / (double)data.Length;
}

The data is sorted, then for each value x in the data, we iterate through the sorted vector looking for the first value that is greater than x.

We’ll use one of NMath’s non-linear least squares minimization routines to fit a normal distribution CDF() function to our empirical CDF. NMath provides classes for fitting generalized one variable functions to a set of points. In the space of the function parameters, beginning at a specified starting point, these classes finds a minimum (possibly local) in the sum of the squared residuals with respect to a set of data points.

A one variable function takes a single double x, and returns a double y:

y = f(x)

A generalized one variable function additionally takes a set of parameters, p, which may appear in the function expression in arbitrary ways:

y = f(p1, p2,..., pn; x)

For example, this code computes y=a*sin(b*x + c):

public double MyGeneralizedFunction( DoubleVector p, double x )
{
  return p[0] * Math.Sin( p[1] * x + p[2] );
}

In the distribution fitting example, we want to define a parameterized function delegate that returns CDF(x) for the distribution described by the given parameters (mean, variance):

Func f =
  ( DoubleVector p, double x ) =>
    new NormalDistribution( p[0], p[1] ).CDF( x );

Now that we have our data and the function we want to fit, we can apply the curve fitting routine. We’ll use a bounded function fitter, because the variance of the fitted normal distribution must be constrained to be greater than 0.

var fitter = new BoundedOneVariableFunctionFitter( f );
var start = new DoubleVector( new double[] { 0.1, 0.1 } );
var lowerBounds = new DoubleVector( new double[] { Double.MinValue, 0 } );
var upperBounds = 
   new DoubleVector( new double[] { Double.MaxValue, Double.MaxValue } );
var solution = fitter.Fit( data, cdfY, start, lowerBounds, upperBounds );
var fit = new NormalDistribution( solution[0], solution[1] );

Console.WriteLine( "Fitted distribution:\nmean={0}\nvariance={1}",
  fit.Mean, fit.Variance );

The output for one run is

Fitted distribution: 
mean=0.567334190790594
variance=2.0361207956132

which is a reasonable approximation to the original distribution (given 100 points).

We can also visually inspect the fit by plotting the original data and the CDF() function of the fitted distribution.

ToChart( data, cdfY, SeriesChartType.Point, fit,
  NMathStatsChart.DistributionFunction.CDF );

private static void ToChart( DoubleVector x, DoubleVector y,
  SeriesChartType dataChartType, NormalDistribution dist,
  NMathStatsChart.DistributionFunction distFunction )
{
  var chart = NMathStatsChart.ToChart( dist, distFunction );
  chart.Series[0].Name = "Fit";

  var series = new Series() {
    Name = "Data",
    ChartType = dataChartType
  };
  series.Points.DataBindXY( x, y );
  chart.Series.Insert( 0, series );

  chart.Legends.Add( new Legend() );
  NMathChart.Show( chart );
}

We can also look at the probability density function (PDF) of the fitted distribution, but to do so we must first construct an empirical PDF using a histogram. The x-values are the midpoints of the histogram bins, and the y-values are the histogram counts converted to probabilities, scaled to integrate to 1.

int numBins = 10;
var hist = new Histogram( numBins, data );

var pdfX = new DoubleVector( hist.NumBins );
var pdfY = new DoubleVector( hist.NumBins );
for ( int i = 0; i < hist.NumBins; i++ )
{
  // use bin midpoint for x value
  Interval bin = hist.Bins[i];
  pdfX[i] = ( bin.Min + bin.Max ) / 2;

   // convert histogram count to probability for y value
   double binWidth = bin.Max - bin.Min;
   pdfY[i] = hist.Count( i ) / ( data.Length * binWidth );
}

ToChart( pdfX, pdfY, SeriesChartType.Column, fit,
  NMathStatsChart.DistributionFunction.PDF );

You might be tempted to try to fit a distribution PDF() function directly to the histogram data, rather than using the CDF() function like we did above, but this is problematic for several reasons. The bin counts have different variability than the original data. They also have a fixed sum, so they are not independent measurements. Also, for continuous data, fitting a model based on aggregated histogram counts, rather than the original data, throws away information.

Ken

Download the source code

The post Distribution Fitting Demo appeared first on CenterSpace.

The following tests were performed on a DoubleVector of length 10,000,000.

1) Create a new vector. This isn’t really clearing out an existing vector but we thought we should include it for completeness.

 DoubleVector v2 = new DoubleVector( v.Length, 0.0 );

The big drawback here is that you’re creating new memory. Time: 419.5ms

2) Probably the first thing to come to mind is to simply iterate through the vector and set everything to zero.

for ( int i = 0; i < v.Length; i++ )
{
  v[i] = 0.0;
}

We have to do some checking in the index operator. No new memory is created. Time: 578.5ms

3) In some cases, you could iterate through the underlying array of data inside the DoubleVector.

 
for ( int i = 0; i < v.DataBlock.Data.Length; i++ )
{
  v.DataBlock.Data[i] = 0.0;
}

This is a little less intuitive. And, very importantly, it will not work with many views into other data structures. For example, a row slice of a matrix. However, it's easier for the CLR to optimize this loop. Time: 173.5ms

4) We can use the power of Intel's MKL to multiply the vector by zero.

 v.Scale( 0.0 );

Scale() does this in-place. No new memory is created. In this example, we assume that MKL has already been loaded and is ready to go which is true if another MKL-based NMath call was already made or if NMath was initialized. This method will work on all views of other data structures. Time: 170ms

5) This surprised us a bit but the best method we could find was to clear out the underlying array using Array.Clear() in .NET

 Array.Clear( v.DataBlock.Data, 0, v.DataBlock.Data.Length );

This creates no new memory. However, this will not work with non-contiguous views. However, this method is very fast. Time: 85.8ms

To make efficiently clearing a vector simpler for NMath users we have created a Clear() method and a Clear( Slice ) method on the vector and matrix classes. It will do the right thing in the right circumstance. It will be released in NMath 5.2 in 2012.

Test Code

using System;
using CenterSpace.NMath.Core;

namespace Test
{
  class ClearVector
  {
    static int size = 100000000;
    static int runs = 10;
    static int methods = 5;
    
    static void Main( string[] args )
    {
      System.Diagnostics.Stopwatch sw = new System.Diagnostics.Stopwatch();
      DoubleMatrix times = new DoubleMatrix( runs, methods );
      NMathKernel.Init();

      for ( int run = 0; run < runs; run++ )
      {
        Console.WriteLine( "Run {0}...", run );
        DoubleVector v = null;

        // Create a new one
        v = new DoubleVector( size, 1.0, 2.0 );
        sw.Start();
        DoubleVector v2 = new DoubleVector( v.Length, 0.0 );
        sw.Stop();
        times[run, 0] = sw.ElapsedMilliseconds;
        Console.WriteLine( Assert( v2 ) );

        // iterate through vector
        v = new DoubleVector( size, 1.0, 2.0 );
        sw.Reset();
        sw.Start();
        for ( int i = 0; i < v.Length; i++ )
        {
          v[i] = 0.0;
        }
        sw.Stop();
        times[run, 1] = sw.ElapsedMilliseconds;
        Console.WriteLine( Assert( v ) );

        // iterate through array
        v = new DoubleVector( size, 1.0, 2.0 );
        sw.Reset();
        sw.Start();
        for ( int i = 0; i < v.DataBlock.Data.Length; i++ )
        {
          v.DataBlock.Data[i] = 0.0;
        }
        sw.Stop();
        times[run, 2] = sw.ElapsedMilliseconds;
        Console.WriteLine( Assert( v ) );
        
        // scale
        v = new DoubleVector( size, 1.0, 2.0 );
        sw.Reset();
        sw.Start();
        v.Scale( 0.0 );
        sw.Stop();
        times[run, 3] = sw.ElapsedMilliseconds;
        Console.WriteLine( Assert( v ) );

        // Array Clear
        v = new DoubleVector( size, 1.0, 2.0 );
        sw.Reset();
        sw.Start();
        Array.Clear( v.DataBlock.Data, 0, v.DataBlock.Data.Length );
        sw.Stop();
        times[run, 4] = sw.ElapsedMilliseconds;
        Console.WriteLine( Assert( v ) );
        Console.WriteLine( times.Row( run ) );
      }
      Console.WriteLine( "Means: " + NMathFunctions.Mean( times ) );
    }

    private static bool Assert( DoubleVector v )
    {
      if ( v.Length != size )
      {
        return false;
      }
      for ( int i = 0; i < v.Length; ++i )
      {
        if ( v[i] != 0.0 )
        {
          return false;
        }
      }
      return true;
    }
  }
}

- Trevor

The post Clearing a vector appeared first on CenterSpace.

There is a one-time delay when the appropriate x86 or x64 native code is loaded. This cost can be easily controlled by the developer by using the NMathKernel.Init() method. Please see Initializing NMath for more details.

– Trevor

The post Initializing NMath appeared first on CenterSpace.

For instance, let’s try fitting a circle to a set of points. A circle is defined by three parameters: a center (a,b), and a radius r. The circle is all points such that (x-a)^2 + (y – b)^2 = r^2. To setup the minimization problem, we first need to define a function which given a set of circle parameters, returns the residuals with respect to a cloud of 2D points. There are several methods for doing this in NMath. In the following C# code, we extend the abstract base class DoubleMultiVariableFunction, and override the Evaluate() method:

public class CircleFitFunction : DoubleMultiVariableFunction
{

  public CircleFitFunction( DoubleVector x, DoubleVector y )
    : base( 3, x.Length )
  {
    if( x.Length != y.Length )
      throw new Exception( "Unequal number of x,y values." );

    X = x;
    Y = y;
  }

  public DoubleVector X { get; internal set; }
  public DoubleVector Y { get; internal set; }

  public override void Evaluate( DoubleVector parameters, ref DoubleVector residuals )
  {
    // parameters of circle with center (a,b) and radius r
    double a = parameters[0];
    double b = parameters[1];
    double r = parameters[2];

    for( int i = 0; i < X.Length; i++ )
    {
      // distance of point from circle center
      double d = Math.Sqrt( Math.Pow( X[i] - a, 2.0 ) + Math.Pow( Y[i] - b, 2.0 ) );

      residuals[i] = d - r;
    }
  }
}

The constructor takes the set of x,y points to fit, which are stored on the class. Note that we call the base constructor with the dimensions of the domain and range of the function:

: base( 3, x.Length )

In this case, the function maps the 3-dimensional space of the circle parameters to the x.Length-dimension space of the residuals.

Next we override the Evaluate() method, which takes an input vector and output vector passed by reference. Our implementation computes the residual for each point with respect to a circle defined by the given parameters. We calculate the distance, d, of each point from the circle center; the residual is then equal to d - r. The nonlinear least squares method will minimize the L2 norm of this function.

To demonstrate the fitting process, let's first start with a set of points generated from a circle of known parameters. For example, this C# code creates 20 x,y points evenly spaced around a circle with center (0.5, 0.25) and radius 2, plus some added noise:

int n = 20;
DoubleVector x = new DoubleVector( n );
DoubleVector y = new DoubleVector( n );

double a = 0.5;
double b = 0.25;
double r = 2;

RandGenUniform noise = new RandGenUniform( -.75, .75 );
for( int i = 0; i < n; i++ )
{
  double t = i * 2 * Math.PI / n;
  x[i] = a + r * Math.Cos( t ) + noise.Next();
  y[i] = b + r * Math.Sin( t ) + noise.Next();
}

Now that we have our function defined and some points to fit, performing the minimization takes only a few lines of code. This C# code finds the minimum and prints the solution and final residual:

CircleFitFunction f = new CircleFitFunction( x, y );
TrustRegionMinimizer minimizer = new TrustRegionMinimizer();
DoubleVector start = new DoubleVector( "0.1 0.1 0.1" );
DoubleVector solution = minimizer.Minimize( f, start );

Console.WriteLine( "solution = " + solution );
Console.WriteLine( "final residual = " + minimizer.FinalResidual );

Sample output:

solution = [ 0.446122611523468 0.175483486509012 1.93511389538286 ]
final residual = 1.62414259527024

Starting from point (0.1, 0.1, 0.1) in the parameter space of the circle, we minimized the sum of the squared residuals with respect to the (x,y) points. In the run above, the minimum found was center (0.45, 0.18) and radius 1.9, close to the actual parameters which generated our (noisy) data. To visually inspect the fit, we can use NMath with the Microsoft Chart Controls for .NET.

Now that we're sure the minimization is working as desired, let's try it again with some random points:

int n = 10;
RandGenUniform rng = new RandGenUniform( -1, 2 );
DoubleVector x = new DoubleVector( n, rng );
DoubleVector y = new DoubleVector( n, rng );

CircleFitFunction f = new CircleFitFunction( x, y );
TrustRegionMinimizer minimizer = new TrustRegionMinimizer();
DoubleVector start = new DoubleVector( "0.1 0.1 0.1" );
DoubleVector solution = minimizer.Minimize( f, start );

Again, plotting the solution:

In some runs, the fit may seem counter-intuitive. For example:

But if you think about it, this sort of fit makes perfect sense, given how we defined our fit function. We're asking the minimizer to minimize the residuals with respect to the circle perimeter, and allowing it vary the circle center and radius however it can to achieve that goal. In an extreme case, if our points fall approximately along a line, the best fit will be a circle with a very large radius, so that the circle perimeter is nearly linear as it passes through the points.

In this case, the fitted circle has center (428.0, -757.8) and radius 870.6.  If we wish to avoid solutions such as these, we could define our fit function differently. For example, we might constrain the circle center to be the center of mass of the points, and vary only the radius.

Obviously, a similar technique can be used to fit other geometric primitives, though as the complexity of the shape increases, so does the complexity of the fit function.

Ken

The post Fitting Geometric Primitives to Points Using Nonlinear Least Squares appeared first on CenterSpace.

Curve fitting is one of the most practical applications of mathematics as we are often asked to explain or make predictions based on a collection of data points. For example, if we collect a series of changing data values over time, we look to explain the relationship that time effects the generated values or in mathematical terms y=f(x). Here we are using x to represent time values and f(x) to represent the relationship or function that generates the resulting values or y. So if we can find a function f(x) that represents a good fit of the data we should be able to predict the result at any moment in time.

With this in mind, let us get started with some data points (x, y) that we have collected. I have entered the following values in an Excel spreadsheet.

We can now use Excel’s charting function to create the following XY scatterplot with our data:

At this point, we can add an Excel trendline to get the best fit it can provide. By right clicking on a data value in our chart we will get the option to add a trendline. Choose a 2nd order polynomial and these options.

Name the trendline “2nd Order Polynomial” and check “Display equation on chart” and “Display R-squared value on chart”.

Excel calculates and plots the line while returning the equation and the R2 value. If our R2 equals 1 we would have found a perfect fit. For a second order polynomial Excel returned a value of 0.8944 which means that roughly 10.6 percent (1-.894) are not on this line.

If we continue increasing our polynomial orders up to the maximum of six we can achieve the best R2 value of 0.9792, but look at the curve we have fitted to these points.

A better fit might be an exponential function so let us try Excel’s trendline option using exponentials.

Clearly the results are visually a better fit but the R2 value tells us that over 15% of the data points are not on this line. This pretty much represents the best we can do with Excel’s trendline functions for our data.

Looking to CenterSpace’s NMath and NStat libraries to give us more robust analysis, we can utilize more powerful curve fitting tools quickly and with little effort.

Using the linkage provided by ExcelDNA that we examined in our previous posts, we can create the following C# code for our ExcelDNA text file.

 fitter = new  OneVariableFunctionFitter
( AnalysisFunctions.FourParameterLogistic );
  DoubleVector solution = fitter.Fit(TempVx, TempVy, TempVs);
  return solution.ToArray();
 }

 [ExcelFunction(Description="Four parameterized R2")]
 public static double NOneVarFunctFitFourR2(double[] xValues, double[] yValues, double[] start)
 {
  DoubleVector TempVx = new DoubleVector(xValues);
  DoubleVector TempVy = new DoubleVector(yValues);
  DoubleVector TempVs = new DoubleVector(start);
  OneVariableFunctionFitter
 fitter = new  OneVariableFunctionFitter
( AnalysisFunctions.FourParameterLogistic );
  DoubleVector solution = fitter.Fit(TempVx, TempVy, TempVs);
  GoodnessOfFit gof = new GoodnessOfFit(fitter, TempVx, TempVy, solution);
  return gof.RSquared;
 }
}
]]>

As you can see by the code, I have chosen to use NMath’s OneVariableFunctionFitter with the FourParameterLogistic predefined generalized function.

From CenterSpace’s documentation, we get the above equation and need to solve for the model parameters a, b, c, d .

The OneVariableFunction call will require us to provide a starting guess along with the ranges containing the xValues and yValues. The following screen shows us making the function call from Excel with the necessary parameters.

After computing these values, we can call for the R2 value and generate some points to be plotted.

Note that I choose to hide the rfows from r17 to r107 for display purposes. You will need to have them unhid for the chart to look right. As you can see we returned the best R2 value so far at 0.9923.

In the next screen, we will have added the series to our chart and drawn a line through our calculated points.

This example illustrates the ease that Excel can use the power of NMath curve fitting routines to compute accurate fits to a collection of data.

Mike Magee

The post Advanced Curve Fitting using Excel and NMath appeared first on CenterSpace.

A Chart object contains one or more ChartAreas, each of which contain one or more data Series. Each Series has an associated chart type, and a DataPoint collection. DataPoints can be manually appended or inserted into the collection, or added automatically when a series is bound to a datasource using either the DataBindY() or DataBindXY() method. Since any IEnumerable can act as a datasource, it’s easy to use an NMath vector or vector view (of a matrix row or column, for example) as a datasource.

For example, suppose we want to create a scatter plot of the first two columns of a 20 x 5 DoubleMatrix (that is, column 0 vs. column 1).

DoubleMatrix data = new DoubleMatrix( 20, 5, new RandGenUniform() );
DoubleVector x = data.Col( 0 );
DoubleVector y = data.Col( 1 );

Begin by creating a new Chart object, and optionally adding a Title.

Chart chart = new Chart()
{
  Size = new Size( 500, 500 ),
};

Title title = new Title()
{
  Name = chart.Titles.NextUniqueName(),
  Text = "My Data",
  Font = new Font( "Trebuchet MS", 12F, FontStyle.Bold ),
};
chart.Titles.Add( title );

Next, add a ChartArea.

ChartArea area = new ChartArea()
{
  Name = chart.ChartAreas.NextUniqueName(),
};
area.AxisX.Title = "Col 0";
area.AxisX.TitleFont = new Font( "Trebuchet MS", 10F, FontStyle.Bold );
area.AxisX.MajorGrid.LineColor = Color.LightGray;
area.AxisX.RoundAxisValues();
area.AxisY.Title = "Col 1";
area.AxisY.TitleFont = new Font( "Trebuchet MS", 10F, FontStyle.Bold );
area.AxisY.MajorGrid.LineColor = Color.LightGray;
area.AxisY.RoundAxisValues();
chart.ChartAreas.Add( area );

Finally, add a new data Series, and bind the datasource to the NMath x,y vectors.

Series series = new Series()
{
  Name = "Points",
  ChartType = SeriesChartType.Point,
  MarkerStyle = MarkerStyle.Circle,
  MarkerSize = 8,
};
series.Points.DataBindXY( x, y );
chart.Series.Add( series );

To display the chart, we can use a utility function like this, which shows the given chart in a default form running in a new thread.

public static void Show( Chart chart )
{
  Form form = new Form();
  form.Size = new Size( chart.Size.Width + 20, chart.Size.Height + 40 );
  form.Controls.Add( chart );
  Thread t = new Thread( () => Application.Run( form ) );
   t.Start();
}

After calling

Show( chart );

the result looks like this:

Of course, you can easily plot more complicated data as well. For instance, suppose we fit a 4th degree polynomial to this (random) data:

int degree = 4;
PolynomialLeastSquares pls = new PolynomialLeastSquares( degree, x, y );

To add the fitted polynomial to the cart, just add a new data Series interpolating over the range of x values.

Series series2 = new Series()
{
  Name = "Polynomial",
  ChartType = SeriesChartType.Line,
  MarkerStyle = MarkerStyle.None,
};
int numInterpolatedValues = 100;
double xmin = NMathFunctions.MinValue( x );
double xmax = NMathFunctions.MaxValue( x );
double step = ( xmax - xmin ) / ( numInterpolatedValues - 1 );
DoubleVector xi = new DoubleVector( numInterpolatedValues, xmin, step );
series2.Points.DataBindXY( xi, pls.FittedPolynomial.Evaluate( xi ) );
chart.Series.Add( series2 );

Let’s also add a subtitle with the fitted function, and a Legend.

Title subtitle = new Title()
{
  Name = chart.Titles.NextUniqueName(),
  Text = "f(x) = " + pls.FittedPolynomial.ToString( "N2" ),
};
chart.Titles.Add( subtitle );

Legend legend = new Legend()
{
  Name = chart.Legends.NextUniqueName(),
  DockedToChartArea = chart.ChartAreas[0].Name,
};
chart.Legends.Add( legend );

Now the chart looks like this:

If you add a Chart control to your application using the visual designer (by dragging a Chart control from the Data category in the Toolbox), this generates code to create a default Chart, with a single ChartArea, Series, and Legend. You can then edit the properties in the Properties tab, and bind the generated Series to an NMath datasource. For example:

public Form1()
{
  InitializeComponent();

  DoubleMatrix data = new DoubleMatrix( 20, 5, new RandGenUniform() );
  this.chart1.Series[0].Points.DataBindXY( data.Col(0), data.Col(1) );
}

The Microsoft Chart Controls for .NET provide an easy (and free) solution for visualizing NMath numerical types.

Ken

The post Using NMath with Microsoft Chart Controls for .NET appeared first on CenterSpace.

For example, this C# code creates a Niederreiter quasirandom number generator object to generate quasirandom vectors of length 2:

int dim = 2;
NiederreiterQuasiRandomGenerator nqrg =
  new NiederreiterQuasiRandomGenerator(dim);

You can use an NMath quasirandom generator to fill a matrix or array. The points are the columns of the matrix, so the number of rows in the given matrix must be equal to the dimension of the quasirandom number generator. Here we create and fill a 2 x 500 matrix with quasirandom numbers from a uniform (0,1) distribution.

int n = 500;
DoubleMatrix A = new DoubleMatrix(dim, n);
nqrg.Fill(A);

We can plot the points to see the distribution.

Let’s compare the results above with the distribution from a standard pseudorandom generator.

int seed = 0x345;
RandomNumberStream stream = new RandomNumberStream( seed, RandomNumberStream.BasicRandGenType.MersenneTwister );
DoubleRandomUniformDistribution unifDist = new DoubleRandomUniformDistribution();
DoubleMatrix B = new DoubleMatrix( dim, n, stream, unifDist );

Plotting the results shows that the pseudorandom points are much less uniformly distributed. Regions of higher and lower density are clearly evident.

Quasirandom numbers have many applications, especially in Monte Carlo simulations. For instance, a classic use of the Monte Carlo method is for the evaluation of definite integrals. Let’s use an NMath Sobol generator to approximate the integral of a function F over a 6-dimensional unit cube. The function is defined by

F(x) = 1*cos(1*x1) * 2*cos(2*x2) * 3*cos(3*x3) *…* 6*cos(6*x6)

where x = (x1, x2,…, x6) is a point in 6-dimensional space.

This is the implementation of the function F:

static double F( DoubleVector x )
{
  double y = 1;
  for ( int i = 1; i <= x.Length; i++ )
  {
    y *= i * Math.Cos( i * x[i - 1] );
  }
  return y;
}

In the following C# code we provide our own primitive polynomials for initializing the quasirandom generator. Primitive polynomials have coefficients in {0, 1} and are specified by BitArray’s containing the polynomial coefficients with the leading coefficient at index 0. We can supply either dim polynomials, or dim – 1 polynomials. If we specify dim – 1 polynomials, the primitive polynomial for the first dimension will be initialized with a default value.

dim = 6;

BitArray[] primitivePolys = new BitArray[dim];
primitivePolys[0] = new BitArray( new bool[] { true, true } ); // x + 1
primitivePolys[1] = new BitArray( new bool[] { true, true, true } ); // x^2 + x + 1
primitivePolys[2] = new BitArray( new bool[] { true, false, true, true } ); // x^3 + x + 1
primitivePolys[3] = new BitArray( new bool[] { true, true, false, true } ); // x^3 + x^2 + 1
primitivePolys[4] = new BitArray( new bool[] { true, false, false, true, true } ); // x^4 + x + 1
primitivePolys[5] = new BitArray( new bool[] { true, true, false, false, true } ); // x^4 + x^3 + 1

SobolQuasiRandomGenerator sobol = new SobolQuasiRandomGenerator( dim, primitivePolys );

Now let’s use this generator to approximate the integral:

int numPoints = 180000;
DoubleMatrix points = new DoubleMatrix( dim, numPoints );
sobol.Fill( points );

double sum = 0;
for ( int i = 0; i < numPoints; i++ )
{
  sum += F( points.Col( i ) );
}
sum /= numPoints;

double actualIntegralValue = -0.022;
Console.WriteLine( "Actual integral value = " + actualIntegralValue );
Console.WriteLine( "\nMonte-Carlo approximated integral value = " + sum );

The output is:

Actual integral value = -0.022
Monte-Carlo approximated integral value = -0.0232768655843053

Ken

The post Quasirandom Points appeared first on CenterSpace.

The outcome of these blog articles should illustrate that Excel can become a powerful tool to quickly access a comprehensive .NET library such as CenterSpace’s NMath without needing a large supporting programming environment. This is all made possible by Govert van Drimmelen’s freeware tool ExcelDNA. ExcelDNA can interpret VB, C#, and F# instructions stored in a text file as Excel is loaded eliminating the need for a separate compiler. ExcelDNA does require the text file to have the extension .DNA. I recommend using an editor like NotePad++ to edit this file. In our last blog post, we named our file NMathExcel.dna to hold our VB code. Below, I have provided C# code that performs the same functions as the VB code in our previous post. This code below provides examples on how to use several basic routines in NMath.

In reviewing the differences between the VB and C# versions, the first order of business was to add the language specification to “CS” for C# so that ExcelDNA knows to switch from the default language of VB. Aside from the obvious differences between the two languages, we have added a reference to our NMathShared assembly and an explicit reference to using ExcelDNA.Integration. In general, the C# code is slightly simpler than VB and enables very simple code to call the library. In our C# sample code function for creating a DoubleMatrix with a random generation, we used the library’s built in conversion function ToArray() to marshal the data into an array format for Excel. Using NMath’s built-in array handling capabilities we can simply copy data in and out in the proper formats. This illustrates the ease by which we can utilize the NMath math libraries and Excel.

To further demonstrate this capability, we can add the following code to our NMathExcel.dna text file that calls the matrix transpose function in NMath. We will show how to create array data in Excel, pass it to the library in the proper format, and then return a result for display.

   public static double[,] NDoubleMatrixTranspose(double[,] InputAr)
   {
      DoubleMatrix TempAr = new DoubleMatrix(InputAr);
     TempAr = TempAr.Transpose();
      return TempAr.ToArray();
   }

Make sure you close Excel and reopen it after saving your code changes to the .DNA file so that ExcelDNA has the opportunity to incorporate the new code.

We can now send our library a range of data cells to be acted on and display the result. In the following screen shot I show setting up a 5 by 5 range with values to be transposed.

We can now select an available empty cell to insert our transpose computation. The following screenshot shows selecting the input range for calling our NMath transpose function.

As in our previous blog, the returned result is stored in a single cell and only one value is displayed. As a quick review to display the full result, first paint (select) the area for the data to be displayed with the function call in the upper left hand corner, then press the F2 key, next press Ctrl-Shift (hold), followed by the Enter key – this will build the array formula to populate the selected area with the result. Your result should look like the following screen.

As a further example of what is possible, we can tackle an least squares example from the NMath documentation. The problem is to calculate the residual norm square using the Cholesky method, something that would be awkward to do natively in Excel alone. In this example the inputs are the number of rows and columns of a DoubleMatrix uniformly filled with random numbers. To accomplish this we need to add the following code to our DNA text file.

using CenterSpace.NMath.Matrix;
  .
  .
public static object NDoubleCholeskyLeastSqRNS(int rsize, int csize, int RandLBx, int RandUBx)
{
	RandGenUniform randomGenUniform = new RandGenUniform(RandLBx,RandUBx);
	randomGenUniform.Reset(0x124);
	DoubleMatrix TempAr = new DoubleMatrix(rsize,csize,randomGenUniform);
	DoubleCholeskyLeastSq cholLsq = new DoubleCholeskyLeastSq(TempAr);
	if (cholLsq.IsGood)
	{
 		DoubleVector b = new DoubleVector(TempAr.Rows,randomGenUniform);
		DoubleVector x = cholLsq.Solve(b);
		return cholLsq.ResidualNormSqr(b);
	}
	else
		return "The random matrix doesn't have full rank";
}

Summary

Approaching the complex computations with Excel by performing the necessary computation by calling a C# library with data passed in from Excel, creates a robust approach to solving problems. The following screenshot shows how providing our function with the input parameters from the example produces the desired result.

If we tried to implement the solution line by line in Excel, we would be loosing the power of an object language like C# and powerful third-party .NET libraries, and increasing the generation of tedious Excel code without a strong development environment.

To wrap up, combining Excel, ExcelDNA, and the CenterSpace’s NMath libraries can be used to quickly generate solutions to complex problems without an extensive programming environment. In my next blog, I plan to solve a curve fitting example with this approach and using more of Excel features to display the results.

Mike Magee

The post Using C# and ExcelDNA to call .NET Libraries appeared first on CenterSpace.