Added functionality includes:

• Upgraded to Intel MKL 11.3 Update 2 with resulting performance increases.
• Updated NMath Premium GPU code to CUDA 7.5.
• Added classes for performing Discrete Wavelet Transforms (DWTs) using most common wavelet families, including Harr, Daubechies, Symlet, Best Localized, and Coiflet.
• Added classes for solving stiff ordinary differential equations. The algorithm uses higher order methods and smaller step size when the solution varies rapidly.
• Added classes for performing two-way ANOVA with unbalanced designs.
• Added classes for performing Partial Least Squares Discriminant Analysis (PLS-DA), a variant of PLS used when the response variable is categorical.

For more complete changelogs, see:

• NMath changelog
• NMath Stats changelog

Upgrades are provided free of charge to customers with current annual maintenance contracts. To request an upgrade, please visit our upgrade page, or contact sales@centerspace.net. Maintenance contracts are available through our webstore.

The post Announcing NMath 6.2 and NMath Stats 4.2 appeared first on CenterSpace.

Added functionality includes:

• Upgraded to Intel MKL 11.2 Update 2 with resulting performance increases.
• Updated NMath Premium GPU code to CUDA 6.
• Added classes for solving linear and nonlinear programming problems with integer or binary constraints.
• Added class SpecialFunctions containing special functions such as factorial, binomial, the gamma function and related functions, Bessel functions, elliptic integrals, and many more. (Prior versions of a few of these functions, such as StatsFunctions.IncompleteGamma, are now deprecated.)
• Added a new native library, nmath_sf_x86.dll and nmath_sf_x64.dll, with high-performance C language implementations of the special functions.
• Added single-precision versions of general sparse matrix and vector types.

For more complete changelogs, see here and here.

Upgrades are provided free of charge to customers with current annual maintenance contracts. To request an upgrade, please visit our upgrade page, or contact sales@centerspace.net. Maintenance contracts are available through our webstore.

The post Announcing NMath 6.1 and NMath Stats 4.1 appeared first on CenterSpace.

Added functionality includes:

• Upgraded to Intel MKL 11.1 Update 3 with resulting performance increases.
• Added Adaptive Bridge™ technology to NMath Premium edition, with support for multiple GPUs, per-thread control for binding threads to GPUs, and automatic performance tuning of individual CPU–GPU routing to insure optimal hardware usage.
• NMath linear programming, nonlinear programming, and quadratic programming classes are now built on the Microsoft Solver Foundation (MSF). The Standard Edition of MSF is included with NMath.
• Added classes for solving nonlinear programming problems using the Stochastic Hill Climbing algorithm, for solving quadratic programming problems using an interior point algorithm, and for solving constrained least squares problems using quadratic programming methods.
• Added support for MKL Conditional Numerical Reproducibility (CNR).

For more complete changelogs, see here and here.

Upgrades are provided free of charge to customers with current annual maintenance contracts. To request an upgrade, please contact sales@centerspace.net. Maintenance contracts are available through our webstore.

The post Announcing NMath 6.0 and NMath Stats 4.0 appeared first on CenterSpace.

Jaroslav says:

I use NMath in RobomapStudio for evolutionary computations and robot pose estimation for navigation in known and partially known environments using 2D laser data. My  Covariance Matrix Adaptation Evolution Strategy (CMAES) implementation is completely built on NMath. I also use NMath in a variety of other ways–for example, for various types of statistical distribution generating (Cauchy, Gauss etc.). NMath provides significantly better results (accuracy) and stability in comparison to the Iridium (Math.NET) library, for example. NMath is a great tool to solve, simply and easily, many numerical problems in MS VS C++/CLI .NET language.

RobomapStudio is part of the international OpenSLAM project of the University of Freiburg. Results of the RobomapStudio are used in several research groups at the Czech Technical University in Prague (for example, here and here), and by many other research centers around the world.

For more information on Jaroslav’s work using NMath, see his recent publication in the journal Evolutionary Intelligence (Sept. 2013).

How are you using NMath? We’re always interested in hearing about interesting applications of NMath “in the wild,” and in receiving suggestions for how NMath can be improved. Let us know at info@centerspace.net.

Ken

The post Customer Story: Evolutionary Robotics Using NMath appeared first on CenterSpace.

Version 5.3 of NMath adds:

• IEnumerable<T> support for matrix classes, facilitating their use with LINQ.
• An in-place solve option for LU factorization.
• Improved support for NMath configuration within ASP.NET web applications.
• Matrix and vector visualizers for debugging in Visual Studio 2012.
• Performance increases in several areas, due to an upgrade to Intel MKL 11.0 Update 3.
• Here is a complete changelog.

Version 3.6 of NMath Stats adds:

• New LogisticRegression and related classes for performing binomial logistic regression.
• New classes for process control statistics, such as Cp, Cpm, Cp, Pp, and Ppk.
• Here is a complete changelog.

Upgrades are provided free of charge to customers with current annual maintenance contracts. Maintenance contracts are available through our webstore.

We will shortly also be announcing the general availability of the new Premium Edition of NMath and NMath Stats, which provides GPU acceleration of linear algebra (dense systems) and 1D and 2D FFT. We’ve gotten great feedback from users in our beta program, and are excited to make this option available to everyone. We think you will be too, when you see how easy it is to add GPU acceleration to your existing NMath applications. Check out our whitepaper, NMath Premium: GPU-Accelerated Math Libraries for .NET, and watch this space for more information soon.

The post Announcing NMath 5.3 and NMath Stats 3.6 appeared first on CenterSpace.

If no license key is found at runtime, a default evaluation license key is used which provides a free 30-day evaluation period for NMath on the current machine.

Three mechanisms are supported for setting your NMath license key:

• by setting the NMATH_LICENSE_KEY environment variable
• by setting the NMathLicenseKey configuration app setting
• by programmatically setting the LicenseKey property on class NMathConfiguration

Settings are applied in that order, and resetting the license key takes precedent over any earlier values. For example, here the environment variable is used:

 > set NMATH_LICENSE_KEY="<your key here>"

This code uses an app config file:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <appSettings>
    <add key="NMathLicenseKey" value="<your key here>" />
  </appSettings>
</configuration>

And this code accomplishes the same thing programmatically:

NMathConfiguration.LicenseKey = "<your key here>";

Note 1: Setting your license key works the same way for development and deployment machines.

Note 2: If you forget to set your license key on a machine, your code NMath code will work initially, using the default evaluation license, but this license will expire in 30 days.

The post Setting the NMath License Key appeared first on CenterSpace.

Changes for version 5.2 of NMath include:

• Upgraded to Intel MKL 10.3 Update 11 with resulting performance increases.
• Added class NMathConfiguration for controlling the loading of the NMath
  license key, kernel assembly, and native library. Logging can be enabled for debugging configuration issues.
  License files are no longer used. See here for more information.
• Replaced all custom NMath delegate types in the API with Func or Action
  equivalents, and deprecated the older signatures.
• Added support for postive and negative strided signals in all FFT classes.

A complete changelog is located here.

Version 3.5 of NMath Stats adds:

• Classes DoubleFactorAnalysis, FactorAnalysisCorrelation, FactorAnalysisCovariance, and supporting types for performing factor analysis.
• Class OneSampleAndersonDarlingTest for performing an Anderson-Darling test of the distribution of one sample.
• Class ShapiroWilkTest for testing the null hypothesis that a sample comes from a normally distributed population.
• Method NMathFunctions.FishersExactTest() for computing the Fisher’s Exact Test p-value for a specified 2 x 2 contingency table.

A complete changelog is located here.

The new release is build-compatible with the prior release. Upgrades are provided free of charge to customers with current annual maintenance contracts. Maintenance contracts are available through our webstore.

The post New Versions of NMath Libraries Released appeared first on CenterSpace.

An NMath license file is no longer used.

NMath License Key

NMath license information is stored in a license key which must be found at runtime. The license key governs the properties of your NMath installation. If no license key is found, a default evaluation license key is used which provides a free 30-day evaluation period for NMath on the current machine.

When you purchase one or more developer seats of NMath, you will be issued a license key describing the terms of your license. To enter your license key:

1. Open CenterSpace Software | License NMath from your Start menu.
2. Enter your name, email, and license key, and click OK.

The license key will be written to the registry. You can also specify your license key using various other mechanisms: by environment variable, by configuration app setting, and programmatically. These mechanisms may be preferable in group development environments, and at deployment. (See below.)

NMath Configuration

NMath configuration settings govern the loading of the NMath license key, kernel assembly, and native library. Property values can be set three ways:

• by environment variable
• by configuration app setting
• by programmatically setting properties on class NMathConfiguration

Settings are applied in that order, and resetting a property takes precedent over any earlier values. For example, here an environment variable is used:

 > set NMATH_NATIVE_LOCATION="C:\tmp"

This code uses an app config file:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <appSettings>
    <add key="NMathNativeLocation" value="C:\tmp" />
  </appSettings>
</configuration>

And this code accomplishes the same thing programmatically:

NMathConfiguration.NativeLocation = @"C:\tmp";

Also, all paths can be specified relative to the executable. So to place the natives in an existing directory named resources adjacent to the executable one would
write:

NMathConfiguration.NativeLocation = @"..\resources";

The supported environment variables, configuration app setting keys, and property names are show below.

Environment Variable
Configuration Setting
Property


NMATH_LOG_LOCATION
NMathLogLocation
LogLocation


NMATH_LICENSE_KEY
NMathLicenseKey
LicenseKey


NMATH_NATIVE_LOCATION
NMathNativeLocation
NativeLocation


NMATH_USE_SEQUENTIAL_THREADING
NMathUseSequentialThreading
UseSequentialThreading


NMATH_USE_EXTERNAL_THREADING
NMathUseExternalThreading
UseExternalThreading

NOTE- Assembly loading and license checking is normally performed the first time you make an NMath call. If you wish to explicitly control when these operations occur–at application start-up, for example–use the static NMathConfiguration.Init() method.

For example, when using Mono on a Mac, to configured NMath with a sequentially threaded library set the environmental variable as follows.

 > export NMATH_USE_SEQUENTIAL_THREADING="True" 

Logging

To debug configuration issues, specify a log file location. For example, setting the property programmatically to place the NMath log file in the same directory as the executable one would write:

NMathConfiguration.LogLocation = @".";

Or if you prefer to use a global temporary directory you could specify an absolute path.

NMathConfiguration.LogLocation = @"C:\temp\logs";

This creates a file named NMathConfiguration.log at the specified location containing log output. The specified location must exist. To turn off logging, set the log location to null.

License Key

You can specify your NMath license key using the LicenseKey property, or the equivalent environment variable or app config setting. If so, any keys in the registry are ignored.

Native Location

The NMath native assembly must be found at runtime. Failure to locate this file is one of the most common configuration issues, especially in deployment. The search order is determined by your PATH (on Windows systems). Some standard locations are automatically prepended to your (process-specific) PATH. You can also use the NativeLocation property, or the equivalent environment variable or app config setting, to prepend another location.

Alternative Kernel and Native Assemblies

The names of the NMath kernel and native assemblies are determined by your platform (x86 or x64), and the values of the UseSequentialThreading and UseExternalThreading properties, as shown below:

Standard
     Kernel
          NMathKernelx86.dll
          NMathKernelx64.dll
     Native
          nmath_native_x86.dll
          nmath_native_x64.dll

Sequentially-Threaded
     Kernel
          NMathKernelx86Sequential.dll
          NMathKernelx64Sequential.dll
     Native
          nmath_native_x86_seq.dll
          nmath_native_x64_seq.dll

Externally-Threaded
     Kernel
          NMathKernelx86External.dll
          NMathKernelx64External.dll
     Native
          nmath_native_x86_ext.dll
          nmath_native_x64_ext.dll

Sequentially-threaded and externally-threaded kernel and native assemblies are available upon request from CenterSpace Software:

• Sequentially-Threaded: MKL contains highly optimized, extensively threaded math routines. In rare cases, these can cause conflicts between the Intel OMP threading library (libiomp.dll) and the .NET threading model. If your .NET application is itself highly multi-threaded, you may wish to use the sequentially-threaded version of MKL.
• Externally-Threaded: NMath normally statically links in the Intel OMP threading library described above. Sometimes this can cause collisions with libraries from other vendors that also use OMP. The externally-threaded version of NMath dynamically-links in OMP.

To trigger loading of these assemblies, use properties UseSequentialThreading and UseExternalThreading on class NMathConfiguration, or the equivalent environment variables or app config settings. Both properties default to false.

UseSequentialThreading and UseExternalThreading are mutually exclusive, and UseSequentialThreading takes precedence; UseExternalThreading only has an effect if UseSequentialThreading is false.

The post NMath Configuration appeared first on CenterSpace.

Let’s start by creating a data set: 100 values drawn from a normal distribution with known parameters (mean = 0.5, variance = 2.0).

int n = 100;
double mean = .5;
double variance = 2.0;
var data = new DoubleVector( n, new RandGenNormal( mean, variance ) );

Now, compute y values based on the empirical cumulative distribution function (CDF), which returns the probability that a random variable X will have a value less than or equal to x–that is, f(x) = P(X <= x). Here’s an easy way to do, although not necessarily the most efficient for larger data sets:

var cdfY = new DoubleVector( data.Length );
var sorted = NMathFunctions.Sort( data );
for ( int i = 0; i < data.Length; i++ )
{
  int j = 0;
  while ( j < sorted.Length && sorted[j] <= data[i] ) j++;
  cdfY[i] = j / (double)data.Length;
}

The data is sorted, then for each value x in the data, we iterate through the sorted vector looking for the first value that is greater than x.

We’ll use one of NMath’s non-linear least squares minimization routines to fit a normal distribution CDF() function to our empirical CDF. NMath provides classes for fitting generalized one variable functions to a set of points. In the space of the function parameters, beginning at a specified starting point, these classes finds a minimum (possibly local) in the sum of the squared residuals with respect to a set of data points.

A one variable function takes a single double x, and returns a double y:

y = f(x)

A generalized one variable function additionally takes a set of parameters, p, which may appear in the function expression in arbitrary ways:

y = f(p1, p2,..., pn; x)

For example, this code computes y=a*sin(b*x + c):

public double MyGeneralizedFunction( DoubleVector p, double x )
{
  return p[0] * Math.Sin( p[1] * x + p[2] );
}

In the distribution fitting example, we want to define a parameterized function delegate that returns CDF(x) for the distribution described by the given parameters (mean, variance):

Func f =
  ( DoubleVector p, double x ) =>
    new NormalDistribution( p[0], p[1] ).CDF( x );

Now that we have our data and the function we want to fit, we can apply the curve fitting routine. We’ll use a bounded function fitter, because the variance of the fitted normal distribution must be constrained to be greater than 0.

var fitter = new BoundedOneVariableFunctionFitter( f );
var start = new DoubleVector( new double[] { 0.1, 0.1 } );
var lowerBounds = new DoubleVector( new double[] { Double.MinValue, 0 } );
var upperBounds = 
   new DoubleVector( new double[] { Double.MaxValue, Double.MaxValue } );
var solution = fitter.Fit( data, cdfY, start, lowerBounds, upperBounds );
var fit = new NormalDistribution( solution[0], solution[1] );

Console.WriteLine( "Fitted distribution:\nmean={0}\nvariance={1}",
  fit.Mean, fit.Variance );

The output for one run is

Fitted distribution: 
mean=0.567334190790594
variance=2.0361207956132

which is a reasonable approximation to the original distribution (given 100 points).

We can also visually inspect the fit by plotting the original data and the CDF() function of the fitted distribution.

ToChart( data, cdfY, SeriesChartType.Point, fit,
  NMathStatsChart.DistributionFunction.CDF );

private static void ToChart( DoubleVector x, DoubleVector y,
  SeriesChartType dataChartType, NormalDistribution dist,
  NMathStatsChart.DistributionFunction distFunction )
{
  var chart = NMathStatsChart.ToChart( dist, distFunction );
  chart.Series[0].Name = "Fit";

  var series = new Series() {
    Name = "Data",
    ChartType = dataChartType
  };
  series.Points.DataBindXY( x, y );
  chart.Series.Insert( 0, series );

  chart.Legends.Add( new Legend() );
  NMathChart.Show( chart );
}

We can also look at the probability density function (PDF) of the fitted distribution, but to do so we must first construct an empirical PDF using a histogram. The x-values are the midpoints of the histogram bins, and the y-values are the histogram counts converted to probabilities, scaled to integrate to 1.

int numBins = 10;
var hist = new Histogram( numBins, data );

var pdfX = new DoubleVector( hist.NumBins );
var pdfY = new DoubleVector( hist.NumBins );
for ( int i = 0; i < hist.NumBins; i++ )
{
  // use bin midpoint for x value
  Interval bin = hist.Bins[i];
  pdfX[i] = ( bin.Min + bin.Max ) / 2;

   // convert histogram count to probability for y value
   double binWidth = bin.Max - bin.Min;
   pdfY[i] = hist.Count( i ) / ( data.Length * binWidth );
}

ToChart( pdfX, pdfY, SeriesChartType.Column, fit,
  NMathStatsChart.DistributionFunction.PDF );

You might be tempted to try to fit a distribution PDF() function directly to the histogram data, rather than using the CDF() function like we did above, but this is problematic for several reasons. The bin counts have different variability than the original data. They also have a fixed sum, so they are not independent measurements. Also, for continuous data, fitting a model based on aggregated histogram counts, rather than the original data, throws away information.

Ken

Download the source code

The post Distribution Fitting Demo appeared first on CenterSpace.

As previously described, NMath 5.1 and NMath Stats 3.4 include classes for plotting NMath types using the Microsoft Chart Controls for .NET. (Free adapter code is also available for using NMath with Syncfusion Essential Chart.) Let’s use Anscombe’s data to explore how NMath’s new visualization capabilities can be used to reveal the differences in the data sets.

First, we’ll load the data into a DoubleMatrix.

DoubleMatrix A = new DoubleMatrix( @"11x8 [
  10.0 8.04  10.0 9.14 10.0 7.46  8.0  6.58
  8.0  6.95  8.0  8.14 8.0  6.77  8.0  5.76
  13.0 7.58  13.0 8.74 13.0 12.74 8.0  7.71
  9.0  8.81  9.0  8.77 9.0  7.11  8.0  8.84
  11.0 8.33  11.0 9.26 11.0 7.81  8.0  8.47
  14.0 9.96  14.0 8.10 14.0 8.84  8.0  7.04
  6.0  7.24  6.0  6.13 6.0  6.08  8.0  5.25
  4.0  4.26  4.0  3.10 4.0  5.39  19.0 12.50
  12.0 10.84 12.0 9.13 12.0 8.15  8.0  5.56
  7.0  4.82  7.0  7.26 7.0  6.42  8.0  7.91
  5.0  5.68  5.0  4.74 5.0  5.73  8.0  6.89 ]" );

Now let’s perform some simple descriptive statistics.

 int groups = 4;
 Slice rows = Slice.All;
 Slice xCols = new Slice( 0, groups, 2 );
 Slice yCols = new Slice( 1, groups, 2 );
 double unbiased = (double)A.Rows / ( A.Rows - 1 );

 Console.WriteLine( "Mean of x: {0}",
   NMathFunctions.Mean( A[ rows, xCols ] ) );
 Console.WriteLine( "Variance of x: {0}",
   NMathFunctions.Variance( A[rows, xCols] ) * unbiased );
 Console.WriteLine( "Mean of y: {0}",
   NMathFunctions.Round( NMathFunctions.Mean( A[rows, yCols] ), 2 ) );
 Console.WriteLine( "Variance of y: {0}",
   NMathFunctions.Round(
     NMathFunctions.Variance( A[rows, yCols] ) * unbiased, 3 ) );

 Console.Write( "Correlation of x-y: " );
 for (int i = 0; i < A.Cols; i += 2 )
 {
   Console.Write( NMathFunctions.Round(
    StatsFunctions.Correlation( A.Col(i), A.Col(i + 1) ), 3 ) + " " );
 }
 Console.WriteLine();

You can see from the output that the statistics are nearly identical for all four data sets:

Mean of x: [ 9 9 9 9 ]
Variance of x: [ 11 11 11 11 ]
Mean of y: [ 7.5 7.5 7.5 7.5 ]
Variance of y: [ 4.127 4.128 4.123 4.123 ]
Correlation of x-y: 0.816 0.816 0.816 0.817

Now let's fit a linear model to each data set.

 LinearRegression[] lrs = new LinearRegression[groups];

 for( int i = 0; i < groups; i ++ )
 {
   Console.WriteLine( "Group {0}", i + 1 );

   bool addIntercept = true;
   lrs[i] = new LinearRegression( new DoubleMatrix( A.Col( 2 * i ) ),
     A.Col( 2 * i + 1 ), addIntercept );
   Console.WriteLine( "equation of regression line: Y = {0} + {1}X",
     Math.Round( lrs[i].Parameters[0], 2 ),
     Math.Round( lrs[i].Parameters[1], 3 ) );

   LinearRegressionParameter param =
     new LinearRegressionParameter( lrs[i], 1 );
   Console.WriteLine( "standard error of estimate of slope: {0}",
     Math.Round( param.StandardError, 3 ) );
   Console.WriteLine( "t-statistic: {0}",
     Math.Round( param.TStatistic( 0 ), 2 ) );

   LinearRegressionAnova anova = new LinearRegressionAnova( lrs[i] );
   Console.WriteLine( "regression sum of squares: {0}",
     Math.Round( anova.RegressionSumOfSquares, 2 ) );
   Console.WriteLine( "residual Sum of squares: {0}",
     Math.Round( anova.ResidualSumOfSquares, 2 ) );
   Console.WriteLine( "r2: {0}", Math.Round( anova.RSquared, 2 ) );

   Console.WriteLine();
 }

Again, the output is nearly identical for each data set:

Group 1
equation of regression line: Y = 3 + 0.5X
standard error of estimate of slope: 0.118
t-statistic: 4.24
regression sum of squares: 27.51
residual Sum of squares: 13.76
r2: 0.67

Group 2
equation of regression line: Y = 3 + 0.5X
standard error of estimate of slope: 0.118
t-statistic: 4.24
regression sum of squares: 27.5
residual Sum of squares: 13.78
r2: 0.67

Group 3
equation of regression line: Y = 3 + 0.5X
standard error of estimate of slope: 0.118
t-statistic: 4.24
regression sum of squares: 27.47
residual Sum of squares: 13.76
r2: 0.67

Group 4
equation of regression line: Y = 3 + 0.5X
standard error of estimate of slope: 0.118
t-statistic: 4.24
regression sum of squares: 27.49
residual Sum of squares: 13.74
r2: 0.67

Finally, let's use the new NMath charting functionality to plot each linear regression fit. Note that we make use of the Compose() method to combine multiple charts into a single composite Chart control.

 List charts = new List();
 for( int i = 0; i < lrs.Length; i++ )
 {
   charts.Add( NMathStatsChart.ToChart( lrs[i], 0 ) );
 }
 Chart all = NMathStatsChart.Compose( charts, 2, 2,
   NMathChart.AreaLayoutOrder.RowMajor );
 for( int i = 0; i < groups; i++ )
 {
   all.ChartAreas[i].AxisX.Title = "x" + ( i + 1 );
   all.ChartAreas[i].AxisX.Minimum = 2;
   all.ChartAreas[i].AxisX.Maximum = 22;
   all.ChartAreas[i].AxisX.Interval = 4;

   all.ChartAreas[i].AxisY.Title = "y" + ( i + 1 );
   all.ChartAreas[i].AxisY.Minimum = 2;
   all.ChartAreas[i].AxisY.Maximum = 14;
   all.ChartAreas[i].AxisY.Interval = 4;

   all.Series[2 * i].Color = Color.DarkOrange;
   all.Series[2 * i + 1].Color = Color.SteelBlue;
 }
 NMathStatsChart.Show( all );

The charts reveal  dramatic differences between the data sets, despite the identical fitted models. Group 1 shows a linear relationship, while in Group 2 the releationship is clearly non-linear.  Groups 3 and 4 demonstrate how a single outlier can have a large effect on simple statistics.

Ken

References

Anscombe, F. J. (1973). "Graphs in Statistical Analysis". American Statistician 27 (1): 17–21. JSTOR 2682899.
Tufte, Edward R. (2001). The Visual Display of Quantitative Information (2nd ed.). Cheshire, CT: Graphics Press

The post The Importance of Graphing Your Data appeared first on CenterSpace.