1. Install Visual Studio Code.
2. Create a folder.
3. Run VS Code then open the folder with File|Open Folder….
4. View|Terminal to bring up a command-line.
5. In the terminal window, type dotnet new console to create a new project. You’ll see it creates a .csproj file and a Program.cs.
6. dotnet add package CenterSpace.NMath.Standard.Windows.X64 to add NMath.
7. Write some code in Program.cs.
8. Compile using dotnet build.
9. Run using dotnet run.

The post Getting Started with NMath appeared first on CenterSpace.

Release of NMath 7.2

NMath version 7.2 has been released and includes the following packages:

• CenterSpace.NMath.Standard.Windows.X64
• CenterSpace.NMath.Standard.Linux.X64
• CenterSpace.NMath.Standard.OSX.X64
• CenterSpace.NMath.Charting.Framework.Windows.X64

The post NMath and X86 appeared first on CenterSpace.

Beginning with the release of NMath 7.2 the following table lists the deprecated MS Solver Foundation classes on the left and their Google OR-Tools replacements, if necessary.

API Changes

The primary change between the deprecated MSF classes and the GORT classes is in the reduced algorithm parameterization. For example, in the toy MIP problem coded below the only change in the API regards constructing the new GORT solver and the lack of need for the parameter helper class. Note that the entire problem setup and related classes are unchanged making it a simple job to migrate to the new solver classes.

  // The problem setup is identical between the new and the deprecated API. 

  // minimize -3*x1 -2*x2 -x3
  var mip = new MixedIntegerLinearProgrammingProblem( new DoubleVector( -3.0, -2.0, -1.0 ) );

  // x1 + x2 + x3 <= 7
  mip.AddUpperBoundConstraint( new DoubleVector( 1.0, 1.0, 1.0 ), 7.0 );

  // 4*x1 + 2*x2 +x3 = 12
  mip.AddEqualityConstraint( new DoubleVector( 4.0, 2.0, 1.0 ), 12.0 );

  // x1, x2 >= 0
  mip.AddLowerBound( 0, 0.0 );
  mip.AddLowerBound( 1, 0.0 );

  // x3 is 0 or 1
  mip.AddBinaryConstraint( 2 );
 
  // Make a new Google OR-Tools solver and solve the MIP
  var solver = new PrimalSimplexSolverORTools();
  solver.Solve( mip, true ); // true -> minimize

  // Solve the same MIP with the old deprecated API
  var solver = new PrimalSimplexSolver();
  var solverParams = new PrimalSimplexSolverParams { Minimize = true };
  solver.Solve( mip, solverParams );

_{[1] MS Solver Foundation variable limits: NonzeroLimit = 100000, MipVariableLimit = 2000, MipRowLimit = 2000, MipNonzeroLimit = 10000, CspTermLimit = 25000, LP variable limit = 1000}

The post Updated NMath API for LP and MIP related classes appeared first on CenterSpace.

• Noise removal
• Baseline adjustment
• Peak finding
• Peak modeling
• Peak statistical analysis

In this blog article we will discuss each of these activities and provide some NMath C# code on how they may be accomplished. This is big subject but the goal here is to get you started solving your spectographic data analysis problems, perhaps introduce you to a new technique, and finally to provide some helpful code snippets that can be expanded upon.

Throughout this article we will be using the following electrophoretic data set below in our code examples. This data set contains four obvious peaks and one partially convolved peak infilled with underlying white noise.

Noise Removal

Chromatographic, spectographic, fMRI or EEG data, and many other types of time series are non-stationary. This non-stationarity means that Fourier based filtering methods are ill suited to removing noise from these signals. Fortunately we can effectively apply wavelet analysis, which does not depend on signal periodicity, to suppress the signal noise without altering the signal’s phase or magnitude. Briefly, the discrete wavelet transform (DWT) can be used to recursively decompose the signal successively into details and approximations components. From a filtering perspective the signal details contain the higher frequency parts and the approximations contain the lower frequency components. As you’d expect the inverse DWT can elegantly reconstruct the original signal but, to meet our noise removal goals, the higher frequency noisy parts of the signal can be suppressed during the signal reconstruction and be effectively removed. This technique is called wavelet shrinkage and is described in more detail in an earlier blog article with references.

These results can be refined, but this starting point is seen to have successfully removed the noise but not altered the position or general shape of the peaks. Choosing the right wavelet for wavelet shrinkage is done empirically with a representative data set at hand.

public DoubleVector SuppressNoise( DoubleVector DataSet  )  
{
  var wavelet = new DoubleWavelet( Wavelet.Wavelets.D4 );
  var dwt = new DoubleDWT( DataSet.ToArray(), wavelet );
  dwt.Decompose( 5 );
  double lambdaU = dwt.ComputeThreshold( FloatDWT.ThresholdMethod.Sure, 1 );

  dwt.ThresholdAllLevels( FloatDWT.ThresholdPolicy.Soft, new double[] { lambdaU, lambdaU, lambdaU, lambdaU, lambdaU } );

  double[] reconstructedData = dwt.Reconstruct();
  var filteredData= new DoubleVector( reconstructedData );
  return filteredData;
}

With our example data set a Daubechies 4 wavelet worked well for noise removal. Note that the same threshold was applied to all DWT decomposition levels; Improved white noise suppression can be realized by adopting other thresholding strategies.

Baseline Adjustment

Dozens of methods have been developed for modeling and removing a baseline from various types of spectra data. The R package baseline has collected together a range of these techniques and can serve as a good starting point for exploration. The techniques variously use regression, iterative erosion and dilation, spectral filtering, convex hulls, or partitioning and create baseline models of lines, polynomials, or more complex curves that can then be subtracted from the raw data. (Another R package, MALDIquant, contains several more useful baseline removal techniques.) Due to the wide variety of baseline removal techniques and the lack of standards across datasets, NMath does not natively offer any baseline removal algorithms.

Example baseline modeling

The C# example baseline modeling code below uses z-scores and iterative peak suppression to create a polynomial model of the baseline. Peaks that extend beyond 1.5 z-scores are iteratively cut down by a quarter and then a polynomial is fitted to this modified data set. Once the baseline polynomial fits well and stops improving upon iterative suppression the model is returned.

private PolynomialLeastSquares findBaseLine( DoubleVector x, DoubleVector y, int PolynomialDegree )
 {
   var lsFit = new PolynomialLeastSquares( PolynomialDegree, x, y );
   var previousRSoS = 1.0;

   while ( lsFit.LeastSquaresSolution.ResidualSumOfSquares > 0.1 && Math.Abs( previousRSoS - lsFit.LeastSquaresSolution.ResidualSumOfSquares ) > 0.00001 )
   {
     // compute the Z-scores of the residues and erode data beyond 1.5 stds.
     var residues = lsFit.LeastSquaresSolution.Residuals;
     var Zscores = ( residues - NMathFunctions.Mean( residues ) ) / Math.Sqrt( NMathFunctions.Variance( residues ) );
     previousRSoS = lsFit.LeastSquaresSolution.ResidualSumOfSquares;

     y[0] = Zscores[0] > 1.5 ? 0 : y[0];
     for ( int i = 1; i < this.Length; i++ )
     {
       if ( Zscores[i] > 1.5 )
       {
         y[i] = y[i-1] / 4.0;
       }
     }
     lsFit = new PolynomialLeastSquares( PolynomialDegree, x, y );
    }
    return lsFit;
 }

This algorithm has proven reliable for estimating both 1 and 2 degree polynomial baselines with electrophoretic data sets. It is not designed to model the wandering baselines sometimes found in mass spec data. The SNIP [2] method or Asymmetric Least Squares Smoothing [1] would be better suited for those data sets.

Peak Finding

Locating peaks in a data set usually involves, at some level, finding the zero crossings of the first derivative of the signal. However, directly differentiating a signal amplifies noise and so more sophisticated indirect methods are usually employed. Savitzky-Golay polynomials are commonly used to provide high quality smoothed derivatives of a noisy signal and are widely employed with chromatographic and other data sets (See this blog article for more details).

// Code snippet for locating peaks.
var sgFilter = new SavitzkyGolayFilter( 4, 4, 2 );
DoubleVector filteredData = sgFilter.Filter( DataSet );
var rbPeakFinder = new PeakFinderRuleBased( filteredData );
rbPeakFinder.AddRule( PeakFinderRuleBased.Rules.MinHeight, 0.005 );
List<int> pkIndicies = rbPeakFinder.LocatePeakIndices();

Without thresholding many small noisy undulations are returned as peaks. Thresholding with this data set works well in separating the data peaks from the noise, however sometimes peak modeling is necessary for separating data peaks from noise when they are both present at similar scales.

Peak Modeling and Statistics

In addition to separating out false peaks, peaks are also modeled to compute various peak statistical measures such as FWHM, CV, area, or standard deviation. The Gaussian is an excellent place to start for peak modeling and for many applications this model is sufficient. However there are many other peak models including: Lorenzian, Voigt, [3] CSR, and variations on exponentially modified Gaussians (EMG’s). Many combinations, convolutions, and refinements of these models are gathered together and presented in a useful paper by DeMarko & Bombi, 2001. Their paper focused on chromatographic peaks but the models surveyed therein have wide application.

/// <summary>
/// Gaussian Func<> for trust region fitter.
/// p[0] = mean, p[1] = sigma, p[2] = baseline offset
/// </summary>
private static Func<DoubleVector, double, double> Gaussian = delegate ( DoubleVector p, double x )
{
   double a = ( 1.0 / ( p[1] * Math.Sqrt( 2.0 * Math.PI ) ) );
   return a * Math.Exp( -1 * Math.Pow( x - p[0], 2 ) / ( 2 * p[1] * p[1] ) ) + p[2];
};

Above is a Func<> representing a Gaussian that allows for some vertical offset. The TrustRegionMinimizer in NMath is one of the most powerful and flexible methods for peak fitting. Once the start and end indices of the peaks are determined, the following code snippet fits this Gaussian model to the peak’s data.

// The DoubleVector's xValues and yValues contain the peak's data.

// Pass in the model (above) to the function fitter ctor
var modelFitter = new BoundedOneVariableFunctionFitter<TrustRegionMinimizer>( Gaussian );

// Gaussian for peak finding
var lowerBounds = new DoubleVector( new double[] { xValues[0], 1.0, -0.05 } );
var upperBounds = new DoubleVector( new double[] { xValues[xValues.Length - 1], 10.0, 0.05 } );
var initialGuess = new DoubleVector( new double[] { 0.16, 6.0, 0.001 } );

// The lower and upper bounds aren't required, but are suggested.
var soln = modelFitter.Fit( xValues, yValues, initialGuess, lowerBounds, upperBounds );

// Fit statistics
var gof = new GoodnessOfFit( modelFitter, xValues, yValues, soln );

The GoodnessOfFit class is a very useful tool for peak modeling. In one line of code one gets the f-statistics for the goodness of the fit of the model along with confidence intervals for all of the model parameters. These statistics are very useful in automating the sorting out of noisy peaks from actual data peaks and of course for determining if the model is appropriate for the data at hand.

Peak Area

Computing peak areas or peak area proportions is essential in most applications of spectographic or electrophoretic data analysis. This is this a two-liner with NMath.

// The peak starts and ends at: startIndex, endIndex.
var integrator = new DiscreteDataIntegrator();
double area =  integrator.Integrate( DataSet[ new Slice( startIndex, endIndex - startIndex + 1) ] );

The DiscreteDataIntegrator defaults to integrating with cubic spline segments. Other discrete data integration methods available are trapezoidal and parabolic.

Summary

Contact us if you need help or have questions about analyzing your team’s data sets. We can quickly help you get started solving your computing problems using NMath or go deeper and accelerate your team’s application development with consulting.

Assorted References

1. Eilers, Paul & Boelens, Hans. (2005). Baseline Correction with Asymmetric Least Squares Smoothing. Unpubl. Manuscr.
2. C.G. Ryan, E. Clayton, W.L. Griffin, S.H. Sie, and D.R. Cousens. 1988. SNIP, a statistics-sensitive background treatment for the quantitative analysis of pixe spectra in geoscience applications. Nuclear Instruments and Methods in Physics Research Section B: Beam Interactions with Materials and Atoms, 34(3): 396-402.
3. García-Alvarez-Coque MC, Simó-Alfonso EF, Sanchis-Mallols JM, Baeza-Baeza JJ. A new mathematical function for describing electrophoretic peaks. Electrophoresis. 2005;26(11):2076-2085. doi:10.1002/elps.200410370

The post Chromatographic and Spectographic Data Analysis appeared first on CenterSpace.

The Weibull probability distribution, over the random variable x, has two parameters:

• k > 0, is the shape parameter
• λ > 0, is the scale parameter

Frequently engineers have data that is known to be well modeled by the Weibull distribution but the shape and scale parameters are unknown. In this case a data fitting strategy can be used; NMath now has a maximum likelihood Weibull fitting function demonstrated in the code example below.

    public void WiebullFit()
    {
      double[] t = new double[] { 16, 34, 53, 75, 93, 120 };
      double initialShape = 2.2;
      double initialScale = 50.0;

      WeibullDistribution fittedDist = WeibullDistribution.Fit( t, initialScale, initialShape );

      // fittedDist.Shape parameter will equal 1.933
      // fittedDist.Scale parameter will equal 73.526
    }

If the Weibull fitting algorithm fails the returned distribution will be null. In this case improving the initial parameter guesses can help. The WeibullDistribution.Fit() function accepts either arrays, as seen above, or DoubleVectors.

The latest version of NMath, including this maximum likelihood Weibull fit function, is available on the CenterSpace NuGet gallery.

The post Fitting the Weibull Distribution appeared first on CenterSpace.

This version requires developers to be using at least .NET Core 2.0 or at least .NET Framework 4.6.1. Adding support of .NET Core to NMath has been in the works for over a year and was done at the request of many of our active developers.

Future development work will concentrate on the .NET Standard based NMath 7.0. However, NMath 6.2, build on .NET 4.0, but not supporting the .NET Standard Library, will be available for years to come.

Below is a list of major changes released in NMath 7.0:

• 32-bit support has been dropped. Demand has been waning for years for this. Dropping it has made usage simpler and easier.
• GPU support has been dropped. As developers, we liked the automatic GPU offloading. However, the technical advantages have dissipated as multi-core processors have improved. We believe that this is no longer compelling for a general math library.
• NMath Stats has been merged into NMath. This is for ease of use for our users.
• In summer of 2019, our pricing will be streamlined to reflect these changes. There will be one price for a perpetual NMath license and there will be one price for annual NMath maintenance which includes technical support and all upgrades available on NuGet. NMath Stats will no longer be sold separately.
• We have merged the four NMath namespaces into one, CenterSpace.NMath.Core, to simplify development. Originally, CenterSpace had four NMath products and four namespaces, these namespaces CenterSpace.NMath.Core, CenterSpace.NMath.Matrix, CenterSpace.NMath.Stats, CenterSpace.NMath.Analysis reflect that history. We have left stubs so users won’t have any breaking changes.
• We have dropped charting. The ecosystem is full of powerful visualization packages. We have only three main data structures in NMath, vectors, matrices and data frames, and all can be easily used with different charting packages.
• Some of our optimizations use Microsoft Solver Foundation. If you use these, you’ll need to be on the .NET Framework track and not on the .NET Core track.
• We have dropped the installers. The compelling ease of NuGet for our users has made these obsolete.

NMath 7.0 on Windows and Linux

NMath 7.0 on Windows

NMath 7.0 on Linux

NMath 7.0 on OSX

Please try the new versions on NuGet. Feedback welcome as always.

The post NMath 7.0 & the .NET Standard Library appeared first on CenterSpace.

With the release of NMath 6.2.0.41, on March 10, 2018, NMath no longer supports OSX or the Linux x86 operating systems. We are dropping the support of these operating systems due to a decline of demand by our customers. Please contact us with any concerns regarding this change. This release is currently available on NuGet.

Going forward NMath and NMath Premium will naturally continue to support both the 32-bit and 64-bit Windows and 64-bit Linux.

Adding .NET Standard and .NET Core Support

By the end of 2018, NMath will support both the .NET Core and .NET Standard. Supporting both of these .NET standards have been increasingly requested by our customers. If you are unfamiliar with these newest additions to the .NET world, the following briefly defines them.

• .NET Core: This is the latest .NET implementation. It’s open source and available for multiple OSes. With .NET Core, you can build cross-platform console apps and ASP.NET Core Web applications and cloud services.
• .NET Standard: This is the set of fundamental APIs (commonly referred to as base class library or BCL) that all .NET implementations must implement. By targeting .NET Standard, you can build libraries that you can share across all your .NET apps, no matter on which .NET implementation or OS they run.

For further reading on these .NET standards see this MSDN magazine article for an introduction.

Please don’t hesitate to contact us in the comments below or via email with any questions regarding these changes to the CenterSpace .NET NMath library.

The post NMath is Adding .NET Core Support and has Dropped Support of OSX and Linux86 appeared first on CenterSpace.

Recall that the least squares solution to the multiple linear problem is given by
(1)

And that problems occurred finding when the matrix
(2)

was close to being singular. The Principal Components Regression approach to addressing the problem is to replace in equation (1) with a better conditioned approximation. This approximation is formed by computing the eigenvalue decomposition for and retaining only the r largest eigenvalues. This yields the PCR solution:
(3)

where is an r x r diagonal matrix consisting of the r largest eigenvalues of
are the corresponding eigenvectors of . In this piece we shall develop code for computing the PCR solution using the NMath libraries.

[eds: This blog article is final entry of a three part series on principal component regression. The first article in this series, “Principal Component Regression: Part 1 – The Magic of the SVD” is here. And the second, “Principal Components Regression: Part 2 – The Problem With Linear Regression” is here.]

Review: Eigenvalues and Singular Values

In order to develop the algorithm, I want to go back to the Singular Value Decomposition (SVD) of a matrix and its relationship to the eigenvalue decomposition. Recall that the SVD of a matrix X is given by
(4)

Where U is the matrix of left singular vectors, V is the matrix of right singular vectors, and Σ is a diagonal matrix with positive entries equal to the singular values. The eigenvalue decomposition of is given by
(5)

Where the eigenvalues of X are the diagonal entries of the diagonal matrix and the columns of V are the eigenvectors of (V is also composed of the right singular vectors of X).
Recall further that if the matrix X has rank r then X can be written as
(6)

Where is the jth singular value (jth diagonal element of the diagonal matrix ), is the jth column of U, and is the jth column of V. An equivalent way of expressing the PCR solution (3) to the least squares problem in terms of the SVD for X is that we’ve replaced X in the solution (1) by its rank r approximation shown in (6).

Principal Components

The subject here is Principal Components Regression (PCR), but we have yet to mention principal components. All we have talked about are eigenvalues, eigenvectors, singular values, and singular vectors. We’ve seen how singular stuff and eigen stuff are related, but what are principal components?
Principal component analysis applies when one considers statistical properties of data. In linear regression each column of our matrix X represents a variable and each row is a set of observed value for these variables. The variables being observed are random variables and as such have means and variances. If we center the matrix X by subtracting from each column of X its corresponding mean, then we’ve normalized the random variables being observed so that they have zero mean. Once the matrix X is centered in this way, the matrix is then proportional to the variance/covariance for the variables. In this context the eigenvectors of are called the Principal Components of X. For completeness (and because they are used in discussing the PCR algorithm), we define two more terms.
In the SVD given by equation (4), define the matrix T by
(7)

The matrix T is called the scores for X. Note that T is orthogonal, but not necessarily orthonormal. Substituting this into the SVD for X yields
(8)

Using the fact that V is orthogonal we can also write
(9)

We call the matrix V the loadings. The goal of our algorithm is to obtain the representation given by equation (8) for X, retaining all the most significant principal components (or eigenvalues, or singular values – depending on where your heads at at the time).

Computing the Solution

Using equation (3) to compute the solution to our problem involves forming the matrix and obtaining its eigenvalue decomposition. This solution is fairly straight forward and has reasonable performance for moderately sized matrices X. However, in practice, the matrix X can be quite large, containing hundreds, even thousands of columns. In addition, many procedures for choosing the optimal number r of eigenvalues/singular values to retain involve computing the solution for many different values of r and comparing them. We therefore introduce an algorithm which computes only the number of eigenvalues we need.

The NIPALS Algorithm

We will be using an algorithm known as NIPALS (Nonlinear Iterative PArtial Least Squares). The NIPALS algorithm for the matrix X in our least squares problem and r, the number of retained principal components, proceeds as follows:
Initialize and . Then iterate through the following steps –

1. Choose as any column of
2. Let
3. Let
4. If is unchanged continue to step 5. Otherwise return to step 2.
5. Let
6. If stop. Otherwise increment j and return to step 1.

Properties of the NIPALS Algorithm

Let us see how the NIPALS algorithm produces principal components for us.
Let and write step (2) as
(10)

Setting in step 3 yields
(11)

This equation is satisfied upon completion of the loop 2-4. This shows that and are an eigenvalue and eigenvector of . The astute reader will note that the loop 2-4 is essentially the power method for computing a dominant eigenvalue and eigenvector for a linear transformation. Note further that using and equation (11) we obtain
(12)

After one iteration of the NIPALS algorithm we end up at step 5 with and
(13)

Note that and
are orthogonal:
(14)

Furthermore, since is initially picked as a column of , it is orthogonal to . Upon completion of the algorithm we form the following two matrices:

• , whose columns are the vectors , is orthogonal
• whose columns are the , is orthonormal.

(15)

If r is equal to the rank of X then, using the information obtained from equations (12) and (14), it follows that (15) yields the matrix decomposition (8). The idea behind Principal Components Regression is that after choosing an appropriate r the important features of X have been captured in . We then perform a linear regression with in place of X,
(16) .

The least squares solution then gives
(17)

Note that since is diagonal it is easy to invert. Also note that we left out the loadings matrix . This is due to the fact that the scores are linear combinations of the columns of X, and the PCR method amounts to singling out those combinations that are best for predicting y. Finally, using (9) and (16) we rewrite our linear regression problem as
(18)

From (18) we see that the PCR estimation is given by
(19) .

Steve

The post Principal Components Regression: Part 3 – The NIPALS Algorithm appeared first on CenterSpace.

Please check them out.

– Trevor

The post CenterSpace partner releases symbolic, computational library appeared first on CenterSpace.

Added functionality includes:

• Upgraded to Intel MKL 11.3 Update 2 with resulting performance increases.
• Updated NMath Premium GPU code to CUDA 7.5.
• Added classes for performing Discrete Wavelet Transforms (DWTs) using most common wavelet families, including Harr, Daubechies, Symlet, Best Localized, and Coiflet.
• Added classes for solving stiff ordinary differential equations. The algorithm uses higher order methods and smaller step size when the solution varies rapidly.
• Added classes for performing two-way ANOVA with unbalanced designs.
• Added classes for performing Partial Least Squares Discriminant Analysis (PLS-DA), a variant of PLS used when the response variable is categorical.

For more complete changelogs, see:

• NMath changelog
• NMath Stats changelog

Upgrades are provided free of charge to customers with current annual maintenance contracts. To request an upgrade, please visit our upgrade page, or contact sales@centerspace.net. Maintenance contracts are available through our webstore.

The post Announcing NMath 6.2 and NMath Stats 4.2 appeared first on CenterSpace.