by finding the roots to,

Algorithms

I will demonstrate three well known univariate root finding algorithms, Newton-Raphson, Ridders, and Secant. All three methods require a delegate to the function and a bracketed range where a root is expected to reside, additionally, Newton-Raphson requires a continuous, non-zero derivative of f(x)in the region of the root. None of these methods are recommended for finding roots of polynomials in general, although Newton-Raphson will work well if the polynomial isn’t ill-conditioned by containing an even number of duplicated roots (Bracketing such roots isn’t possible because the function doesn’t change sign in the region of these roots.), or by containing root pairs extremely close together.

C# Example with NMath

All of these roots finders have a similar interface and properties in NMath, so exchanging one with the other is easy. As seen in the example below, both the Ridders and Secant root finders implement the IOneVariableRootFinder interface, and the Newton-Raphson root finder implements the slightly different IOneVariableDRootFinder, where the ‘D’ stands for derivative.

Using Wolfram|Alpha’s online services, I’ve computed and graphed the roots of the following non-linear univariate function.

The code block below demonstrates the use of each of these root finders, and shows the how to define a function delegate using a lamda expression. I find lamda expressions are convenient for defining delegates to mathematical expressions and that they create very readable code. (As an aside, lamda expressions are a C# language feature, and are not part of the .NET framework. Therefore, applications targeting the .NET 2.0 framework can leverage lamda expressions as long as the compiler supports them, as VS9 and VS10 both do.).

using CenterSpace.NMath.Analysis;
 
void RootFindingExample()
    {
 
     // Create the function delegate with a lamda expression, 
     // and the interval which contains a root.
      OneVariableFunction f = 
        new OneVariableFunction(x => 9 * Math.Log(x) * Math.Sin(x) - x * x * x);
      Interval bracket = new Interval(1, 1.5, Interval.Type.Closed);
 
      IOneVariableDRootFinder nrRoots = new NewtonRaphsonRootFinder(3);
      Double rootnewton = nrRoots.Find(f, f.Derivative(), bracket);
      Console.WriteLine(String.Format("Newton-Raphson root is = {0}", rootnewton));
 
      IOneVariableRootFinder rootFinder = new RiddersRootFinder(3);
      Double rootridder = rootFinder.Find(f, bracket);
      Console.WriteLine(String.Format("Ridder's root is = {0}", rootridder));
 
      rootFinder = new SecantRootFinder(3);
      Double rootsecant = rootFinder.Find(f, bracket);
      Console.WriteLine(String.Format("Secant root is = {0}", rootsecant));
 
    }

With each root finder limited to just 3 iterations, this program creates the output,

Newton-Raphson root is = 1.26760302048967
Ridder's root is = 1.26760302063032
Secant root is = 1.3334994219432

Where Newton-Raphson found the first root most accurately with it’s knowledge of derivative information. Besides specifying the maximum number of iterations, a root tolerance can also be used to control convergence effort.

A second somewhat more difficult example which can cause problems for both the Secant and Newton-Raphson methods.

I gave the Secant method 1000 iterations, and still it’s root estimate was very poor – in fact with an initial bracket of [-5, -0.01], the Secant method will never converge on the root. There is a stable orbit around which the Secant method spins forever, preventing convergence. Because the Newton-Raphson requires a continuous derivative any starting bracket that contains the origin will cause an exception to be thrown because of the discontinuous derivative at zero.

Newton-Raphson root is = -1.05539938820662 (with 8 iterations)
Ridder's root is = -1.05539938548287 (with 8 iterations)
Secant root is = -0.0912954387917323 (with 1000 iterations)

As a general recommendation, I would start my root search using Ridder’s method and then finish up by polishing with Newton-Raphson.

-Happy Computing,
Paul

” We were busting at the seams so it was time for a move.” said CenterSpace CEO Trevor Misfeldt  “Our team is growing so we can expand the functionality of the NMath software and provide more consulting services to our customers.”

Our new location is located on the third floor of the historic Crees Building in downtown Corvallis Oregon.

CenterSpace Software

230 SW 3rd Street

Corvallis OR, 97330

+1.541.896.1301

Art Deco Crees Building Front Address

At the time of its construction in 1926, the Crees Building was the largest commercial building to ever be built in downtown Corvallis and it has played a major role in the commerce of the area.   Visitors can take a ride to our offices on the third floor in the oldest working (we hope!) elevator in Corvallis.

Crees Building Signage

“With almost twice the square footage of our former location, we will not only be able to handle the growth of our staff but also provide a unique environment for our new series of quarterly training classes starting in August 2010.” says Misfeldt

About CenterSpace Software
CenterSpace Software is a leading provider of enterprise class numerical component libraries for the .NET platform. Developers worldwide use CenterSpace products to develop .NET financial, engineering, and scientific applications. CenterSpace Software has offices in Corvallis, OR, and can be found on the Internet at http://www.centerspace.net.

Pure C#

Both libraries are now supported by a new pure C# math kernel doing away with our old C++ kernel. Because we are now a pure .NET assembly, deployment of NMath based applications is simplified by eliminating the Microsoft C++ runtime library dependency. As with all releases we will be posting our updated performance benchmarks at the time of the release.

Full Control

Additionally, our libraries have been re-architected to dynamically link to both native numerical libraries and ( and perhaps more importantly for our customers ) the Intel OMP threading library (libiomp.dll) . This means that our customers will have complete control over the threading library. In the past, we statically linked in OMP. Now, we are picking up OMP dynamically and thereby avoid collisions between statically and dynamically linked OMP libraries. In a nutshell, NMath will now play more nicely with libraries from other vendors.

New Features

Now for the fun stuff. The table below summarized the new features in NMath 4.1 and NMath Stats 3.2.

I hope you find these new additions to the library useful in your application work. If you are looking for something specific that isn’t currently supported in our library, please contact us. We build custom numeric classes for existing and new customers on a regular basis.

Happy Computing,
Paul

Partnership with Innovative Genomic Services Company

Corvallis, Oregon (May 13, 2009) – CenterSpace Software, a leading provider of enterprise class numerical component libraries for the .NET platform, and Floragenex, an innovative genomic research services company, today announced that they have teamed up to build a flexible genomics data analysis pipeline in the cloud.

Rapid growth of the research services business at Floragenex requires expansion of computing resources beyond the company’s current infrastructure. CenterSpace has deep knowledge of high-powered analytical software and experience developing applications utilizing the cloud computing power of Amazon Web Services. Together, the two companies are working to build a cloud-computing infrastructure to improve ease of use, lower costs, and accelerate data throughput for Floragenex.

“The new system retrieves genetic sequence data from the sequencing facility of choice, places it into storage in the cloud and runs analysis based on the particular needs of the project. The Floragenex end user can login to a web page to select their data, choose their parameters and run their analysis. The work is done on Amazon Elastic Compute Cloud (EC2) computers and results are archived on Amazon S3 for easy retrieval. The costs are small and the benefits large,” says Trevor Misfeldt, CEO of CenterSpace Software.

“The massive volume of data that genome sequencers produce has required us to look for creative ways to scale our business. Working with CenterSpace allows us to lay the foundation for continued growth without making capital investments in IT hardware,” says Nathan Lillegard, CEO of Floragenex. “The bioinformatics work we do for our customers is particularly well suited to cloud computing, with large sets of data requiring intensive but discreet data processing. Building a cloud infrastructure now will allow us to grow our business and expand our service offerings more responsively.”

About the Companies

CenterSpace Software is a leading provider of enterprise class numerical component libraries for the .NET platform. Developers worldwide use CenterSpace products to develop .NET financial, engineering, and scientific applications. CenterSpace Software has offices in Corvallis, OR, and can be found on the Internet at www.centerspace.net.


Floragenex is a research services company focused on genomic technology applications in plant sciences. Floragenex enables scientists to do more, expanding their capability using a suite of advanced genomic services matched to the specific resources and goals of each project. Floragenex is based in Eugene, OR, and can be found on the internet at www.floragenex.com.

###

Free Examples

Currently there are two free examples, one using the Infragistics .NET WinForm’s Chart tool and a second using Infragisitcs .NET WPF Chart tool each deriving data from CenterSpace’s math libraries. The two example are summarized below.

• The Window Forms C# example demonstrates Savitzy-Golay data smoothing in a single Infragsitics chart along with two filtering controls. This sample includes a helper class to organize NMath Vectors into a data table with column headers that is consumed by the Infragistics chart.
• 

  Periodogram of sun spot data.

  The WPF C# example demonstrates estimating the power spectral density of historic sun spot data using NMath’s FFT and filtering classes. The raw sun spot data, periodogram, and the power spectral density are shown in a series of three WPF Infragistics charts.

In order to compile and run either of these examples, please follow the links from the partnership page and download trial versions of both the CenterSpace & Infragistics .NET toolsets.

Feedback

Please let us know if you have found these examples helpful and what else you’d like to see cooked up. Customers frequently ask us about visualization tools sets and we hope through this partnership to provide information and code examples to accelerate your development projects.

Happy Computing,

Paul

Resources

• CenterSpace Software’s partners.

SIGA is a world leader in designing and developing novel countermeasures to prevent and treat serious infectious diseases, with an emphasis on biological warfare defense.

In a previous blog post, Ken outlined the techniques for using NMath to compute various common linear trend lines and non-linear trend lines linearizable via a simple variable substitution.  This post extends that curve fitting article to the non-linear case.

Non-linear Curving Fitting – The Logistic

The logistic model is a fundamental non-linear model for many systems, and is widely used in the life sciences, medicine, and environmental toxicology.

Non-linear curve fit with logistic model

This image shows a fit of a 4-parameter logistic model to the measured inhibitory response of an infectious agent to a treatment at various drug dose levels – this is a classic dose-response curve.  The 4-parameter model was used here because the underlying physical process is expected to be symmetric, and is defined as:

 a - response at x = 0

 b - slope factor

 c - mid-range (50%) concentration

 d - response as x -> infinity

The 5-parameter logistic adds an additional asymmetry parameter that allow the leading and trailing tails to have different curvatures. It is defined as:

 a - response at x = 0

 b - slope factor

 c - mid-range (50%) concentration

 d - response as x -> infinity

 g - asymmetry factor

Using NMath the following C# code fits the four-parameter logistic to the given data. By extending the initial conditions vector by one element, and changing the model function delegate we could use the same code to fit the FiveParameterLogistic function.

using CenterSpace.NMath.Core;
using CenterSpace.NMath.Analysis;
 
 public void NonlinearFitExampleCode()
 {
    // Make some synthetic data
    DoubleVector xValues = new DoubleVector("30000 10000 3000 500 200 60  25 10");
    DoubleVector yValues = new DoubleVector("112.8 115 131 118 26 1.7 -0.8 -1.4");
 
    // This is the function delegate that the curve fitting 
    // class will use - any delegate can be used.
    NMathFunctions.GeneralizedDoubleUnaryFunction f 
      = AnalysisFunctions.FourParameterLogistic;
 
    // The starting point in the function parameter space.
    DoubleVector start = new DoubleVector("0.1 0.1 0.1 0.1");
 
    // Construct a curve fitting object for our function, then perform the fit.
    OneVariableFunctionFitter fitter 
      = new OneVariableFunctionFitter(f);
    DoubleVector solution = fitter.Fit(xValues, yValues, start);
 
    // Display solution
    Console.WriteLine("4 Parameter Logistic Model Parameters: {0}", solution.ToString("0.####"));
}

The model parameters are returned in a vector, in the order defined above [a, b, c, d].

4 Parameter Logistic Model Parameters:
  [ -0.1755 6.0556 246.8343 119.6108 ]

The fitting class OneVariableFunctionFitter is based on the Trust Region algorithm which is an extension of the Levenberg-Marquardt minimization algorithm. As used in the example above, the partial derivatives with respect to the parameters are numerically estimated. In our experience numerically estimating the partials works very well. However, for improved precision and performance it is also possible to explicitly specify delegates for the partials, using the property:

 NMathFunctions.GeneralizedDoubleUnaryFunction[] PartialDeriviates 

If the curvature of the function surface is low, as is typically the case for the logistic model, and the local linearized estimate closely approximates the function surface during the minimization process, the partial derivatives are not even evaluated for performance reasons.

Bounded Parameters

Because the OneVariableFunctionFitter is based on the Trust Region algorithm, it is possible to place bounds on the parameters. This is a very powerful feature, which is frequently employed when modeling a dose-response curve with a logistic. There’s an old debate between the chemists and biologists regarding the dose-response curve on whether the curve should be relative or absolute. In the image above, a relative dose-response curve is shown, as there were no bounds placed on the four parameters. However, we can create an absolute dose-response curve by adding the constraints 100 >= a >= 0 and 0 <= d <= 100 and re-fitting the curve. This will enforce all predicted responses to be between 0% and 100%.

Logistic model fit with parameter bounds 100 >= a >= 0 and 0 <= d <= 100

Here is a C# code example on how to do this with NMath.

using CenterSpace.NMath.Core;
using CenterSpace.NMath.Analysis;
 
 public void BoundedNonlinearFitExampleCode()
 {
 
  // Make some synthetic data
  DoubleVector xValues = new DoubleVector("30000 10000 3000 500 200 60  25 10");
  DoubleVector yValues = new DoubleVector("112.8 115 131 118 26 1.7 -0.8 -1.4");
 
  // The starting point in the function parameter space.
  DoubleVector start = new DoubleVector("0.1 0.1 0.1 100.0");
 
  // 4-parameter bounds in the order of a, b, c, & d
  DoubleVector lowerBounds = new DoubleVector("0.0 0.0 0.0 0.0");
  DoubleVector upperBounds = new DoubleVector("100.0 100000.0 100000.0 100.0");
 
  // Convert the 4-parameter logistic to a Func<>
  Func<Double[], Double, Double> fitFunction = 
    new Func<double[], double, double>((double[] soln, Double x) => AnalysisFunctions.FourParameterLogisticFunction(new   DoubleVector(soln), x));
 
  // This class is not included with the current release.
  FunctionFitterWithBounds fitter = 
    new FunctionFitterWithBounds(fitFunction, lowerBounds, upperBounds);
 
  //Now define and fit the data as shown above.
  DoubleVector solution = fitter.Fit(xValues, yValues, start);
 
  // Display solution
   Console.WriteLine("4 Parameter Logistic Model Parameters: {0}", solution.ToString("0.####"));
}

The bounded model parameters returned, in the order defined above [a, b, c, d] are:

4 Parameter Logistic Model Parameters: [ 0 0.1652 0.787 100 ]

Note that the first and last parameters are now within the specified bounds.

The class FunctionFitterWithBounds is unreleased (and won't be in this form) but you can download it and use it if you have the latest release of NMath installed. The non-linear curving fitting NMath API will be extended and refined in our next NMath release, and bounded non-linear curve fitting will be more clearly accessible.

• Click here to download the free C# FunctionFitterWithBounds class.

Goodness-Of-Fit Model Statistics

After we are done fitting our model, the fit may look good, but really how good is the fit? To better understand the goodness-of-fit of our model, we need to analyze the residuals of the fit and related statistics. The next release of NMath Stats (expected in June) will offer a new class GoodnessOfFit that will provide a complete set of statistical measures that can be used to assess the goodness-of-fit for a non-linear model, including the F-Statistic, t-Statistic, and parameter confidence intervals. Please contact us if you are interested in obtaining an advanced copy of this class.

Happy Computing,
Paul

Resources

• The five-parameter logistic: A characterization and comparison with the four-parameter logistic, Paul G. Gotschalk, John R. Dunn, Analytical Biochemistry 343 (2005) pp 54-65.
• Principles of Curve Fitting for Multiplex Sandwich Immunoassys, D. Davis, A. Zhang, C. Etienne, I. Huang, & M. Malit, tech note 2861.
• A brief technical introduction to Trust Region methods, by Mark S. Gockenbach
  ]]> http://www.centerspace.net/blog/nmath/non-linear-curve-fitting/feed/ 2 http://www.centerspace.net/blog/corporate/nevron-partnership/ http://www.centerspace.net/blog/corporate/nevron-partnership/#comments Wed, 31 Mar 2010 16:00:57 +0000 Paul Shirkey http://www.centerspace.net/blog/?p=1903 All of us at CenterSpace software are proud to announce a partnership with Nevron, leaders in data visualization. Nevron builds .NET components which enable developers to quickly build clean, enterprise quality, user interfaces incorporating well rendered maps, gauges, diagrams, or charts. Check out our partnership page for more information and free examples.]]> All of us at CenterSpace software are proud to announce a partnership with Nevron, leaders in data visualization. Nevron builds .NET components which enable developers to quickly build clean, enterprise quality, user interfaces incorporating well rendered maps, gauges, diagrams, or charts. Nevron has built it’s visualization framework with the developer in mind and has created components that are powerful, yet quick to pick up and work with.

  Free Project Examples

  As part of of this partnership our software engineers are working together to develop a series of example applications to help our customers quickly become productive using our tool set. Our first example demonstrates how to use the Nevron .NET chart component with some of NMath Stat’s fundamental capabilities: regression and prediction, polynomial curve fitting, and data smoothing. To build and run this example, you’ll need to download this VS8 project, along with evaluation copies of the CenterSpace NMath Suite and the Nevron .NET Chart package. All of this is freely available from our partnership page.

  As I mentioned this is the first in a series of examples that we will be jointly developing. Our next example application is in the planning stages, and will cover the building of common charts in statistical process control. Statistical process control is employed across many industries ranging from manufacturing to medicine and has been an area of interest by customers of both companies. We’ll post up a blog article about the example once it’s finished and you’ll see a tweet about it if you are following us.

  If you have an idea for an example using Nevon’s and CenterSpace’s tools together, which would help speed along your own project, leave a comment or drop us an email. We just may cook it up!

  Discount Bundle

  As part of our partnership with Nevron, we are able to offer a substantial discount on package licenses. If you are interested, please email our sales staff and request the Nevron partnership discount.

  Thanks for reading,

  -The CenterSpace Team
  
  Resources

  • CenterSpace partnership information partnership page.
  • Nevron technology partnership information page.
  • Nevron’s .NET Chart home page.
  ]]> http://www.centerspace.net/blog/corporate/nevron-partnership/feed/ 0 http://www.centerspace.net/blog/nmath/nmath-tutorial/excel-trendlines/ http://www.centerspace.net/blog/nmath/nmath-tutorial/excel-trendlines/#comments Thu, 25 Mar 2010 14:45:35 +0000 Ken Baldwin http://www.centerspace.net/blog/?p=1870 We are sometimes asked how to reproduce the various Excel Trendline types in NMath: Linear, Logarithmic, Exponential, Power, Polynomial, and Moving Average. In this post, we show you how to compute each trendline using NMath, including printing out the form of the equation and the R2 value (coefficient of determination). ]]> We are sometimes asked how to reproduce the various Excel Trendline types in NMath, including printing out the form of the equation and the R2 value (coefficient of determination). Excel offers these trend types:
  • Linear
  • Logarithmic
  • Exponential
  • Power
  • Polynomial
  • Moving Average

  These can all be easily computed using NMath. Let’s see how. First, let’s start with a simple data series:

  DoubleVector x = new DoubleVector(11, 15, 18, 23, 26, 31, 39, 44, 54, 64, 74);
  DoubleVector y = new DoubleVector(0.00476, 0.0105, 0.0207, 0.0619, 0.337, 0.74, 1.7, 2.45, 3.5, 4.5, 5.09);

  These data represent the evolution of an algal bloom in the Adriatic Sea. The x-values are time expressed in days, and the y-values are the size of the surface of the bloom in mm².

  Linear

  The Linear trendline fits a line, y = a*x + b, to the data. This is easily computed using the LinearRegression class in NMath Stats, as shown in the following C# code:

  bool addIntercept = true;
  LinearRegression lr = new LinearRegression(new DoubleMatrix(x), y, addIntercept);
  LinearRegressionAnova lrAnova = new LinearRegressionAnova(lr);
  double a = lr.Parameters[0];
  double b = lr.Parameters[1];
  double r2 = lrAnova.RSquared;
   
  Console.WriteLine("y = {0}x + {1}", a, b);
  Console.WriteLine("r2 = {0}", r2);

  Output

  y = -1.63908651994092x + 0.0913403802489977
  r2 = 0.967828167696715

  Note that to compute the coefficient of determination (R2), we construct a LinearRegressionAnova object from the LinearRegression instance. This class tests overall model significance for regressions computed by LinearRegression.

  Logarithmic

  The Logarithmic trendline fits a line to ln(x), y–that is, y = a*ln(x) + b. Again, we can use the LinearRegression class for this:

  DoubleVector logX = NMathFunctions.Log(x);
  lr = new LinearRegression(new DoubleMatrix(logX), y, addIntercept);
  a = lr.Parameters[0];
  b = lr.Parameters[1];
  r2 = new LinearRegressionAnova(lr).RSquared;
   
  Console.WriteLine("y = {0}ln(x) + {1}", a, b);
  Console.WriteLine("r2 = {0}", r2);

  Output:

  y = -8.17805531083818ln(x) + 2.87283105614145
  r2= 0.840393353070466

  Exponential

  The Exponential trendline fits the function y = a * e ^(b * x). This can also be computed using LinearRegression by fitting a line to x, ln(y). Taking the log of both sides of the function gives:

  ln(y) = ln(a * e ^(b * x)) = ln(a) + bx

  The following C# code does this:

  DoubleVector logY = NMathFunctions.Log(y);
  lr = new LinearRegression(new DoubleMatrix(x), logY, addIntercept);
  a = Math.Exp(lr.Parameters[0]);
  b = lr.Parameters[1];
  r2 = new LinearRegressionAnova(lr).RSquared;
   
  Console.WriteLine("y = {0}e^{1}x", a, b);
  Console.WriteLine("r2 = {0}", r2);

  Output:

  y = 0.00552689525130073e^0.112876522212724x
  r2 = 0.812366433832176

  Note that because the intercept of the fitted line is the natural log of the “a” parameter in the exponential function, we need to take the exponential of the found intercept (using Math.Exp) to recover the value of “a”.

  Power

  The Power trendline fits the function y = a * x^b. This can be computed using LinearRegression by fitting a line to ln(x), ln(y). Taking the log of both sides of the equation gives:

  ln(y) = ln(a * x^b) = ln(a) + b * ln(x)

  lr = new LinearRegression(new DoubleMatrix(logX), logY, addIntercept);
  lrAnova = new LinearRegressionAnova(lr);
  a = Math.Exp(lr.Parameters[0]);
  b = lr.Parameters[1];
  r2 = new LinearRegressionAnova(lr).RSquared;
   
  Console.WriteLine("y = {0}x^{1}", a, b);
  Console.WriteLine("r2 = {0}", r2);

  Output:

  y = 2.46993343563889E-07x^4.11443630332377
  r2 = 0.947447653331871

  Again, we recover the value of parameter “a” by taking the exponential of the found intercept.

  Polynomial

  NMath provides class PolynomialLeastSquares for fitting a polynomial of the specified degree to paired vectors of x- and y-values. For example, this code fits a cubic to our data:

  int degree = 3;
  PolynomialLeastSquares pls = new PolynomialLeastSquares(degree, x, y);
   
  // compute R2
  DoubleVector predictions = pls.FittedPolynomial.Evaluate(x);
  double regressionSumOfSquares = StatsFunctions.SumOfSquares(predictions - StatsFunctions.Mean(y));
  double residualSumOfSquares = pls.LeastSquaresSolution.Residuals.TwoNormSquared();
  r2 = regressionSumOfSquares / (regressionSumOfSquares + residualSumOfSquares);
   
  Console.WriteLine("y = " + pls.FittedPolynomial);
  Console.WriteLine("r2 = {0}", r2);

  Output:

  y = -4.68278158094222E-05x^3 + 0.00640408381023593x^2 -
        0.1643720340709x + 1.12703300920657
  r2 = 0.998634459376868

  Note that PolynomialLeastSquares does not currently provide the R2 value, so in the code above we compute it directly.

  Moving Average

  Unlike the trendlines we’ve examined so far, a moving average does not fit a functional form to data. Rather, it filters data in order to smooth out noise. NMath provides class MovingWindowFilter for this purpose.

  Class MovingWindowFilter replaces data points with a linear combination of the data points immediately to the left and right of each point, based on a given set of coefficients to use in the linear combination. Static class methods are provided for generating coefficients to implement a moving average filter and a Savitzky-Golay smoothing filter.

  For example, the following C# code filters the data using a window of width 3 (the “period” parameter in Excel):

  int numberLeft = 1;
  int numberRight = 1;
  DoubleVector movingAvgCoefficents = MovingWindowFilter.MovingAverageCoefficients(numberLeft, numberRight);
  MovingWindowFilter filter = new MovingWindowFilter(numberLeft, numberRight, movingAvgCoefficents);
  DoubleVector yfiltered = filter.Filter(y, MovingWindowFilter.BoundaryOption.DoNotFilterBoundaryPoints);
  Console.WriteLine("yfiltered = " + yfiltered);

  Output:

  yfiltered = [ 0.00476 0.0119866666666667 0.0310333333333333
                0.139866666666667 0.379633333333333 0.925666666666667 1.63
                2.55 3.48333333333333 4.36333333333333 5.09 ]

  Advanced Curve Fitting

  That covers the simple trendlines produced by Excel. For advanced curve fitting, NMath provides class OneVariableFunctionFitter which fits arbitrary one variable functions to a set of points. (You must supply at least as many data points to fit as your function has parameters.) In a future post, we’ll take a look at this class. In the meantime, see this page for an example demonstrating the use of OneVariableFunctionFitter, including how to define your own function.

  Ken

  ]]> http://www.centerspace.net/blog/nmath/nmath-tutorial/excel-trendlines/feed/ 1 http://www.centerspace.net/blog/nmath/using-nmath-in-net-4-0-applications/ http://www.centerspace.net/blog/nmath/using-nmath-in-net-4-0-applications/#comments Wed, 17 Mar 2010 15:53:34 +0000 Ken Baldwin http://www.centerspace.net/blog/?p=1859 Version 4.0 of the .NET Framework introduced a new runtime activation policy, which enables an application to activate multiple versions of the common language runtime (CLR) in the same process. (See here for more information.)

  Because of this support for side-by-side runtimes, .NET 4.0 has changed the way that it binds to older mixed-mode assemblies, such as the NMath Kernel assembly, which is written in C++\CLI and uses the Intel Math Kernel Library internally.

  When using NMath in .NET 4.0 applications, the NMath Kernel fails to load, because it was built against an earlier version of .NET and includes unmanaged code. The error is:

  System.TypeInitializationException : The type initializer for
  'CenterSpace.NMath.Core.NMathKernel' threw an exception.
  ----> CenterSpace.NMath.Core.KernelLoadException : Could
  not load kernel assembly NMathKernelx86

  There are two solutions:

  1. Activate the v2.0 runtime activation policy by adding the following lines to your app.config when using runtime v4.0:

     <configuration>
       <startup useLegacyV2RuntimeActivationPolicy="true">
         <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0"/>
       </startup>
     </configuration>

     See here for more information on the <startup> element.

  2. Request assemblies for .NET 3.5  from Centerspace Technical Support. (see below for correction)

  In the next release of NMath, the shipping assemblies will be built against .NET 3.5. (Assemblies for .NET 2.0 will remain available upon request.) << see below for correction

  Ken

  Update

  This issue can only be solved with the application config change or with assemblies built with .NET 4.0.

  If you are widely deploying your application, you should probably target .NET 3.5 or earlier. In this case, the application config is the best solution.

  If you know your application will be run within .NET 4.0 or later and don’t wish to make the application config change, please contact us at support@centerspace.net. We can send you NMath assemblies built and tested with .NET 4.0.

  Since we don’t use any .NET 3.5 features and it’s too early to target .NET 4.0 with a class library, we will be targeting .NET 2.0 in the next release (expected in June, 2010).

  

  ]]> http://www.centerspace.net/blog/nmath/using-nmath-in-net-4-0-applications/feed/ 6 http://www.centerspace.net/blog/statistics/priniciple-components-regression-in-csharp/ http://www.centerspace.net/blog/statistics/priniciple-components-regression-in-csharp/#comments Thu, 04 Mar 2010 17:17:07 +0000 Steve Sneller http://centerspace.net/blog/?p=1816 explanatory variables and a response variable by fitting a linear equation to observed data. This is the second part in a three part series on PCR. ]]> This is the second part in a three part series on PCR, the first article on the topic can be found here.

  The Linear Regression Model

  Multiple Linear Regression (MLR) is a common approach to modeling the relationship between one or two or more explanatory variables and a response variable by fitting a linear equation to observed data. First let’s set up some notation. I will be rather brief, assuming the audience is somewhat familiar with MLR.

  In multiple linear regression it is assumed that a response variable, depends on k explanatory variables, , by way of a linear relationship:

  

  The idea is to perform several observations of the response and explanatory variables and then to chose the linear coefficients which best fit the observed data.

  Thus, a multiple linear regression model is:

  
  
  
  
  
  
  
  
  
  

  In matrix notation we have

  

  where

  .

  The solution for the coefficient vector which “best” fits the data is given by the so called “normal equations”

  

  This is known as the least squares solution to the problem because it minimizes the sum of the squares of the errors.

  Now, consider the following example in which
  

  and

  

  Solving this simple linear regression model using the normal equations yields

  

  which is quite far off from the actual solution

  

  The reason behind this is the fact that the matrix is ill conditioned. Since the second column of is approximately twice the first, the matrix is almost singular.

  One solution to this problem would be to change the model. Since the second column is approximately twice the first, these two explanatory variables encode basically the same information, thus we could remove one of them from the model.
  However, it is usually not so easy to identify the source of the bad conditioning as it is in this example.

  Another method for removing information from a model that is responsible for impreciseness in the least squares solution is offered by the technique of principal component regression (PCR). Henceforth we shall assume that the data in the matrix is centered. By this we mean that the mean of each explanatory variable has been subtracted from each column of X so that the explanatory variables all have mean zero. In particular this implies that the matrix is proportional to the covariance matrix for the explanatory variables.

  Removing the Source of Imprecision

  Let be an mxn matrix, and recall from the part 1 of this series that we can write as

  

  where is a diagonal matrix containing the eigenvalues (in ascending order down the diagonal) of , and is orthogonal. The condition number for is just the absolute value of the ratio of the largest and smallest eigenvalues:

  

  Thus we can see that if the smallest eigenvalue is much smaller than the largest eigenvalue, we get a very large condition number which implies a poorly conditioned matrix. The idea then is to remove these small eigenvalues from thus giving us an approximation to that is better conditioned. To this end, suppose that we wish to retain the r (r less than or equal to n) largest eigenvalues of in our approximation, and thus write

  ,

  where

  is an r x r diagonal matrix consisting of the r largest eigenvalues of , is a (n-r) x (n-r) diagonal matrix consisting of the remaining n – r eigenvalues of , and the n x n matrix is orthogonal with consisting of the first r columns of , and consisting of the remaining n – r columns of . Using this formulation we can write an approximation to using the r largest eigenvalues as

  .

  If we substitute this approximation into the normal equations 2, and do some simplification, we end up with the principal components estimator

  .

  While we could use equation 3 directly, it is usually not the best way to perform principal components regression. The next article in this series will illustrate an algorithm for PCR and implement it using the NMath libraries.

  -Steve

  ]]> http://www.centerspace.net/blog/statistics/priniciple-components-regression-in-csharp/feed/ 0