Large matrices and vectors
Wednesday, September 16th, 2009Customers frequently ask us how large their matrices can be.
NMath matrices (and vectors) are stored internally as an array of floats or doubles.
32-bit .NET processes are limited to 1.2GB of memory. Theoretically a matrix could take up most of that space. Let’s say it’s feasible to have a 1 GB matrix. That matrix would contain 134,217,728 doubles or 268,435,456 floats—for example, a 11,585 x 11,585 square DoubleMatrix or a 16,384 x 16,384 square FloatMatrix.
There’s a workaround in 32-bit .NET to increase the memory to 2.4GB…
1. Add the /3GB switch to boot.ini.
2. After building the application, run the
linker as follows:
link -edit -LARGEADDRESSAWARE myapp.exe
where myapp.exe is your application.
Increasingly, our customers are switching to 64-bit computing in part to get around these memory limitations. Although a 64-bit .NET process’s memory is only limited by the available RAM, the .NET runtime limits any one object to 2GB. For that reason, our matrices are limited to a theoretical maximum of 402,653,184 doubles or 805,306,368 floats —for example, a 20,066 x 20,066 square DoubleMatrix or a 28,377 x 28,377 square FloatMatrix.
- Trevor
Update
We have spoken with Microsoft. They are well aware of this issue and are working on a solution. That solution could involve a blob object that could be created of arbitrary size. We have no word on when this will be available.
