NMath User's Guide

TOC | Previous | Next | Index

39.4 Bridge Management (.NET, C#, CSharp, VB, Visual Basic, F#)

In NMath Premium, a Bridge is an abstraction for controlling the routing of computations between the CPU and a GPU:

● Compute Devices

GPU devices are represented in the NMath Premium API by either an inte­ger CUDA device number or a IComputeDevice instance.

● Thread-Level Control

Executing threads are assigned to devices. This is a many-to-one relation­ship—any number of threads can be routed to a particular compute device.1

● Routing

Bridge instances control whether a particular operation is sent to the CPU or a GPU. Bridges are assigned to devices and there is a strict one-to-one relationship between each Bridge and IComputeDevice. Once assigned, the bridge instance governs when computations will be sent to its paired device.

The singleton BridgeManager provides the primary means for managing devices and bridges. The relationship between the BridgeManager, bridges, and compute devices is shown in Figure 7.

Figure 7 – Adaptive Bridge

At startup, the default behavior creates a default Bridge and assigns it to the GPU with CUDA device number of 0 (generally the fastest GPU installed). Also by default, all unassigned threads execute on device 0.

Getting the BridgeManager Singleton

The static Instance property on BridgeManager returns the singleton instance.

Code Example – C# GPU

BridgeManager bmgr = BridgeManager.Instance;

Code Example – VB GPU

Dim BMgr As BridgeManager = BridgeManager.Instance

Working With Devices

Each CUDA-capable GPU is assigned an integer device ID. Note that PCI slot numbers do not necessarily correspond to GPU device numbers. NVIDIA assigns device number 0 to the fastest detected GPU, and installing an additional GPU into a machine may re-number the device numbers for the previously installed GPUs.

GetComputeDevice() returns the IComputeDevice encapsulating a specified GPU.

Code Example – C# GPU

IComputeDevice dev0 = BridgeManager.Instance.GetComputeDevice( 0 );

Code Example – VB GPU

Dim Dev0 As IComputeDevice = 
  BridgeManager.Instance.GetComputeDevice( 0 )

The GPUCount property on BridgeManager returns the number of installed and properly configured GPUs.

The CPU is assigned device ID -1. You can also use the provided CPU property to access the CPU device.

Code Example – C# GPU

IComputeDevice cpu = BridgeManager.Instance.CPU;

Code Example – VB GPU

Dim Cpu As IComputeDevice = BridgeManager.Instance.CPU

To find all available computing devices, the BridgeManager provides a Devices property which returns an array of IComputeDevices containing all detected compute devices, including the CPU.

Method SetDefaultComputeDevice() sets the device to which all unassigned threads will run.

Assigning Bridges

Assigning a Bridge instance to a device is one line of code with the BridgeManager.

Code Example – C# GPU

IComputeDevice dev0 = BridgeManager.Instance.GetComputeDevice( 0 );
BridgeManager.Instance.SetBridge( dev0, bridge );

Code Example – VB GPU Logging

Dim Dev0 As IComputeDevice =
  BridgeManager.Instance.GetComputeDevice( 0 )
BridgeManager.Instance.SetBridge( Dev0, Bridge )

Assigning Threads

Once a bridge is paired to a device, threads may be assigned to that device for execution. Assigning a thread to a device is also accomplished using the BridgeManager.

Code Example – C# GPU

BridgeManager.Instance.SetComputeDevice( dev0,
  Thread.CurrentThread );

Code Example – VB GPU

BridgeManager.Instance.SetComputeDevice( Dev0,
  Thread.CurrentThread )

By default, all unassigned threads run on the default device (typically device 0).

Given three tasks and three GPUs, this code demonstrates how to use one GPU per task.

Code Example – C# GPU thread-level control

var gpu0 = BridgeManager.Instance.GetComputeDevice( 0 );
var gpu1 = BridgeManager.Instance.GetComputeDevice( 1 );
var gpu2 = BridgeManager.Instance.GetComputeDevice( 2 );

if( gpu0 != null && gpu1 != null && gpu2 != null)
{
   Task[] tasks = new Task[3]
   {
      Task.Factory.StartNew(() => Task1Worker( gpu0 )),
      Task.Factory.StartNew(() => Task2Worker( gpu1) ),
      Task.Factory.StartNew(() => Task2Worker( gpu2) ),
   };

   //Block until all tasks complete.
   Task.WaitAll( tasks );
}

Code Example – VB GPU thread-level control

Dim Gpu0 = BridgeManager.Instance.GetComputeDevice( 0 )
Dim Gpu1 = BridgeManager.Instance.GetComputeDevice( 1 )
Dim Gpu2 = BridgeManager.Instance.GetComputeDevice( 2 )

If (Gpu0 <> Nothing & Gpu1 <> Nothing & Gpu2 <> Nothing) Then

   Dim Tasks(3) As Task
   Tasks(0) = Task.Factory.StartNew(Function() Task1Worker( Gpu0 ))
   Tasks(1) = Task.Factory.StartNew(Function() Task1Worker( Gpu1 ))
   Tasks(2) = Task.Factory.StartNew(Function() Task1Worker( Gpu2 ))

   ' Block until all tasks complete.
   Task.WaitAll( Tasks )

End If

Overloads of GetComputeDevice() provide the computing device for a given thread. If no device is assigned to the given thread, the default device is returned.



  1. Threads are identified using a combination of the thread ID and thread name, because thread IDs can be reused after a thread exits. If the thread name is null, one is assigned. This property is write-once, so if you want to set thread names, do so before using the bridge.

Top

Top