3

When I try to debug an arbitrary CUDA application, e.g. the matrix multiplication or convolutionSeparable sample from the Nvidia GPU Computing SDK 4.0, I always get an output similar to:


Parallel Nsight Debug
CUDA grid launch failed: CUcontext: 2059192 CUmodule: 348912936 Function: _Z9matrixMulILi32EEvPfS0_S0_ii
……
……

And a file with the following content is showing up:


                   Parallel Nsight CUDA Debugger

The application being debugged with the Nexus CUDA debugger, was unable to find any associated source. This could be for a number of reasons:

1) CUDA has not been initialized. Make sure cuInit has been called, and it returned a successful result.

2) No CUDA contexts have been created. Once a context is created, memory can be examined in the context. Each context shows up as a single "Thread" in the Visual Studio Threads view. (Debug | Windows | Threads)

3) There are no active CUDA grids in any context. A grid must be launched in order to hit breakpoints.

4) You have selected the "Default Context" in the Visual Studio Threads view. This context is a placeholder shown when there are no available actual CUDA contexts. It does not show real data.

5) No CUDA modules have been loaded. You can see which modules are loaded in each CUDA context by showing the Visual Studio Modules view. (Debug | Windows | Modules)

6) Symbolics were not found for the loaded .cubin. The module needs to be built with debug information. Please specify the -G0 switch when building.

7) A grid launch failed while running a kernel.


Each breakpoint within the corresponding “.cu” file is completely ignored during the run. When I just run the application, without Nsight Debugging, the program executes without any problems.

What can I do to tackle this problem?

My Setup:

  • 1xIntel GPU and 1x NV 570GTX, I want to use the local debugging option
  • Win 7. Pro 64Bit
  • Dev Env.: VS2008 or VS2010
  • CUDA 4.0 & Parallel Nsight 2.0
  • NV Driver Vers.: 285.38
  • WPF is disabled
  • TDR is disabled
  • Windows runs in Basic mode (no aero)
  • Project Propertys: Cuda Runtime API -> GPU-> Generate GPU Debug Information -> Yes (-G0)

4 Answers 4

2

Firstly, you need to ensure that your display is driven by the Intel integrated graphics and not the NVIDIA GPU. This is because when you hit a breakpoint in CUDA code you are stalling the entire GPU, so if the same GPU was used for display then your system would lock up naturally.

Note that the hardware requirements for Parallel Nsight indicate you need two supported GPUs whereas you only have one, but if I understand correctly it's possible to use a non-Intel GPU for display (I haven't tried).

Assuming the above is working you should start by trying out the samples included with Parallel Nsight. You can find them in the Parallel Nsight menu group in the start menu.

3
  • My display(s) are currently driven by the intel device, thats for sure becaue otherwise this would cause some other NSight error messages. I know because I tried this already. As far as I understand, one additional arbitrary graphic device is required, which in my opinion sounds resonable.
    – Impulse
    Oct 17, 2011 at 12:18
  • @Impulse Have you tried the Parallel Nsight samples (as opposed to the CUDA SDK samples)?
    – Tom
    Oct 17, 2011 at 20:42
  • Yes, I tried SDK samples and the matrixMul sample delivered with Parallel Nsight. Output for this particular example: Parallel Nsight Debug CUDA grid launch failed: CUcontext: 46427088 CUmodule: 139693888 Function: _Z9matrixMulPfS_S_ii
    – Impulse
    Oct 18, 2011 at 8:41
2

CUDA Grid Launch has a wide variety of causes. This one is probably accessing an array beyond its allocated size. what in the x86 world is called a segmentation fault. i debug these by selectively commenting out parts of the kernel you are testing until the error goes away. (what we used to call wolf fence debugging). Another cause of grid launch failure is if the kernel is taking too long (1 or 2 seconds) to execute.

the reason the debugger isnt helping is that the debugger ONLY stops 1 thread in 1 block! your access error is coming before then. also you cant use the printf to find the bug as the output does not get returned in the event of a grid launch failure.

1
  • 1
    But the strange thing is that the kernel is running normaly without debugging option.
    – Impulse
    Nov 2, 2011 at 12:47
1

To add potential solution on top of the answers given already, one way to avoid the error is to run the NSight monitor with administrator right.

1
  • This did it for me after nothing has changed and debugging failed, while without debugging the kernel was executed without problems. Oct 10, 2017 at 7:48
0

The answer for this is definitely using the correct driver for the installation of Parallel NSight. For the latest version (2.1 RC2, currently), this is driver version 285.86. For the current stable version 2.0, this is driver version 270.81, as another poster mentioned.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.