I have a program that creates a gradient image. If I compile this for my GPU and look at the output of the compile_to_lowered_stmnt I see it starts with (after the produce statement) a halide_copy_to_host and then starts the outer loop. If I nest functions, the halide_copy_to_host is at the same position, but then for the outermost func. Note I do nothing with scheduling. I'd like to understand why it is at that location. I expect it to be at the end of the program, to copy the result back to the host, not just before the calculations have finished. And if I want the result to stay on the GPU (e.g. output to screen) the algorithm should run faster without the copy. Is there a way to "remove" the halide_copy_to_host?
1 Answer
The copy_to_host is probably there in case the output buffer is dirty on the GPU. If it is, and we didn't copy to host, then you'd have a buffer dirty on CPU and GPU, which is impossible to reconcile without tracking which parts of the buffer are dirty on each GPU vs CPU.
However, copy_to_host is a no-op if the dev_dirty flag is set to false, so it shouldn't actually be doing anything in your case. I think it's likely that no copies are happening. If you enable the -debug target flag you can check for yourself.
Halide doesn't do a copy_to_host after GPU compute in case you want the result to stay on the GPU, for the reasons you say.
-
ok, I understand. Later in the code I see a set_dev_dirty call. How do I know which part of my code generates that statement and/or how do I prevent this? Oct 26, 2016 at 11:14
-
I did see someone suggesting setting the host pointer to NULL, does that mean this only works for AOT compiled Halide? Oct 27, 2016 at 7:02
-
Got it working. You need to use a buffer and pass that buffer to the realize func. Then you can get it to the host using copy_to_host if you want to. Looking at the lowered code however still shows the set_dev_dirty and the copy_to_device is also in the code, but it seems it is not called anymore. Oct 27, 2016 at 8:30