1

I need to write an application that hashes words from a dictionary to make WPA pre-shared-keys. This is my thesis for a "Networking Security" course. The application needs to be parallel for increased performance. I have some experience with MPI from my IT studies but I would like to tie it up with CUDA. The idea is to use MPI to distribute the load evenly to the nodes of the cluster and then utilize CUDA to run the individual chunks in parallel inside the GPUs of the nodes.

Distributing the load with MPI is something I can easily do and have done in the past. Also computing with CUDA is something I can learn. There is also a project (pyrit) that does more or less what I need to do (actually a lot more) and I can get ideas from there.

I would like some advice on how to make the connection between MPI and CUDA. If there is somebody that has built anything like this I would greatly appreciate his advice and suggestions. Also if you happen to know of any resources on the topic please do point them to me.

Sorry for the lengthy intro but I thought it was necessary to give some background.

3
  • You might be interested in mvapich.cse.ohio-state.edu which integrates GPU computing into the MPI model. There are other similar efforts under way, not sure how far advanced any of them are. Jul 9, 2012 at 9:03
  • Most current MPI implementations support transparent operations on GPU memory - you just directly supply a device pointer to MPI_* and the library does the rest. It might also save you some memory copy time if you run on InfiniBand or other RDMA interconnect as RDMA operations could in principle be done on device memory. You shuold also be aware of some pecularities when working with pinned memory. Jul 9, 2012 at 12:03
  • 2
    To first order, there isn't a connection between MPI and CUDA; they're orthogonal. You do communications between nodes w/ MPI, and do computation on nodes w/ CUDA. To second order, as @HristoIliev and HP Mark point out, if you have new enough network cards, new enough CUDA devices, and new enough MPI libraries, you can blur the boundaries a bit by having MPI directly send/recv the data to/from the GPU memory rather than going through the CPU and the host memory, but that's just a latency optimization which you implement after you have everything else working. Jul 9, 2012 at 12:17

1 Answer 1

3

This question is largerly open-ended and so it's hard to give a definitive answer. This one is just a summary of the comments made High Performance Mark, me and Jonathan Dursi. I do not claim authorship and thus made this answer a community wiki.

MPI and CUDA are orthogonal. The former is an IPC middleware and is used to communicate between processes (possibly residing on separate nodes) while the latter provides highly data-parallel shared-memory computing to each process that uses it. You can break the task into many small subtasks and use MPI to distribute them to worker processes running on the network. The master/worker approach is suitable for this kind of application, especially if words in the dictionary vary greatly in their length and variance in processing time is to be expected. Provided with all the necessary input values, worker processes can then use CUDA to perform the necessary computations in parallel and then return results back using MPI. MPI also provides the mechanisms necessary to launch and control multinode jobs.

Although MPI and CUDA could be used separately, modern MPI implementations provide some mechanisms that blur the boundaries between those two. It could be either direct support for device pointers in MPI communication operations that transparently call CUDA functions to copy memory when necessary or it could be even support for RDMA to/from device memory without intermediate copy to main memory. The former simplifies your code while the latter can save different amount of time, depending on how your algorithm is structured. The latter also requires both failry new CUDA hardware and drivers and newer networking equipment (e.g. newer InfiniBand HCA).

MPI libraries that support direct GPU memory operations include MVAPICH2 and the trunk SVN version of Open MPI.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.