0

I am currently working on a system in which I reading in a file of over ~200 million records (lines), so thought I could user a producer-consumer model to improve performance (working as I read). However, I am not achieving strong performance and am concerned my general design is wrong. To put it into context :

int i = 0;
string buffer[MAX_SIZE];

//critical regions exist for map_a and map_b (shared below) in the task function 

#pragma omp parallel shared(map_a), shared(map_b), num_threads(X) 
#pragma omp single
{
    while (getline(fin, line) && !fin.eof())
    {
        buffer[i] = line;
        if (++i == MAX_SIZE)
        {
#pragma omp task firstprivate(buffer)
            work_on_data(buffer, map_a, map_b);
            i = 0;
        }
    }
}

Each record in the buffer takes roughly 49-95μ to process in work_on_data, with variance due to conditionals and I suspect the pragma omp critical regions (one for each shared map). For the two critical regions :

  1. For map_a : if a certain case holds according to the record, an entry will need to be added to the map with the key derived from the record. If an entry already exists, it will need to be updated. There is a critical region over the map read, potential update, and write.
  2. For map_b : for each record, the map has to be updated. The critical region covers the same actions as (1), i.e. a read, potential update/insertion, and write.

So, regarding my approach. Should I be using a separate pthread to buffer IO? Should I simply be buffering into one huge memory-allocated buffer and creating tasks that pragma omp parallel for over a subset of it's records? I am not experienced with this kind of programming.

Thanks in advance!


Edit : Clarified use of critical region.

2
  • Can you give more details on the maps and how they are used ? You may be able to remove critical regions.
    – ElderBug
    Jan 6, 2015 at 9:56
  • @ElderBug I have clarified slightly above, and I can add more code later (pushed for time at the moment). Unfortunately I don't believe the critical regions can be removed, there is no uniqueness of records in each task that prevents reading/updating/writing to the same key entries in both maps. I tried using atomic write, however am stuck with an old version of OpenMP.
    – PidgeyBAWK
    Jan 6, 2015 at 10:03

1 Answer 1

2

About the IO, I don't think you can gain much performance, as it should already be decently buffered by the OS. You can always try to implement large buffering yourself (potentially with producer/consumer), or use memory mapped file, but I'm afraid you will be disappointed by the performance gain (and getline is so much simpler).

About the file analysis, you should of course try to optimize the computing itself, but there is potentially much better possible gain if you can remove the critical regions. Usually, the goal is to completely remove dependence on shared objects. How you do that depends on your application, but the general idea is to have independent processing in each thread, and then merge the results together. In your case, you could allocate independent maps in each thread, and then update the real maps afterwards. If you need the original maps for processing, read them but don't update/write them, write independent objects and update later. This way you can remove the critical regions (read operations are thread-safe).

As a side note, this is very application specific, and also hardware specific. If your processing time is short compared to file read (which can greatly depends on your CPU/HDD/SSD), you might gain much performance with better IO buffering, and it might even make multi-threading useless. Also, if the result merging is too heavy, splitting the results might not be worth it. How you split/merge the results is important ; you can just build a list of updates to do, or build an actual map that you will merge. It is also possible that the critical regions weren't problematic. Try to experiment to see what is better for you.

3
  • You were certainly right to look into simply removing critical regions. The first one was a major hog, and I got considerable speed up. Now I have the challenge of an efficient data merge... I'll get back to you with the results!
    – PidgeyBAWK
    Jan 6, 2015 at 14:58
  • I took your advice and have been working on merging the data. Unfortunately I am having some trouble extracting the data from each pragma omp task, as explained on this question I asked stackoverflow.com/questions/27849876/…
    – PidgeyBAWK
    Jan 9, 2015 at 0:30
  • @PidgeyBAWK I commented on the other question
    – ElderBug
    Jan 9, 2015 at 8:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.