0

I have some process that iterates over a group of large files in order and extracts information from them to write\update our databases. These files regularly have several thousand lines in each so I have built a Parallel.ForEach to process multiple lines concurrently in a given file(Only processing one file at a time due to needing to apply each file in order). Now I have a need to know approximately how much of the current file has been handled so that I can give management an indication of remaining runtime. So far I have the following

public void MyProcess(FileItem file)
{
    List<string> lines = file.GetLines(); //some process to get the lines to handle
    long cntr = 0; //The counter to track

    Parallel.ForEach(lines, crntLine =>
    {
       Console.Writeline(String.Format("Currently finished {0} out of {1} lines",cntr,lines.Count());
       InterLocked.Increment(ref cntr);

       //...Code to process crntLine here

    });
}

I don't care at all about which lines have been processed, only how many total so that can answer the question of how far along it is in the current file. Is this going to give me what I'm looking for reliably?

1 Answer 1

5

Interlocked.Increment will safely increment your counter as an atomic operation.

However, if you want a count of lines completed, you need to increment it after the line processing, not before:

public void MyProcess(FileItem file)
{
    List<string> lines = file.GetLines(); //some process to get the lines to handle
    long cntr = 0; //The counter to track

    Parallel.ForEach(lines, crntLine =>
    {
       //...Code to process crntLine here

       InterLocked.Increment(ref cntr);
       Console.Writeline(String.Format("Currently finished {0} out of {1} lines",cntr,lines.Count());
    });
}

If you want the percentage, divide the count completed by the total count, then multiply by 100:

double percentage = (double)cntr / lines.Count * 100
9
  • It seems I can't accept the answer for 5 more minutes, but this gave me what I need. Thank you!
    – Rocky
    Jan 19, 2017 at 14:46
  • I'm hitting a snag. I do need the percentage, but that answer appears to always be "0.00%" when I do it the way you have suggested. Here's the actual line of code i'm using double percentDone = (lineCount / results.Count()) * 100; What's going on?
    – Rocky
    Jan 19, 2017 at 15:04
  • @Rocky It's cntr * 100 / lines.Count
    – Rabban
    Jan 19, 2017 at 15:09
  • 1
    @Rocky Good point, I'll add it to my answer. You should only need a cast on one of the numbers
    – horns
    Jan 19, 2017 at 15:46
  • 1
    @Rabban order of operations matter when doing integer division, doing cntr * 100 / lines.Count will give a diffrent result than cntr / lines.Count * 100 because when you divide a int by a int it throws away the remainder, so you want that throw away to happen after the * 100 instead of before. However you still would not get any decimal points, so casting to a double is the better option if you want the decimal portion. Jan 19, 2017 at 15:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.