I have some process that iterates over a group of large files in order and extracts information from them to write\update our databases. These files regularly have several thousand lines in each so I have built a Parallel.ForEach to process multiple lines concurrently in a given file(Only processing one file at a time due to needing to apply each file in order). Now I have a need to know approximately how much of the current file has been handled so that I can give management an indication of remaining runtime. So far I have the following
public void MyProcess(FileItem file)
{
List<string> lines = file.GetLines(); //some process to get the lines to handle
long cntr = 0; //The counter to track
Parallel.ForEach(lines, crntLine =>
{
Console.Writeline(String.Format("Currently finished {0} out of {1} lines",cntr,lines.Count());
InterLocked.Increment(ref cntr);
//...Code to process crntLine here
});
}
I don't care at all about which lines have been processed, only how many total so that can answer the question of how far along it is in the current file. Is this going to give me what I'm looking for reliably?