1

I have a list of files: List<string> Files in my C#-based WPF application.

Files contains ~1,000,000 unique file paths.

I ran a profiler on my application. When I try to do parallel operations, it's REALLY laggy because it's IO bound. It even lags my UI threads, despite not having dispatchers going to them (note the two lines I've marked down):

Files.AsParallel().ForAll(x =>
{
    char[] buffer = new char[0x100000];
    using (FileStream stream = new FileStream(x, FileMode.Open, FileAccess.Read)) // EXTREMELY SLOW
    using (StreamReader reader = new StreamReader(stream, true))
    {
        while (true)
        {
            int bytesRead = reader.Read(buffer, 0, buffer.Length); // EXTREMELY SLOW
            if (bytesRead <= 0)
            {
                break;
            }
        }
    }
}

These two lines of code take up ~70% of my entire profile test runs. I want to achieve maximum parallelization for IO, while keeping performance such that it doesn't cripple my app's UI entirely. There is nothing else affecting my performance. Proof: Using Files.ForEach doesn't cripple my UI, and WithDegreeOfParallelism helps out too (but, I am writing an application that is supposed to be used on any PC, so I cannot assume a specific degree of parallelism for this computation); also, the PC I am on has a solid-state hard disk. I have searched on StackOverflow, and have found links that talk about using asynchronous IO read methods. I'm not sure how they apply in this case, though. Perhaps someone can shed some light? Also; how can you tune down the constructor time of a new FileStream; is that even possible?

Edit: Well, here's something strange that I've noticed...the UI doesn't get crushed so bad when I swap Read for ReadAsync while still using AsParallel. Simply awaiting the task created by ReadAsync to finish causes my UI thread to maintain some degree of usability. I think this does some sort of asynchronous scheduling that is done in this method to maintain optimal disk usage while not crushing existing threads. And on that note, is there ever a chance that the operating system contends for existing threads to do IO, such as my application's UI thread? I seriously don't understand why its slowing my UI thread. Is the OS scheduling work from IO on my thread or something? Did they do something to the CLR to eat threads that haven't been explicitly affinated using Thread.BeginThreadAffinity or something? Memory is not an issue; I am looking at Task Manager and there is plenty.

16
  • Define "extremely" slow. You know that disk reads are about 100,000x slower than reads from RAM right?
    – aquinas
    Nov 8, 2014 at 7:32
  • Are you only checking if the files exist ? If you are I would write my own search. Put all fileNames in a list. Then start at the base directory and do a recursive search through all directories. When a file is found remove it from the list. You can then return a list of unfound files with the files that have not been removed from the list. If you are not trying this you should explain more on what you are trying to accomplish. Nov 8, 2014 at 7:33
  • @deathismyfriend No; I am reading contents. I have a post on the "existential" problem: stackoverflow.com/questions/26321366/… That's easy, though, and very fast, but no, that's not all I want/need here.
    – Alexandru
    Nov 8, 2014 at 7:39
  • @aquinas Extremely slow: These two lines of code take up ~70% of my entire profile test runs. Cripples my UI. Yes man, everybody knows that disk reads are slower than RAM reads. I find that a little bit insulting LOL
    – Alexandru
    Nov 8, 2014 at 7:41
  • 1
    Not constraining parallelism for IO-bound tasks is a big mistake. You'll end up with 100 threads all competing for IO and very low CPU utilisation. And then thread pool will spawn more threads. Now you have 101 threads all competing for IO and very low CPU utilisation. Open Windows' Resource Monitor and watch your disk queue. Your UI is probably lagging because accessing the page file is now an exercise in contending with your million threads. Nov 8, 2014 at 8:10

4 Answers 4

1

I don't agree with your assertion that you can't use WithDegreeOfParallelism because it will be used on any PC. You can base it on number of CPU. By not using WithDegreeOfParallelism you are going to get crushed on some PCs.

You optimized for a solid state disc where heads don't have to move. I don't think this unrestricted parallel design will hold up on regular disc (any PC).

I would try a BlockingCollection with 3 queues : FileStream, StreamReader, and ObservableCollection. Limit the FileStream to like 4 - it just has to stay ahead of StreamReader. And no parallelism.

A single head is a single head. It cannot read from 5 or 5000 files faster than it can read from 1. On solid state the is no penalty switching from file to file - on a regular disc there is a significant penalty. If your files are fragmented there is a significant penalty (on regular disc).

You don't show what the data write but the next step would be to put the write in a another queue with a BlockingCollection in the BlockingCollection. E.G. sb.Append(text); in a separate queue. But that may be more overhead than it is worth. Keep that head as close to 100% busy on a single contiguous file is the best you are going to do.

private async Task<string> ReadTextAsync(string filePath)
{
    using (FileStream sourceStream = new FileStream(filePath,
        FileMode.Open, FileAccess.Read, FileShare.Read,
        bufferSize: 4096, useAsync: true))
    {
        StringBuilder sb = new StringBuilder();

        byte[] buffer = new byte[0x1000];
        int numRead;
        while ((numRead = await sourceStream.ReadAsync(buffer, 0, buffer.Length)) != 0)
        {
            string text = Encoding.Unicode.GetString(buffer, 0, numRead);
            sb.Append(text);
        }

        return sb.ToString();
    }
}
3
  • Very good point. ReadAsync performance remains to be tested on non-SSD drives in my application. For everyone else: He is saying that, basically, on RPM drives, the penalty for context switching scheduled threads with ReadAsync would be detrimental because these drives require mechanical head movements to read sectors. A note on this though, how detrimental should depend on the number of cores you have on that PC because of the functionality of workers within AsParallel, so it may not end up being as bad as we might think since the degree of parallelism is limited.
    – Alexandru
    Nov 8, 2014 at 17:19
  • @Alexandru It is not about ReadAsync. It is parallel period.
    – paparazzo
    Nov 8, 2014 at 18:54
  • Yeah, I realize that now from usr's comments on my solution. Whether you Read or ReadAsync, its the same thing if you're doing it through an AsParallel iterator.
    – Alexandru
    Nov 8, 2014 at 18:57
0

File access is inherently not parallel. You can only benefit from parallelism, if you treat some files while reading others. It makes no sense to wait for the disk in parallel.

Instead of waiting 100 000 time 1 ms for disk access, you program to wait once 100 000 ms = 100 s.

4
  • But is there no scheduling method that optimizes these read times? I am going to try some tricky stuff with async methods. I mean, ideally, there must be some way to optimize read times. There must be, it must be built in.
    – Alexandru
    Nov 8, 2014 at 7:44
  • @Alexandru: there is some theoretically possible benefit to having multiple read operations concurrently, as it allows the Windows and the disk controller to schedule reads in a more efficient way. But a) the improvement is not going to be terribly dramatic, and b) if you go too far, you'll wind up with too much contention and less performance. You should be starting with a baseline measurement that you can compare to the theoretical maximum performance given the hardware. Otherwise, there's no way to answer whether what you're seeing is "too slow". Nov 8, 2014 at 7:47
  • I agree with Peter. This is really what I meant : There is an incompressible time with disk access, that is due to physical limitations and the way the OS and the controller work. Nov 8, 2014 at 7:53
  • In your other question you state that you get the data of 500 000 files and 80 000 directories in 3.5 s, that is 7 ms per file and 37 ms per directory. Compare this to the theoretical random read time of your hard disk. I think you're pretty close to touch the ground. Nov 8, 2014 at 8:01
0

Unfortunately, it's a vague question without a reproducible code example. So it's impossible to offer specific advice. But my two recommendations are:

  • Pass a ParallelOptions instance where you've set the MaxDegreeOfParallelism property to something reasonably low. Something like the number of cores in your system, or even that number minus one.

  • Make sure you aren't expecting too much from the disk. You should start with the known speed of the disk and controller, and compare that with the data throughput you're getting. Adjust the degree of parallelism even lower if it looks like you're already at or near the maximum theoretical throughput.

Performance optimization is all about setting realistic goals based on known limitations of the hardware, measuring your actual performance, and then looking at how you can improve the costliest elements of your algorithm. If you haven't done the first two steps yet, you really should start there. :)

2
  • Something like the number of cores in your system, or even that number minus one. The CPU will not be the bottleneck. Setting MaxDOP based on that is a bad idea. Nov 8, 2014 at 8:12
  • The OP is complaining that his UI is being blocked. So something is consuming the available CPU resources. Assuming he's implemented the parallel code correctly (and I grant that without a good code example, this is not guaranteed), that suggests that while the I/O should be the bottleneck, he's somehow managed to keep the CPU busy anyway. And a way around that is to specifically reduce the number of concurrent threads. Nov 8, 2014 at 8:42
0

I got it working; the problem was me trying to use an ExtendedObservableCollection with AddRange instead of calling Add multiple times in every UI dispatch...for some reason, the performance of the methods people list in here is actually slower in my situation: ObservableCollection Doesn't support AddRange method, so I get notified for each item added, besides what about INotifyCollectionChanging?

I think because it forces you to call change notifications with .Reset (reload) instead of .Add (a diff), there is some sort of logic in place that causes bottlenecks.

I apologize for not posting the rest of the code; I was really thrown off by this, and I'll explain why in a moment. Also, a note for others who come across the same issue, this might help. The main problem with profiling tools in this scenario is that they don't help much here. Most of your app's time will be spent reading files regardless. So you have to unit test all dispatchers separately.

2
  • Async IO gives you zero perf advantage in this scenario (why would it?). If your code got faster you either changed something else or your measurement is wrong.
    – usr
    Nov 8, 2014 at 13:40
  • @usr You're right, actually. Performance doesn't matter whether I use ReadAsync or not. Just tried it. I thought for some reason ReadAsync uses some sort of "smart scheduling" for this kind of IO bound stuff, but I guess not. Will edit the answer, thanks! It was all the way I was dispatching.
    – Alexandru
    Nov 8, 2014 at 17:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.