22

I need to optimize a WCF service... it's quite a complex thing. My problem this time has to do with tasks (Task Parallel Library, .NET 4.0). What happens is that I launch several tasks when the service is invoked (using Task.Factory.StartNew) and then wait for them to finish:

Task.WaitAll(task1, task2, task3, task4, task5, task6);

Ok... what I see, and don't like, is that on the first call (sometimes the first 2-3 calls, if made quickly one after another), the final task starts much later than the others (I am looking at a case where it started 0.5 seconds after the others). I tried calling

ThreadPool.SetMinThreads(12*Environment.ProcessorCount, 20);

at the beginning of my service, but it doesn't seem to help.

The tasks are all database-related: I'm reading from multiple databases and it has to take as little time as possible.

Any idea why the last task is taking so long? Is there something I can do about it?

Alternatively, should I use the thread pool directly? As it happens, in one case I'm looking at, one task had already ended before the last one started - I would had saved 0.2 seconds if I had reused that thread instead of waiting for a new one to be created. However, I can not be sure that that task will always end so quickly, so I can't put both requests in the same task.

[Edit] The OS is Windows Server 2003, so there should be no connection limit. Also, it is hosted in IIS - I don't know if I should create regular threads or using the thread pool - which is the preferred version?

[Edit] I've also tried using Task.Factory.StartNew(action, TaskCreationOptions.LongRunning); - it doesn't help, the last task still starts much later (around half a second later) than the rest.

[Edit] MSDN1 says:

The thread pool has a built-in delay (half a second in the .NET Framework version 2.0) before starting new idle threads. If your application periodically starts many tasks in a short time, a small increase in the number of idle threads can produce a significant increase in throughput. Setting the number of idle threads too high consumes system resources needlessly.

However, as I said, I'm already calling SetMinThreads and it doesn't help.

3
  • +1 because I had the same issue two years ago and I wasn't happy with my solution. Hope somebody can give a nice answer :)
    – Matten
    Jan 17, 2011 at 22:40
  • Hi, from other comments and your question, it is not clear whether the 5 DB's are on 5 separate boxes or are all on 1 SQL server instance? Also, is the exact number of DB calls per WCF call 5, or if it varies, what is the avg. range of calls that are spawned? I want to create a test and would like to be as precise as possible. Jan 25, 2011 at 11:27
  • Sorry, I haven't seen your comment until now. Multiple boxes, don't think there are 5, but definitely more than one. There are exactly 6 DB calls for each service invocation (I'm combining the data); two of the calls are to the same database. (I'm searching for home and work phones in our phone DB.) Jan 26, 2011 at 22:42

5 Answers 5

7
+50

I have had problems myself with delays in thread startup when using the (.Net 4.0) Task-object. So for time-critical stuff I now use dedicated threads (... again, as that is what I was doing before .Net 4.0.)

The purpose of a thread pool is to avoid the operative system cost of starting and stopping threads. The threads are simply being reused. This is a common model found in for example internet servers. The advantage is that they can respond quicker.

I've written many applications where I implement my own threadpool by having dedicated threads picking up tasks from a task queue. Note however that this most often required locking that can cause delays/bottlenecks. This depends on your design; are the tasks small then there would be a lot of locking and it might be faster to trade some CPU in for less locking: http://www.boyet.com/Articles/LockfreeStack.html

SmartThreadPool is a replacement/extension of the .Net thread pool. As you can see in this link it has a nice GUI to do some testing: http://www.codeproject.com/KB/threads/smartthreadpool.aspx

In the end it depends on what you need, but for high performance I recommend implementing your own thread pool. If you experience a lot of thread idling then it could be beneficial to increase the number of threads (beyond the recommended cpucount*2). This is actually how HyperThreading works inside the CPU - using "idle" time while doing operations to do other operations.

Note that .Net has a built-in limit of 25 threads per process (ie. for all WCF-calls you receive simultaneously). This limit is independent and overrides the ThreadPool setting. It can be increased, but it requires some magic: http://www.csharpfriends.com/Articles/getArticle.aspx?articleID=201

2
  • Thanks... I downloaded this (SmartThreadPool) and tried it... same thing happens. I'm still trying to figure out the freaking thing... it's not a huge deal (in some cases it wouldn't save me any time, as the last thread doesn't always finish last), but it's really bugging me. Jan 21, 2011 at 20:26
  • 1
    The 25 threads applied to .NET 1 - .NET 4 doesn't have have a fixed limit, but depending on various factors - see msdn.microsoft.com/en-us/library/… You can change the version at the top to go back in history. Mar 1, 2011 at 7:42
3

Following from my prior question (yep, should have been a Q against original message - apologies):

Why do you feel that creating 12 threads for each processor core in your machine will in some way speed-up your server's ability to create worker threads? All you're doing is slowing your server down!

As per MSDN do

As per the MSDN docs: "You can use the SetMinThreads method to increase the minimum number of threads. However, unnecessarily increasing these values can cause performance problems. If too many tasks start at the same time, all of them might appear to be slow. In most cases, the thread pool will perform better with its own algorith for allocating threads. Reducing the minimum to less than the number of processors can also hurt performance.".

Issues like this are usually caused by bumping into limits or contention on a shared resource.

In your case, I am guessing that your last task(s) is/are blocking while they wait for a connection to the DB server to come available or for the DB to respond. Remember - if your invocation kicks off 5-6 other tasks then your machine is going to have to create and open numerous DB connections and is going to kick the DB with, potentially, a lot of work. If your WCF server and/or your DB server are cold, then your first few invocations are going to be slower until the machine's caches etc., are populated.

Have you tried adding a little tracing/logging using the stopwatch to time how long it takes for your tasks to connect to the DB server and then execute their operations?

You may find that reducing the number of concurrent tasks you kick off actually speeds things up. Try spawning 3 tasks at a time, waiting for them to complete and then spawn the next 3.

1
  • 1. There are multiple databases - 5 of them to be precise. 2. Since the threads are blocked, they shouldn't cause performance problems. They are not using CPU during that time. 3. Yes, I have detailed timing for everything... opening the connection to the DB only takes a few hundredths of a second. 4. No, reducing the number of concurrent tasks won't speed things up... and anyway 5. The only problem I am trying to solve in this question is: why is a thread starting so much later, and how can I make it start at the same time as the others? Jan 20, 2011 at 16:15
3

When you call Task.Factory.StartNew, it uses a TaskScheduler to map those tasks into actual work items.

In your case, it sounds like one of your Tasks is delaying occasionally while the OS spins up a new Thread for the work item. You could, potentially, build a custom TaskScheduler which already contained six threads in a wait state, and explicitly used them for these six tasks. This would allow you to have complete control over how those initial tasks were created and started.

That being said, I suspect there is something else at play here... You mentioned that using TaskCreationOptions.LongRunning demonstrates the same behavior. This suggests that there is some other factor at play causing this half second delay. The reason I suspect this is due to the nature of TaskCreationOptions.LongRunning - when using the default TaskScheduler (LongRunning is a hint used by the TaskScheduler class), starting a task with TaskCreationOptions.LongRunning actually creates an entirely new (non-ThreadPool) thread for that Task. If creating 6 tasks, all with TaskCreationOptions.LongRunning, demonstrates the same behavior, you've pretty much guaranteed that the problem is NOT the default TaskScheduler, since this is going to always spin up 6 threads manually.

I'd recommend running your code through a performance profiler, and potentially the Concurrency Visualizer in VS 2010. This should help you determine exactly what is causing the half second delay.

0
1

What is the OS? If you are not running the server versions of windows, there is a connection limit. Your many threads are probably being serialized because of the connection limit.

Also, I have not used the task parallel library yet, but my limited experience is that new threads are cheap to make in the context of networking.

1
  • No, it's server - Windows Server 2003. Jan 18, 2011 at 6:39
1

These articles might explain the problem you're having: http://blogs.msdn.com/b/wenlong/archive/2010/02/11/why-are-wcf-responses-slow-and-setminthreads-does-not-work.aspx

http://blogs.msdn.com/b/wenlong/archive/2010/02/11/why-does-wcf-become-slow-after-being-idle-for-15-seconds.aspx

seeing as you're using .Net 4, the first article probably doesn't apply, but as the second article points out the ThreadPool terminates idle threads after 15 seconds which might explain the problem you're having and offers a simple (though a little hacky) solution to get around it.

Whether or not you should be using the ThreadPool directly wouldn't make any difference as I suspect the task library is using it for you underneath anyway.

One third-party library we have been using for a while might help you here - Smart Thread Pool. You still get the same benefits of using the task libraries, in that you can have the return values from the threads and get any exception information from them too.

Also, you can instantiate threadpools so that when you have multiple places each needing a threadpool (so that a low priority process doesn't start eating into the quota of some high priority process) and oh yeah you can set the priority of the threads in the pool too which you can't do with the standard ThreadPool where all the threads are background threads.

You can find plenty of info on the codeplex page, I've also got a post which highlights some of the key differences: http://theburningmonk.com/2010/03/threading-introducing-smartthreadpool/

Just on a side note, for tasks like the one you've mentioned, which might take some time to return, you probably shouldn't be using the threadpool anyway. It's recommended that we should avoid using the threadpool for any blocking tasks like that because it hogs up the threadpool which is used by all sorts of things by the framework classes, like handling timer events, etc. etc. (not to mention handling incoming WCF requests!). I feel like I'm spamming here but here's some of the info I've gathered around the use of the threadpool and some useful links at the bottom:

http://theburningmonk.com/2010/03/threading-using-the-threadpool-vs-creating-your-own-threads/

well, hope this helps!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.