2

I would like to parallelize the application that processes multiple video clips frame by frame. Sequence of each frame per clip is important (obviously). I decided to go with TPL Dataflow since I believe this is a good example of dataflow (movie frames being data).

So I have one process that loads frames from database (lets say in a batch of 500, all bunched up)

Example sequence:    
|mid:1 fr:1|mid:1 fr:2|mid:2 fr:1|mid:3 fr:1|mid:1 fr:3|mid:2 fr:2|mid:2 fr:3|mid:1 fr:4|

and posts them to BufferBlock. To this BufferBlock I have linked ActionBlocks with the filter to have one ActionBlock per MovieID so that I get some kind of data partitioning. Each ActionBlock is sequential, but ideally multiple ActionBlocks for multiple movies can run in parallel.

I do have the above described network working and it does run in parallel, but from my calculations only eight to ten ActionBlocks are executing simultaneously. I timed each ActionBlock's running time and its around 100-200ms. What steps can I take to at least double concurrency?

I did try converting action delegates to async methods and make database access asynchronous within ActionBlock action delegate but it did not help.

EDIT: I implemented extra level of data partitioning: frames for Movies with Odd IDs are processed on ServerA, frames for Even movies are processed on ServerB. Both instances of the application hit the same database. If my problem was DB IO, then I would not see any improvement in total frames processed count (or very little, under 20%). But I do see it doubling. So this leads me to conclude that Threadpool is not spawning more threads to do more frames in parallel (both servers are quad-cores and profiler shows about 25-30 threads per application).

4
  • Does your action block write any data to disk or back to SQL? Does your action block have any IO? This could be limited by getting data from SQL. When you say CPU under 50% is the average or stays under 50%.
    – paparazzo
    Nov 14, 2012 at 18:16
  • ActionBlock does Read/Write data from/to SQL. And CPU does stay under 50% averaging 25%. I did convert the DB access methods to async and it kinda increased productivity by around 20%. But still think I can do better.
    – Dimitri
    Nov 14, 2012 at 18:28
  • Parallel Stacks does show around 18-20 Threadpool threads. If I could bump this up to 50, I'd be a happy camper.
    – Dimitri
    Nov 14, 2012 at 18:31
  • 1
    "What steps can I take to at least double concurrency?" All due respect, you can't just say "I want to double X" and that means it can be done. Whether or not it's possible is dependent on many more factors than you've laid out here. At best, we can give you guidance as to how to improve concurrency or the overall processing.
    – casperOne
    Nov 15, 2012 at 18:18

3 Answers 3

2

Some assumptions:

  • From your example data, you are receiving movie frames (and possibly the frames in the movies) out of order

  • Your ActionBlock<T> instances are generic; they all call the same method for processing, you just create a list of them based on each movie id (you have a list of movie ids beforehand) like so:

// The movie IDs
IEnumerable<int> movieIds = ...;

// The actions.
var actions = movieIds.Select(
    i => new { Id = i, Action = new ActionBlock<Frame>(MethodToProcessFrame) });

// The buffer block.
BufferBlock<Frame> buffer = ...;

// Link everything up.
foreach (var action in actions) 
{
    // Not necessary in C# 5.0, but still, good practice.
    // The copy of the action.
    var actionCopy = action;

    // Link.
    bufferBlock.LinkTo(actionCopy.Action, f => f.MovieId == actionCopy.Id);
}

If this is the case, you're creating too many ActionBlock<T> instances which aren't being given work; because your frames (and possibly movies) are out-of-order, you aren't guaranteed that all of the ActionBlock<T> instances will have work to do.

Additionally, when you create an ActionBlock<T> instance it's going to be created with a MaxDegreeOfParallelism of 1, meaning that it's thread safe because only one thread can access the block at the same time.

Additionally, the TPL DataFlow library ultimately relies on the Task<TResult> class, which schedules by default on the thread pool. The thread pool is going to do a few things here:

  • Make sure that all processor cores are saturated. This is very different from making sure that your ActionBlock<T> instances are saturated and this is the metric you should be concerned with

  • Make sure that while the processor cores are saturated, make sure that the work is distributed evenly, as well as make sure that not too many concurrent tasks are executing (context switches are expensive).

It also looks like your method that processes your movies is generic, and it doesn't matter what frame from what movie is passed in (if it does matter, then you need to update your question with that, as it changes a lot of things). This would also mean that it's thread-safe.

Also, if it can be assumed that the processing of one frame doesn't rely on the processing of any previous frames (or, it looks like the frames of the movie come in order) you can use a single ActionBlock<T> but tweak up the MaxDegreeOfParallelism value, like so:

// The buffer block.
BufferBlock<Frame> buffer = ...;

// Have *one* ActionBlock<T>
var action = new ActionBlock<Frame>(MethodToProcessFrame,
    // This is where you tweak the concurrency:
    new ExecutionDataflowBlockOptions {
        MaxDegreeOfParallelism = 4,
    }
);

// Link.  No filter needed.
bufferBlock.LinkTo(action);

Now, your ActionBlock<T> will always be saturated. Granted, any responsible task scheduler (the thread pool by default) is still going to limit the maximum amount of concurrency, but it's going to do as much as it can reasonably do at the same time.

To that end, if your action is truly thread safe, you can set the MaxDegreeOfParallelism to DataflowBlockOptions.Unbounded, like so:

// Have *one* ActionBlock<T>
var action = new ActionBlock<Frame>(MethodToProcessFrame,
    // This is where you tweak the concurrency:
    new ExecutionDataflowBlockOptions {
        // We're thread-safe, let the scheduler determine
        // how nuts we can go.
        MaxDegreeOfParallelism = DataflowBlockOptions.Unbounded,
    }
);

Of course, all of this assumes that everything else is optimal (I/O reads/writes, etc.)

8
  • I do state in my question that "Sequence of each frame per clip is important". That means that I cannot process frame 2 of movie 1 before frame 1 of movie 1. And I am relying on the output of frame 1 processing to process frame 2 of the same movie. So order in which movies are processed doesn't matter, but order of frames for any given movie matters. That's why I'm partitioning data using ActionBlocks and Filtered Links
    – Dimitri
    Nov 15, 2012 at 20:23
  • On another note, using my current architecture, I commented out data access code, doubling productivity. So that leads me to believe its a DB IO issue
    – Dimitri
    Nov 15, 2012 at 20:24
  • @Dimitri If it is a DB IO issue, then that makes this question "Too Localized" and it should be closed as such. I missed the part about sequences (and it's not obvious, as not all video processing is dependent on the last clip, depending on what you're trying to do, as the output can be reordered), I'll rework the answer to account for that but if you enter it into the BufferBlock<T> in order, then you will get it fed in that order. You just have to make sure you're not processing frames concurrently from the same movie. Also, this will limit just how concurrent you can get, obviously.
    – casperOne
    Nov 15, 2012 at 20:28
  • I might try to change data partitioning from per movie to lets say per Odd Movie ID and Even Movie ID and utilize some kinda blocking to avoid parallel processing of the frames of the same movie
    – Dimitri
    Nov 15, 2012 at 20:33
  • @Dimitri How much dependency do you have between frames? Does the processing of one frame rely on the processing of a previous frame? You might be able to use TPL to handle this aggregation for you.
    – casperOne
    Nov 15, 2012 at 20:42
-1

Odds are that's the optimal degree of parallelization. The thread pool is honestly pretty darn good at determining the optimal number of actual threads to have active. My guess is that your hardware can support about that many parallel processes actually working in parallel. If you added more you wouldn't actually be increasing throughput, you'd just be spending more time doing context switches between threads and less time actually working on them.

If you notice that, over an extended period of time, your CPU load, memory bus, network connection, disk access, etc. are all working below capacity then you might have a problem, and you'd want to check to see what is actually bottlenecking. Chances are though some resource somewhere is at it's capacity, and the TPL has recognized that and ensured that it doesn't over saturate that resource.

2
  • yeah, my CPU is under 50% load, network is around 15%, memory is 70%. I do feel like there's a place to grow. If for example i start two instances of the same application, I could potentially double output. But I hate running multiple instances of the app. Want to have them under one roof
    – Dimitri
    Nov 14, 2012 at 17:04
  • Also I read a post about TPL Dataflow arguing against using TaskScheduler.Default (threadpool) in action blocks, but I can't remember where I saw that thread. It stated that threadpool is very pessimistic in spawning new threads and offered an alternative, but can't remember what was it (not ConcurrentScheduler)
    – Dimitri
    Nov 14, 2012 at 17:06
-1

I suspect you are IO bound. The question is where? On the read or the write. Are you writing more data than reading. CPU may be under 50% because it cannot write out faster.

I am not saying the ActionBlock is wrong but I would consider a producer consumer with BlockingCollection. Optimize how you read and write data.

This different but I have an app where I read blocks of text. Parse the text and then write the words back to SQL. I read the on a single thread, then parallel the parse, and then write on a single thread. I write on a single thread so as not to fracture indexes. If you are IO bound you need to figure out what is the slowest IO then optimize that process.

Tell me more about that IO.

In the question you mention reading from database also.
I would give BlockingCollections a try.
BlockingCollection Class
And have size limit for each as so you don't blow memory.
Make it just big enough that it (almost) never goes empty.
The Blocking Collection after the slowest step will go empty. If you can parallel process then do so.
What I have found is parallel inserts in a table are not faster.
Let one process take lock and hold it and keep that hose open.
Look close at how you insert.
One row at a time is slow.
I use TVP and insert 10,000 at a time but a lot of people like Drapper or BulkInsert.
If you drop indexes and triggers and insert sorted by clustered index will be fastest. Take a tablock and hold it. I am getting inserts in the 10 ms range.
Right now the update is the slowest. Look at that - are you doing just one row at a time?
Look at taking tablock and doing by video clip.
Unless it is an ugly update it should not take longer than in insert.

5
  • The ActionBlock consists of three segments with DB IO: 1) Insert into TableA (40-50ms); 2) Insert into TableB (50-60mx); 3) Update TableC and TableD (70-80ms). Step No.3 is dependent on Step No. 2 so I cannot run those three steps in parallel.
    – Dimitri
    Nov 14, 2012 at 20:52
  • I already use DataFlow BufferBlock that is capacity bound. Inserting in batches is not an option since one frame of one movie cannot wait till second frame of another movie is completed. Plus the update that comes after insert is dependent on the insert, so I can't easily just fire a separate thread for insert. And I can't drop indices on the table, cause the table is also accessed for reads (more reads than inserts).
    – Dimitri
    Nov 15, 2012 at 12:12
  • Don't understand how a batch would prevent you from processing multiple movies - just hold the results in collections. And don't understand how it is difficult to put an update dependent on an insert in a separate thread.
    – paparazzo
    Nov 15, 2012 at 13:52
  • Well, to have a batch I'll have to wait for the Nth frame to then issue a batch insert, correct?! And I'm saying waiting for the Nth frame is not an option. This is not really a movie processor app, but I chose movie processing as an example because it fits well.
    – Dimitri
    Nov 15, 2012 at 14:41
  • N does not need to be a constant. Processing a single row at a time is the slowest way. If you cannot hold "frames" in a collection then you are stuck with slow SQL IO.
    – paparazzo
    Nov 15, 2012 at 17:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.