3

I have this code for adding about 1500 record to database:

async void button7_Click(object sender, EventArgs e)
{
    var task = await Task.Run(() =>
    {
        Random rnd = new Random();
        for (int i = 0; i <= 1500; i++)
        {
            db.Tbls.Add(new Tbl()
            {
                Name = "User" + i + 1,
                Num = rnd.Next(10, i + 10) / 10
            });
            progress.Report(i * 100 / 1500);
        }

        db.SaveChanges();
        return db.Tbls.Count();

    });
}

But it took about 4 second to complete the process but because I used async/await it doesn't freezes the UI. Now my question is: how can I improve this code to run faster. How can I use parallel programming here? Can you show me an example of how use TPL?

EDIT:

I used parallel loop and AddRange as suggested but it has no effect. In all suggested ways my process still take about 4 seconds. I really appreciate if someone can help me to solve this issue.

1

2 Answers 2

3

If performance is the goal then you should use something other than EF like dapper (same thing StackOverflow.com is using) or raw ADO.Net. Other than that you could improve the Entity Framework performance by disabling change auto detection and validation on save:

yourDbContext.Configuration.AutoDetectChangesEnabled = false;
yourDbContext.Configuration.ValidateOnSaveEnabled = false;

You should combine the above code with @Evk advice of using AddRange.

Another solution that might help speedup your inserts is by using BulkInsert (image taken from here):

enter image description here

4
  • Thanks. Please read my comments below the other answer. There is no difference with AddRange and Add and AutoDetectChangesEnabled, it takes more than 3 seconds at least. Why?
    – user5032790
    May 22, 2016 at 13:10
  • As I said, there's only so much you can do to optimize ef code. if 3 seconds is unacceptable for you you should start looking into ADO.Net.
    – Nasreddine
    May 22, 2016 at 13:22
  • I'm really don't understand. Why should ADO.NET be more faster than Entity Framework?
    – user5032790
    May 22, 2016 at 13:27
  • See this other question which goes into more details than I can fit in a comment.
    – Nasreddine
    May 22, 2016 at 13:33
3

Parallel processing won't be of big help here, because your loop by itself takes no time at all. All consumed time goes into two things:

  1. When you add new item to Entity Framework context (and you use EF here as I understand) - it performs some operations on many items in the context (not just the item you added) and it goes slower and slower the more items you added.

  2. Perfoming 1500 database inserts also takes some time.

The easiest thing to do here is to reduce time taken by point 1 in the list above. You can do it like this:

async void button7_Click(object sender, EventArgs e)
{
    var task = await Task.Run(() =>
    {
        Random rnd = new Random();
        var tbls = new List<Tbl>();
        for (int i = 0; i <= 1500; i++)
        {
            tbls.Add(new Tbl()
            {
                Name = "User" + i + 1,
                Num = rnd.Next(10, i + 10) / 10
            });
            progress.Report(i * 100 / 1500);
         }
        db.Tbls.AddRange(tbls);
        db.SaveChanges();
        return db.Tbls.Count();

    });
}

By using AddRange and first collecting all items in simple List you will greately reduce time consumed by your code to at least below 1 second.

Update. If you want to use parallel loop, even if that won't help, you can do it like this:

int seed = Environment.TickCount;
var random = new ThreadLocal<Random>(() => new Random(Interlocked.Increment(ref seed)));
var tbls = new ConcurrentBag<Tbl>();
Parallel.For(0, 1500, (i) => {
    tbls.Add(new Tbl()
    {
        Name = "User" + i + 1,
        Num = random.Value.Next(10, i + 10) / 10
    });
});                
db.Tbls.AddRange(tbls);

Things to note:

  • Random is not thread safe so we use thread local instances (one for each thread inside parallel loop), each one with different seed value.
  • List is not thread safe - we use ConcurrentBag instead.
11
  • Never thought of the AddRange trick. All I knew was to do what @Nasreddine suggests. Will try this as well.
    – Veverke
    May 22, 2016 at 12:54
  • Thanks. I just want to know what is the equivalent of my code in TPL or parallel processing? No matter if it helps or not just want to know how can I do this by TPL?
    – user5032790
    May 22, 2016 at 12:56
  • Also it took 6 seconds again even by using a List.
    – user5032790
    May 22, 2016 at 13:00
  • I've tested this and with AddRange instead of Add inside a loop it takes 700ms for me versus 6.6 seconds with your original code.
    – Evk
    May 22, 2016 at 13:03
  • No. There is no difference for me with AddRange and Add it takes more than 3 seconds at least. Why?
    – user5032790
    May 22, 2016 at 13:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.