20

Query:

update mytable 
    set mycol = null
    where id in (
        583048,
        583049,
        ... (50000 more)
)

Message:

The query processor ran out of internal resources and could not produce a query plan. This is a rare event and only expected for extremely complex queries or queries that reference a very large number of tables or partitions. Please simplify the query. If you believe you have received this message in error, contact Customer Support Services for more information.

My query is very simple, how should I write it so it works ok?

1
  • I had a very similar problem. For me it was a simple fix: Try adding AsParallel() to the Query, just prior to the ToList() or Count(). This option splits the query up and executes them separately, joining the result. Who know's you may get a performance improvement too.
    – Rob L
    Jun 23, 2017 at 3:47

3 Answers 3

24

Insert the list of values into a #temp table then use in on that.

As explained in this answer a large number of IN values can cause it to run out of stack as they get expanded to OR

See also related connect item

5
  • 1
    Well you can not insert values into a #temp table, it fails with the same error.
    – Softlion
    Oct 20, 2011 at 9:01
  • SQL Server Management Studio crashed when the syntax analysis scans the text (too much lines ?). So i used visual studio, cut the list in 10 groups of 10k lines each, and run the query 10 times ...
    – Softlion
    Oct 20, 2011 at 9:02
  • @Softion - What fails? The insert to a #temp table or using the #temp table inside the in? Oct 20, 2011 at 9:03
  • The insert fails with the same error. Or i missed something ?
    – Softlion
    Oct 24, 2011 at 16:36
  • 2
    Put the 50000 values in a CSV file and use BULK INSERT to insert the data into a table. Then you can work with it.
    – Alexander
    Sep 8, 2020 at 9:50
0

If you find this error in SQL Server 2016 or later then try to change the database compatibility level to a lower version 120 or 110, etc. (link)

0

There is an easy workarround, split your predicates Replace this

SELECT * FROM TABLE
WHERE ID like (1,2,3, ... 10000)

By

SELECT * FROM TABLE
WHERE ID like (1,2,3, 999) OR 
ID like (1000, .. 1999) OR 
... OR 
ID like (9000, .. 10000)

I encountered this error in EF and I fixed it using linqkit 1.2.4 package. I replaced

query.Table.Where(t => ids.contains(id))

by

var dividedIds = DivideArray(ids.ToArray());
foreach (var batchPortalIds in dividedPortalIds)
{
     predicate.Or(s => batchPortalIds.Contains(s.PortalId.Value));
}

Where DivideArray is a function transforming single array to list of array

    private List<int[]> DivideArray(int[] originalArray)
        {
            int batchSize = 1000;
    
            List<int[]> dividedArrays = new List<int[]>();
            int currentIndex = 0;
    
            while (currentIndex < originalArray.Length)
            {
                int batchSizeRemaining = Math.Min(batchSize, originalArray.Length - currentIndex);
                int[] batch = new int[batchSizeRemaining];
    
                Array.Copy(originalArray, currentIndex, batch, 0, batchSizeRemaining);
                dividedArrays.Add(batch);
    
                currentIndex += batchSizeRemaining;
            }
    
            return dividedArrays;
        }

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.