47

Deletes on sql server are sometimes slow and I've been often in need to optimize them in order to diminish the needed time. I've been googleing a bit looking for tips on how to do that, and I've found diverse suggestions. I'd like to know your favorite and most effective techinques to tame the delete beast, and how and why they work.

until now:

  • be sure foreign keys have indexes

  • be sure the where conditions are indexed

  • use of WITH ROWLOCK

  • destroy unused indexes, delete, rebuild the indexes

now, your turn.

5
  • Question to the power users: this question does not have a single answer, is more a sort of a knowledge base than a simple question-answer. May be it could be turned into a community wiki? (if I well understand what a c.w. is for)
    – pomarc
    Jun 10, 2009 at 11:19
  • I'd like to see it as an on-going summary. This post was very useful to me but took hours to read through the suggestions. I've submitted one edit with a new summary that was rejected, waiting to see if the second attempt goes through :)
    – xero
    Nov 20, 2014 at 22:45
  • @xero I rolled back you edit, you might flag an administrator for attention (use other) after reading Should the community wiki police be shut down?What can we do to make Community Wiki better? and meta.stackoverflow.com/a/266921
    – bummi
    Nov 21, 2014 at 0:12
  • @bummi - Ah, after reading those, I think it would probably be best to do a CW Answer instead of the question itself. I'll post a summary answer to that effect and let the community have at it. Thanks for the follow up
    – xero
    Dec 11, 2014 at 18:52
  • One of the parameters that help reduce the time taken for delete is SET NOCOUNT ON; Mar 21, 2017 at 20:12

15 Answers 15

24

The following article, Fast Ordered Delete Operations may be of interest to you.

Performing fast SQL Server delete operations

The solution focuses on utilising a view in order to simplify the execution plan produced for a batched delete operation. This is achieved by referencing the given table once, rather than twice which in turn reduces the amount of I/O required.

1
15

I have much more experience with Oracle, but very likely the same applies to SQL Server as well:

  • when deleting a large number of rows, issue a table lock, so the database doesn't have to do lots of row locks
  • if the table you delete from is referenced by other tables, make sure those other tables have indexes on the foreign key column(s) (otherwise the database will do a full table scan for each deleted row on the other table to ensure that deleting the row doesn't violate the foreign key constraint)
2
  • 2
    table lock would prevent insert and update on the table, need to make sure the delete is fast before other transactions start timing out.
    – dsum
    Dec 7, 2011 at 23:20
  • dsum: true, but deleting a large number of records is something that typically happens in a maintenance window without other activity (e.g. at night). Dec 8, 2011 at 21:59
11

Summary of Answers through 2014-11-05

This answer is flagged as community wiki since this is an ever-evolving topic with a lot of nuances, but very few possible answers overall.

The first issue is you must ask yourself what scenario you're optimizing for? This is generally either performance with a single user on the db, or scale with many users on the db. Sometimes the answers are the exact opposite.

For single user optimization

  • Hint a TABLELOCK
  • Remove indexes not used in the delete then rebuild them afterward
  • Batch using something like SET ROWCOUNT 20000 (or whatever, depending on log space) and loop (perhaps with a WAITFOR DELAY) until you get rid of it all (@@ROWCOUNT = 0)
  • If deleting a large % of table, just make a new one and delete the old table
  • Partition the rows to delete, then drop the partition. Read more...

For multi user optimization

  • Hint row locks
  • Use the clustered index
  • Design clustered index to minimize page re-organization if large blocks are deleted
  • Update "is_deleted" column, then do actual deletion later during a maintenance window

For general optimization

  • Be sure FKs have indexes on their source tables
  • Be sure WHERE clause has indexes
  • Identify the rows to delete in the WHERE clause with a view or derived table instead of referencing the table directly. Read more...
10

I wonder if it's time for garbage-collecting databases? You mark a row for deletion and the server deletes it later during a sweep. You wouldn't want this for every delete - because sometimes a row must go now - but it would be handy on occasion.

3
  • I like the idea of that. You could implement this, simply mark a bit, To_be_delted and then run a query every so often to go an delete these values. But I agree that an automatic garbage collection system would be cool Aug 7, 2014 at 9:55
  • 2
    Deleting a row means at least three things: a) making sure no foreign key constraint is violated by the deletion b) marking the space occupied by the row as "available". c) removing the row from all indexes on that table. Of these, a) can be the most expensive (if the referencing tables do not have an index on the foreign key columns) but it should be done immediately, so you can tell the user "you can't delete this row, it's still referenced". b) is probably cheap and c) is usually not that expensive. Therefore, I'm not convinced of this idea. Nov 10, 2014 at 8:57
  • PostgreSQL handles deletes like that, deleting just marks the affected rows as deleted, a separate process called "vacuum" actually frees the occupied space. Sep 12, 2019 at 12:01
6

To be honest, deleting a million rows from a table scales just as badly as inserting or updating a million rows. It's the size of the rowset that's the problem, and there's not much you can do about that.

My suggestions:

  • Make sure that the table has a primary key and clustered index (this is vital for all operations).
  • Make sure that the clustered index is such that minimal page re-organisation would occur if a large block of rows were to be deleted.
  • Make sure that your selection criteria are SARGable.
  • Make sure that all your foreign key constraints are currently trusted.
1
  • 10
    SARGable:In relational databases, a condition (or predicate) in a query is said to be sargable if the DBMS engine can take advantage of an index to speed up the execution of the query (using index seeks, not covering indexes). The term is derived from a contraction of Search ARGument Able. (Wikipedia)
    – pomarc
    Jun 5, 2009 at 12:35
4

(if the indexes are "unused", why are they there at all?)

One option I've used in the past is to do the work in batches. The crude way would be to use SET ROWCOUNT 20000 (or whatever) and loop (perhaps with a WAITFOR DELAY) until you get rid of it all (@@ROWCOUNT = 0).

This might help reduce the impact upon other systems.

3
  • But there are ususally more things going on than just the delete... I guess it may help, but you've have to check that it doesn't (overall) make the system worse... Jun 5, 2009 at 12:34
  • Do not delete indexes just because they are unused in the delete! Other people are using the database for other things!
    – HLGEM
    Jun 5, 2009 at 13:02
  • I should have pointed out that destroy-delete-rebuild should be used only when lenghty deletes are done while the db is not used, such as by night, during batch operations, when the db is an enterprise db used only at daytime. obviously destroying indexes on a live and used db is not a good idea.
    – pomarc
    Jun 5, 2009 at 13:22
4

The problem is you haven't defined your conditions enough. I.e. what exactly are you optimizing?

For example, is the system down for nightly maintenance and no users are on the system? And are you deleting a large % of the database?

If offline and deleting a large %, may make sense to just build a new table with data to keep, drop the old table, and rename. If deleting a small %, you likely want to batch things in as large batches as your log space allows. It entirely depends on your database, but dropping indexes for the duration of the rebuild may hurt or help -- if even possible due to being "offline".

If you're online, what's the likelihood your deletes are conflicting with user activity (and is user activity predominantly read, update, or what)? Or, are you trying to optimize for user experience or speed of getting your query done? If you're deleting from a table that's frequently updated by other users, you need to batch but with smaller batch sizes. Even if you do something like a table lock to enforce isolation, that doesn't do much good if your delete statement takes an hour.

When you define your conditions better, you can pick one of the other answers here. I like the link in Rob Sanders' post for batching things.

1
  • thanks matt. well, my question is quite general, i've had slow deletes in various and different occasions, and this was meant to be a way to gather the tips people could share on the issue.
    – pomarc
    Jun 5, 2009 at 23:00
3

If you have lots of foreign key tables, start at the bottom of the chain and work up. The final delete will go faster and block less things if there are no child records to cascade delete (which I would NOT turn on if I had a large number fo child tables as it will kill performance).

Delete in batches.

If you have foreign key tables that are no longer being used (you'd be surprised how often production databses end up with old tables nobody will get rid of), get rid of them or at least break the FK/PK connection. No sense cheking a table for records if it isn't being used.

Don't delete - mark records as delted and then exclude marked records from all queries. This is best set up at the time of database design. A lot of people use this because it is also the best fastest way to get back records accidentlally deleted. But it is a lot of work to set up in an already existing system.

2

I'll add another one to this:

Make sure your transaction isolation level and database options are set appropriately. If your SQL server is set not to use row versioning, or you're using an isolation level on other queries where you will wait for the rows to be deleted, you could be setting yourself up for some very poor performance while the operation is happening.

2

On very large tables where you have a very specific set of criteria for deletes, you could also partition the table, switch out the partition, and then process the deletions.

The SQLCAT team has been using this technique on really really large volumes of data. I found some references to it here but I'll try and find something more definitive.

2

I think, the big trap with delete that kill the performance is that sql after each row deleted, it updates all the related indexes for any column in this row. what about delting all indexes before bulk delete?

0
1

There are deletes and then there are deletes. If you are aging out data as part of a trim job, you will hopefully be able to delete contiguous blocks of rows by clustered key. If you have to age out data from a high volume table that is not contiguous it is very very painful.

1

If it is true that UPDATES are faster than DELETES, you could add a status column called DELETED and filter on it in your selects. Then run a proc at night that does the actual deletes.

1

Do you have foreign keys with referential integrity activated? Do you have triggers active?

1

Simplify any use of functions in your WHERE clause! Example:

DELETE FROM Claims
WHERE dbo.YearMonthGet(DataFileYearMonth) = dbo.YearMonthGet(@DataFileYearMonth)

This form of the WHERE clause required 8 minutes to delete 125,837 records.

The YearMonthGet function composed a date with the year and month from the input date and set day = 1. This was to ensure we deleted records based on year and month but not day of month.

I rewrote the WHERE clause to:

WHERE YEAR(DataFileYearMonth) = YEAR(@DataFileYearMonth)
AND MONTH(DataFileYearMonth) = MONTH(@DataFileYearMonth)

The result: The delete required about 38-44 seconds to delete those 125,837 records!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.