3

I have a table MyTable with multiple int columns with date and one column containing a date. The date column has an index created like follows

CREATE INDEX some_index_name ON MyTable(my_date_column)

because the table will often be queried for its contents between a user-specified date range. The table has no foreign keys pointing to it, nor have any other indexes other than the primary key which is an auto-incrementing index filled by a sequence/trigger.

Now, the issue I have is that the data on this table is often replaced for a given time period because it was out of date. So they way it is updated is by deleting all the entries within a given time period and inserting the new ones. The delete is performed using

DELETE FROM MyTable 
WHERE my_date_column >= initialDate 
  AND my_date_column < endDate

However, because the number of rows deleted is massive (from 5 million to 12 million rows) the program pretty much blocks during the delete.

Is there something I can disable to make the operation faster? Or maybe specify an option in the index to make it faster? I read something about redo space having to do with this but I don't know how to disable it during an operation.

EDIT: The process runs every day and it deletes the last 5 days of data, then it brings the data for those 5 days (which may have changed in the external source) and reinserts the data.

The amount of data deleted is a tiny fraction compared to the whole amount of data in the table ( < 1%). So copying the data I want to keep into another table and dropping-recreating the table may not be the best solution.

2
  • What fraction of the rows are actually changing. If you are deleting 5 days of data and then re-inserting 5 days of data, could you rewrite the process to do a MERGE instead of a DELETE + INSERT? What "blocks" during the delete? A SELECT statement will never be blocked by DML. Perhaps the application becomes slow(er) because the DELETE is saturating your I/O subsystem? Or is your application trying to lock a row that your DELETE process is deleting? Aug 25, 2015 at 21:29
  • Oh, Brian, it's been months I don't work on that project, so I can't find out about that anymore. I'm not sure what the issue itself was in Oracle, but we moved to PostgreSQL and the same operations were much faster there. That's how we "solved" it. Aug 28, 2015 at 15:14

1 Answer 1

4

I can only think of two ways to speed up this.

  1. if you do this on a regular basis, you should consider partitioning your table by month. Then you just drop the partition of the month you want to delete. That is basically as fast as dropping a table. Partitioning requires an enterprise license if I'm not mistaken
  2. create a new table with the data you want to keep (using create table new_table as select ...), drop the old table and rename the interims table. This will be much faster, but has the drawback that you need to re-create all indexes and (primary, foreign key) constraints on the new table.
3
  • Would this be a viable way in Oracle?: delete the rows in smaller transactions (of say 10k rows per transaction), thus leaving room for other operations to work (i.e. get locks in the meantime when one stops and the the next starts) without waiting for one huge operation to finish and release locks? Sep 22, 2013 at 21:54
  • ypercube, I'll test that and see how it works. Please post that comment as an answer so I can put feedback on it after I test it. Sep 22, 2013 at 21:57
  • 1
    @ypercube: deleting in small batches is usually slower in Oracle because you essentially increase the workload due to the frequent commits. See here for a lenghty discussion: asktom.oracle.com/pls/apex/…
    – user330315
    Sep 23, 2013 at 5:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.