0

I have an SQLite database on which I am running a particular set of queries. If the database file is new, then these queries run in about half a millisecond.

Now if I fill up a table, and delete everything the file will not shrink, but rather the pages in the sqlite file will be marked as free. However next time I run those same queries, they now take between 9 and 11 milliseconds. That is a slowdown of 20x which worries me. Can anyone shed some light as to why this is happening and what can I do to fix this? (if at all possible)

I am accessing the SQLite database using System.Data.Sqlite on c#

I am also using transactions for this. I know inserts are slow without them, but this problem is definitely not caused by that since I know for sure that I am opening a transaction myself (without a transaction they take upwards of 20 milliseconds)

2
  • maybe this is ane explanation: sqlite.org/faq.html#q19 Feb 5, 2013 at 9:42
  • But as I have said in the question, I am already opening a transaction so the problem is not coming from that. If the database file is empty I easily get over 20k queries per second. If the database is not empty then that drops down to about a 100 per second, which is not really that much faster than not using a transaction in the first place. Regarding the synchronous option, I cannot turn that off as I cannot risk database corruption. So there is no way around this? Feb 5, 2013 at 9:48

1 Answer 1

0

Sadly I don't have the reputation yet to just leave this in the comments, but the answer at:

This stack overflow quetion - change sqlite file size after “DELETE FROM table”

should go some way to helping out. The command you are looking for is

vacuum
2
  • Yeah I know about that. I don't know exactly the size of the database. But in some use cases it can reach sizes of up to around 5gb. Vacuum is very expensive then. But I guess it's a necessary evil Feb 5, 2013 at 10:05
  • 1
    Using the Auto-Vacuum mode may mitigate the expense of the "normal" vacuum - it has a few cons though that might be worth looking into. It does the vacuuming of empty pages at the end of a transaction BUT it does tend to increase fragmentation as it moves files around and doesn't do the de-fragmentation that the 'normal' vacuum performs.
    – Kobunite
    Feb 5, 2013 at 11:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.