In the project I have been recently working on, many (PostgreSQL) database tables are just used as big lookup arrays. We have several background worker services, which periodically pull the latest data from a server, then replace all contents of a table with the latest data. The replacing has to be atomic because we don't want a partially completed table to be seen by lookup-ers.
I thought the simplest way to do the replacing is something like this:
BEGIN;
DELETE FROM some_table;
COPY some_table FROM 'source file';
COMMIT;
But I found a lot of production code use this method instead:
BEGIN;
CREATE TABLE some_table_tmp (LIKE some_table);
COPY some_table_tmp FROM 'source file';
DROP TABLE some_table;
ALTER TABLE some_table_tmp RENAME TO some_table;
COMMIT;
(I omit some logic such as change the owner of a sequence, etc.)
I just can't see any advantage of this method. Especially after some discoveries and experiments. SQL statements like ALTER TABLE
and DROP TABLE
acquire an ACCESS EXCLUSIVE lock, which even blocks a SELECT.
Can anyone explain what problem the latter SQL pattern is trying to solve? Or it's wrong and we should avoid using it?
delete
+insert from tmp_table
. It's fast, but might be a problem for a while. You should consider usingmaterialized views
- data truncated from table, have no influence in that kind of query. To updatematerialized view
you have to execute commandREFRESH MATERIALIZED VIEW
postgresql.org/docs/9.6/static/sql-creatematerializedview.htmldelete
is really slow on large tablesTRUNCATE
? It also acquires an ACCESS EXCLUSIVE lock, but avoids creating a new table and swapping back.COPY
process. Using a second table avoids that - the "real" table is only locked for a very brief moment