5

In the project I have been recently working on, many (PostgreSQL) database tables are just used as big lookup arrays. We have several background worker services, which periodically pull the latest data from a server, then replace all contents of a table with the latest data. The replacing has to be atomic because we don't want a partially completed table to be seen by lookup-ers.

I thought the simplest way to do the replacing is something like this:

BEGIN;
DELETE FROM some_table;
COPY some_table FROM 'source file';
COMMIT;

But I found a lot of production code use this method instead:

BEGIN;
CREATE TABLE some_table_tmp (LIKE some_table);
COPY some_table_tmp FROM 'source file';
DROP TABLE some_table;
ALTER TABLE some_table_tmp RENAME TO some_table;
COMMIT;

(I omit some logic such as change the owner of a sequence, etc.)

I just can't see any advantage of this method. Especially after some discoveries and experiments. SQL statements like ALTER TABLE and DROP TABLE acquire an ACCESS EXCLUSIVE lock, which even blocks a SELECT.

Can anyone explain what problem the latter SQL pattern is trying to solve? Or it's wrong and we should avoid using it?

14
  • I speculate the intent is one of two things: (1) To minimize the time when the table is unaccessible or (2) To ensure that either the whole table or nothing is available (no partial data). May 5, 2018 at 11:47
  • 1
    Instead of drop + recreate table i prefer delete + insert from tmp_table. It's fast, but might be a problem for a while. You should consider using materialized views - data truncated from table, have no influence in that kind of query. To update materialized view you have to execute command REFRESH MATERIALIZED VIEW postgresql.org/docs/9.6/static/sql-creatematerializedview.html
    – Lemjur
    May 5, 2018 at 11:51
  • The first solution might take a lot longer as delete is really slow on large tables
    – user330315
    May 5, 2018 at 11:55
  • 1
    @a_horse_with_no_name How about using TRUNCATE? It also acquires an ACCESS EXCLUSIVE lock, but avoids creating a new table and swapping back. May 5, 2018 at 12:40
  • But then the table is locked and not accessible during the COPY process. Using a second table avoids that - the "real" table is only locked for a very brief moment
    – user330315
    May 5, 2018 at 12:41

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.