I have a table like this:
Column | Type | Modifiers
-------------+-----------------------------+-------------------------------------------------------
id | integer | not null default nextval('oks_id_seq'::regclass)
uname | text | not null
ess | text |
quest | text |
details | text |
status | character(1) | not null default 'q'::bpchar
last_parsed | timestamp without time zone |
qstatus | character(1) | not null default 'q'::bpchar
media_wc | integer | not null default 0
Indexes:
"oks_pkey" PRIMARY KEY, btree (id)
"oks_uname_key" UNIQUE CONSTRAINT, btree (uname)
"last_parsed_idx" btree (last_parsed)
"qstatus_idx" btree (qstatus)
"status_idx" btree (status)
And I have a query like this:
SELECT COUNT(status), status FROM oks GROUP BY status ORDER BY status;
Which results in:
count | status
---------+--------
1478472 | d
23599 | p
10178 | q
6278206 | s
(4 rows)
Which is great, but this takes forever, and for some reason Postgres keeps the whole index on disk, because disk activity is really high during the query.
Sort (cost=1117385.91..1117385.92 rows=4 width=2) (actual time=54122.991..54122.993 rows=4 loops=1)
Sort Key: status
Sort Method: quicksort Memory: 25kB
-> HashAggregate (cost=1117385.82..1117385.86 rows=4 width=2) (actual time=54122.280..54122.283 rows=4 loops=1)
-> Seq Scan on oks (cost=0.00..1078433.55 rows=7790455 width=2) (actual time=0.009..47978.616 rows=7790455 loops=1)
Total runtime: 54123.487 ms
(6 rows)
In my config I have the memory usage set at work_mem = 128MB
Any ideas about how I can optimize such queries that use group by on the whole table? This seems unrealistically slow as it would have been much faster with flat files storage.
Edit: I was able to get the query to run in a fraction of a second by modifying the postgres config files. Specifically, setting
fsync = off
synchronous_commit = off
full_page_writes = off
commit_delay = 2000
effective_cache_size = 4GB
work_mem = 512MB
maintenance_work_mem = 512MB
Not sure if these are optimal, but these options work in my case. fsync = off helped the most I think.
status_idx
which should be faster than the seq scan on the table. Does this change if you run avacuum analyze
on the table? – a_horse_with_no_name Sep 24 '15 at 7:09