1

I have a requirement of taking the table backup into S3 in csv format . I already tried taking the pg_dump of the tables, but those are in .sql format. Also I tried using gz format, but they all contains the dump of the data. I have tried this command:

pg_dump -v -h <hostname>  -d <dbname> -t <tablename> -U <username>| gzip | aws s3 cp - s3://xxx/xxx/tablename.csv.gz 

Is it possible to get the tables exactly in csv file format so that it could be easy to create QuickSight reports out of that because we cant use pg_dump directly to create the QuickSight reports.

Any help would be much appreciated.

Note: the table size is between 50 to 100 GB

8
  • Is your PostgreSQL database running on Amazon RDS, or EC2 or your own computer? Aug 4, 2022 at 3:22
  • Hi @JohnRotenstein , its running on Amazon RDS
    – Niki
    Aug 4, 2022 at 3:29
  • hi @JohnRotenstein, i tried to do the export and it is working following the steps you mentioned, but while importing i am getting thie error function aws_s3.table_import_from_s3(unknown, unknown, unknown, unknown, aws_commons._s3_uri_1) does not exist LINE 1: SELECT aws_s3.table_import_from_s3( ^ HINT: No function matches the given name and argument types. You might need to add explicit type casts.
    – Niki
    Aug 4, 2022 at 17:54
  • Why are you using "import from S3"? I though your goal was to export it to S3. Into what are you importing? Aug 4, 2022 at 22:35

1 Answer 1

1

You can use COPY command to take data dump in CSV.

Server Side Export:

copy table_name to 'Absolute/path/to/filename.csv' csv header;

Client Side Export:

\copy table_name to 'Relative/path/to/filename.csv' csv header;
5
  • This does not work with Amazon RDS because there is no access to the filesystem. Aug 4, 2022 at 22:34
  • Oh, but how are you working with pg_dump utility if you don't have OS access. please confirm if you have psql utility access. Aug 5, 2022 at 5:42
  • 1
    psql and pg_dump run externally and connect to the database via an SQL login. They can write to the 'local' filesystem. However, the COPY command executes on the server. Aug 5, 2022 at 5:50
  • Right , There is a difference between COPY and \COPY. \COPY writes on client machine. and it can be used via psql with -c parameter. hope this helps you. Aug 5, 2022 at 6:22
  • Ah, nice! Let's hope that helps them. Aug 5, 2022 at 9:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.