229

I have a question about the ALTER TABLE command on a really large table (almost 30 millions rows). One of its columns is a varchar(255) and I would like to resize it to a varchar(40). Basically, I would like to change my column by running the following command:

ALTER TABLE mytable ALTER COLUMN mycolumn TYPE varchar(40);

I have no problem if the process is very long but it seems my table is no more readable during the ALTER TABLE command. Is there a smarter way? Maybe add a new column, copy values from the old column, drop the old column and finally rename the new one?

Note: I use PostgreSQL 9.0.

5
  • 17
    Just to be clear: You know, that resizing will not make the table occupy less space?
    – A.H.
    Oct 11, 2011 at 17:46
  • 1
    even in my case?I mean the column will have a max size of 40 char (so octets) instead of 255?
    – Labynocle
    Oct 17, 2011 at 13:37
  • 26
    If you say varchar(255) to PostgreSQL then it will not allocate 255 bytes for a value which real length is 40 bytes. It will allocate 40 bytes (plus some internal overhead). The only thing which will be changed by the ALTER TABLE` is the maximum number of bytes you can store in that column without getting an error from PG.
    – A.H.
    Oct 17, 2011 at 13:54
  • 1
    About the overhead A.H. mentioned: What is the overhead for varchar(n)? Mar 18, 2016 at 2:30
  • 2
    Check out the answer here for an update dba.stackexchange.com/questions/189890/… Jan 24, 2018 at 17:58

9 Answers 9

178

In PostgreSQL 9.1 there is an easier way

http://www.postgresql.org/message-id/[email protected]

CREATE TABLE foog(a varchar(10));

ALTER TABLE foog ALTER COLUMN a TYPE varchar(30);

postgres=# \d foog

 Table "public.foog"
 Column |         Type          | Modifiers
--------+-----------------------+-----------
 a      | character varying(30) |
3
  • 9
    Note that it works only because you're specifying a bigger size (30 > 10). If the size is smaller, you'll get the same error than I had.
    – Matthieu
    Nov 25, 2015 at 11:31
  • 6
    Postgres should not throw an error if you lower the varchar size via an ALTER TABLE query unless one of more rows contains a value that exceeds the new size.
    – Tell
    Jun 22, 2018 at 9:55
  • 1
    @Tell, interesting. Does that mean Postgres does a full scan of the table, or somehow keeps the max size in its statistics?
    – Matthieu
    Sep 11, 2018 at 8:35
92

There's a description of how to do this at Resize a column in a PostgreSQL table without changing data. You have to hack the database catalog data. The only way to do this officially is with ALTER TABLE, and as you've noted that change will lock and rewrite the entire table while it's running.

Make sure you read the Character Types section of the docs before changing this. All sorts of weird cases to be aware of here. The length check is done when values are stored into the rows. If you hack a lower limit in there, that will not reduce the size of existing values at all. You would be wise to do a scan over the whole table looking for rows where the length of the field is >40 characters after making the change. You'll need to figure out how to truncate those manually--so you're back some locks just on oversize ones--because if someone tries to update anything on that row it's going to reject it as too big now, at the point it goes to store the new version of the row. Hilarity ensues for the user.

VARCHAR is a terrible type that exists in PostgreSQL only to comply with its associated terrible part of the SQL standard. If you don't care about multi-database compatibility, consider storing your data as TEXT and add a constraint to limits its length. Constraints you can change around without this table lock/rewrite problem, and they can do more integrity checking than just the weak length check.

4
  • 1
    Thank you for the answer. I will check your link. I'm not worry about the manual size check because all my content has a max size of 40 chars. I need to read more about constraint on TEXT because I believed that VARCHAR was better to check lentgh :)
    – Labynocle
    Oct 17, 2011 at 13:46
  • 9
    Change varchar length does not rewrite the table. It just check the constraint length against the entire table exactly as CHECK CONSTRAINT. If you increase length there is nothing to do, just next insert or updates will accept bigger length. If you decrease length and all rows pass the new smaller constraint, Pg doesn't take any further action besides to allow next inserts or updates to write just the new length.
    – Maniero
    Sep 22, 2013 at 0:30
  • 3
    @bigown, just to clarify, your statement is only true for PostgreSQL 9.2+, not the old ones.
    – MatheusOl
    Nov 7, 2013 at 12:03
  • It should be noted that this command requires higher privilege than ALTER TABLE, so by the rule of least privilege, this isn't the best solution.
    – anishtain4
    Apr 19, 2022 at 13:38
63

Ok, I'm probably late to the party, BUT...

THERE'S NO NEED TO RESIZE THE COLUMN IN YOUR CASE!

Postgres, unlike some other databases, is smart enough to only use just enough space to fit the string (even using compression for longer strings), so even if your column is declared as VARCHAR(255) - if you store 40-character strings in the column, the space usage will be 40 bytes + 1 byte of overhead.

The storage requirement for a short string (up to 126 bytes) is 1 byte plus the actual string, which includes the space padding in the case of character. Longer strings have 4 bytes of overhead instead of 1. Long strings are compressed by the system automatically, so the physical requirement on disk might be less. Very long values are also stored in background tables so that they do not interfere with rapid access to shorter column values.

(http://www.postgresql.org/docs/9.0/interactive/datatype-character.html)

The size specification in VARCHAR is only used to check the size of the values which are inserted, it does not affect the disk layout. In fact, VARCHAR and TEXT fields are stored in the same way in Postgres.

5
  • 10
    Never too late to add more info about the "why"! Thank you for all this information
    – Labynocle
    Aug 8, 2013 at 7:26
  • Sometime you need to be consistent in the structure of your database. Even if 2 columns don't have a relation, they can have a relation in the point of view of concept, for example checkout the model EAV.
    – Alexandre
    Feb 11, 2014 at 12:55
  • Yeah, but you are over computing transactions. Think for a moment, you declare a column with a dimm. of 1024, and you store 10K, rows with variable amount of dimms. you are letting the RDBMS (unt his case postgres) compute the "best" size for your columns.
    – Franco Gil
    May 21, 2021 at 21:44
  • @FrancoGil: Firstly, I can't imagine the overhead of "computing the best size" being anything but absolutely negligible. Secondly, Postgres will do that regardless of how you define the column, the on-disk format will be the same for VARCHAR(40) and VARCHAR(1024) if you store the same data.
    – Sergey
    May 23, 2021 at 21:43
  • You just made me win precious time on my PR! THX for this answer Oct 3 at 13:23
59

I was facing the same problem trying to truncate a VARCHAR from 32 to 8 and getting the ERROR: value too long for type character varying(8). I want to stay as close to SQL as possible because I'm using a self-made JPA-like structure that we might have to switch to different DBMS according to customer's choices (PostgreSQL being the default one). Hence, I don't want to use the trick of altering System tables.

I ended using the USING statement in the ALTER TABLE:

ALTER TABLE "MY_TABLE" ALTER COLUMN "MyColumn" TYPE varchar(8)
USING substr("MyColumn", 1, 8)

As @raylu noted, ALTER acquires an exclusive lock on the table so all other operations will be delayed until it completes.

1
  • 3
    the ALTER acquires an exclusive lock on the table and prevents all other operations
    – raylu
    Sep 21, 2018 at 23:02
12

if you put the alter into a transaction the table should not be locked:

BEGIN;
  ALTER TABLE "public"."mytable" ALTER COLUMN "mycolumn" TYPE varchar(40);
COMMIT;

this worked for me blazing fast, few seconds on a table with more than 400k rows.

5
  • 5
    Why would you expect the explicit transaction wrapper to change locking behavior of the ALTER statement? It doesn't. Oct 23, 2017 at 12:26
  • try yourself, with and without the transaction wrapper, you will notice a huge difference.
    – jacktrade
    Oct 23, 2017 at 12:57
  • 3
    Your answer is incorrect on principal. Any DDL statement without explicit transaction wrapper runs inside a transaction implicitly. The only possible effect of the explicit transaction is that locks are kept longer - until the explicit COMMIT. The wrapper only makes sense if you want to put more commands into the same transaction. Oct 23, 2017 at 13:08
  • you are completely right, but I insist: try yourself, go on. and then ask why is not working in the same way.
    – jacktrade
    Oct 23, 2017 at 14:07
  • Didn't help on Postgres 9.3.
    – Noumenon
    Jul 3, 2018 at 23:22
10

Adding new column and replacing new one with old worked for me, on redshift postgresql, refer this link for more details https://gist.github.com/mmasashi/7107430

BEGIN;
LOCK users;
ALTER TABLE users ADD COLUMN name_new varchar(512) DEFAULT NULL;
UPDATE users SET name_new = name;
ALTER TABLE users DROP name;
ALTER TABLE users RENAME name_new TO name;
END;
7

Here's the cache of the page described by Greg Smith. In case that dies as well, the alter statement looks like this:

UPDATE pg_attribute SET atttypmod = 35+4
WHERE attrelid = 'TABLE1'::regclass
AND attname = 'COL1';

Where your table is TABLE1, the column is COL1 and you want to set it to 35 characters (the +4 is needed for legacy purposes according to the link, possibly the overhead referred to by A.H. in the comments).

5

Try run following alter table:

ALTER TABLE public.users 
ALTER COLUMN "password" TYPE varchar(300) 
USING "password"::varchar;
2

I have found a very easy way to change the size i.e. the annotation @Size(min = 1, max = 50) which is part of "import javax.validation.constraints" i.e. "import javax.validation.constraints.Size;"

@Size(min = 1, max = 50)
private String country;


when executing  this is hibernate you get in pgAdmin III 


CREATE TABLE address
(
.....
  country character varying(50),

.....

)
1
  • Thanks for your post! Please do not use signatures/taglines in your posts. Your user box counts as your signature, and you can use your profile to post any information about yourself you like. FAQ on signatures/taglines Feb 24, 2013 at 9:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.