8

I'm looking for an option to remove a column from my BigQuery table like ALTER TABLE TABLE_NAME DROP COLUMN_NAME - but all I found in online was, drop the old table and create a new one.

I'm just wondering, Is there any logical reason not having this option in BigQuery?

3 Answers 3

9

Dropping a column would mean removing the data from all of the Capacitor files that make up the table, which is an expensive operation. If BigQuery were simply to remove metadata related to the column, you would be charged storage for a phantom column that you can't actually query, which wouldn't be ideal.

When you add a column, conversely, BigQuery treats the missing column in past files as having all NULL values, which doesn't require modifying them.

There are a couple of different options to remove a column:

  • Select from the original table, excluding the column that you don't want to keep. Then copy the resulting table and overwrite the original one.
  • Create a logical view over the table with the columns that you want. Now query the logical view instead of the table--you can make other "modifications" as well such as casting or filtering without having to touch the underlying table. The downside is that you will be charged for the old column that you don't need to query any more, however.
1
  • Thank you. This always comes as a surprise for a database using a columnar storage model, which on the face of it would make dropping columns easier!
    – nsandersen
    Jul 7, 2020 at 6:34
4

You could remove column via re-writing table.

CREATE OR REPLACE TABLE
  temp.table_name AS
SELECT
  * EXCEPT (column_name)
FROM
  temp.table_name
1
  • Running this command on partitioned table gives an error: Cannot replace a table with a different partitioning spec. Instead, DROP the table, and then recreate it. New partitioning spec is none and existing spec is interval(type:day,field:activityDate)
    – Avishay28
    Feb 16, 2020 at 17:26
3

BigQuery now supports DROP COLUMN

ALTER TABLE [[project_name.]dataset_name.]table_name
DROP COLUMN [IF EXISTS] column_name [, ...]
2

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.