19

I have few tables in hive that has same prefix like below..

temp_table_name
temp_table_add
temp_table_area

There are few hundreds of tables like this in my database along with many other tables. I want to delete tables that starts with "temp_table". Do any of you know any query that can do this work in Hive?

8 Answers 8

22

There is no such thing as regular expressions for drop query in hive (or i didn't find them). But there are multipe ways to do it, for example :

  • With a shell script :

    hive -e "show tables 'temp_*'" | xargs -I '{}' hive -e 'drop table {}'
    
  • Or by putting your tables in a specific database and dropping the whole database.

    Create table temp.table_name;
    
    Drop database temp cascade;
    
3
  • 1
    hive -e ‘show tables 'temp_*’ | xargs -I ‘{}’ hive -e ‘drop table {}’ is not working for me Jun 26, 2016 at 17:02
  • 2
    The answer is perfect however there is a small change that is needed which actually worked for me. I am modifying @Louxou 's code a bit by adding double quotes. Here is the updated code. hive -e "show tables 'temp_*'" | xargs -I '{}' hive -e 'drop table {}' Sep 26, 2017 at 6:34
  • i want to add something to this. The problem now is that hive ( or beeline ) tends to show the results in "graphical" format. Also if you have multiple DB in hive you need to specify the one you need beeline --showHeader=false --outputformat=csv2 -e "use <db>; show tables 'temp_*'" | xargs -I '{}' beeline -e 'use<db>; drop table {}'
    – habarnam
    Jan 13, 2022 at 7:00
9

Above solutions are good. But if you have more tables to delete, then running 'hive -e drop table' is slow. So, I used this:

hive -e 'use db;show tables' | grep pattern > file.hql

use vim editor to open file.hql and run below commands

:%s!^!drop table  
:%s!$!;

then run

hive -f file.hql

This approach will be much faster.

3
  • Where do you mention the match table pattern here? Sep 25, 2017 at 13:24
  • 1
    @Alex Raj Kaliamoorthy - grep 'pattern'
    – Chandra
    Sep 26, 2017 at 20:43
  • Note that there is space after "drop table", like this: drop table .
    – Autonomous
    Oct 24, 2018 at 21:26
5

My solution has been to use bash script with the following cmd:

hive -e "SHOW TABLES IN db LIKE 'schema*';" | grep "schema" | sed -e 's/^/hive -e \"DROP TABLE db\./1' | sed -e 's/$/\"/1' > script.sh
chmod +x script.sh
./script.sh
4

I was able to delete all tables using following steps in Apache Spark with Scala:

val df = sql("SHOW TABLES IN default LIke 'invoice*'").select("tableName") // to  drop only selected column
val df = sql("SHOW TABLES IN default").select("tableName")
val tableNameList: List[String] = df.as[String].collect().toList
val df2 = tableNameList.map(tableName => sql(s"drop table ${tableName}"))
2

As I had a lot of tables do drop I used the following command, inspired in the @HorusH answer

hive -e "show tables 'table_prefix*'" | sed -e 's/^/ \DROP TABLE db_name\./1' | sed -e 's/$/;/1' > script.sh
hive -f script.sh
0

Try this:

hive -e 'use sample_db;show tables' | xargs -I '{}' hive -e 'use sample_db;drop table {}'

0

Below command will also work.

 hive -e 'show tables' | grep table_prefix |  while read line; do hive -e "drop table $line"; done
0

fastest solution through one shell script:

drop_tables.sh pattern

Shell script content:

hive -e 'use db;show tables' | grep $1 | sed 's/^/drop table db./' | sed 's/$/;/' > temp.hql
hive -f temp.hql
rm temp.hql

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.