5

i am trying to delete a null/HIVE_DEFAULT_PARTITION in from hive external table and also from HDFS directory but i couldn't delete it .. i have been to some other post regarding the same issue and i tried those even tho it is not showing any error the partition is not getting deleted.

process_date=20160208
process_date=__HIVE_DEFAULT_PARTITION__

these are the partitions in my table now. and i just want to delete the hive default partition. (this is external table so i need to delete the same from HDFS too)

i tried this

ALTER TABLE Table_Name DROP IF EXISTS PARTITION(process_date='__HIVE_DEFAULT_PARTITION__');

i am getting below error

Error: Error while compiling statement: FAILED: SemanticException Unexpected unknown partitions for (process_date = null) (state=42000,code=40000)

i tried the below

ALTER TABLE table_name DROP PARTITION(process_date<'1');

no error but hive default partition is still there.

Please help me with the problem.

7
  • You could try deleting the hdfs folder directly using hadoop fs -rm <> and then in hive use the msck repair table to refresh the metastore.
    – K S Nidhin
    Feb 25, 2016 at 20:12
  • i agree it works that way but the problem is still if i want to automate that process what should i do ? real time when i run the ETL load and something fails and i need to remove the partition then what is the best way ?
    – sarath
    Feb 25, 2016 at 20:42
  • in prod scenario. The default hive partition or NULL comes when your data has the partition value NULL , which i suppose is not desired.
    – K S Nidhin
    Feb 25, 2016 at 20:44
  • some cases i am getting partiton value null even tho there is some partition exists. the problem happens because using data is loading to hive tables using ETL tool like informatica.. so if something happens then i am getting this problem. for refreshing view i am taking the latest partition only. so in this case when i get hive default partition i cant refresh view because its pointing to the latest partition. i have to delete that....... and i checked the HDFS path now and hive default partition is not showing there.. the other partition i can see in HSFS path..
    – sarath
    Feb 25, 2016 at 20:48
  • If you have null data as partition value , then the only way is to delete it from hdfs itself.Hive wont be able to recognize null partitions sometimes.
    – K S Nidhin
    Feb 25, 2016 at 20:51

2 Answers 2

3

The way to do this is alter the type of the partition column to string,

Then this syntax will work:

ALTER TABLE Table_Name DROP IF EXISTS PARTITION(process_date='__HIVE_DEFAULT_PARTITION__')

Then alter the type of the partition column back to it's original.

Then See

https://www.ericlin.me/2015/07/how-to-drop-hives-default-partition-hive_default_partition-with-int-partition-column/

0

This answer is to help others who might be facing same/similar issue.

The below query should help when you face the error - SemanticException Unexpected unknown partitions for (part_column = null)

Use '!=' (Not Equal to) and give supported value/values

ALTER TABLE <table_name> DROP IF EXISTS PARTITION(report_year!='2018');

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.