1

I am trying to put a generic script in Pig Latin where I need to filter data based on the last column.

My data-sets will have varying columns counts, where I will filter based on the last column.

Is there any way we can get the last column in Pig.

master = LOAD '/user/data/usps/dataset_1/' using PigStorage(',') ;
B = FILTER master BY last_column is null;

1 Answer 1

0

You can't by using pigstorage. But you can write your own custom loader (quite easy to be honest) to parse data and name the last column "last_column"

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.