0

I have a list of rowkeys and I would like to fetch specific columns for each rowkey. The only way I found is to use get command in a loop. Is there a better way to do it?

I will describe the exact usecase:

I have a table with about 50 columns per row.

I would like to fetch 20 columns for each row that has at least one column that was updated in a given time range.

The way I implemented it:

  • scan the table with time range and the list of columns
  • got all the row keys out of the response (that contains rawkey and the columns that were updated in the time range)
  • loop that creates get requests for each row key with list of columns
  • execute the batch request.
  • dump the full data (20 columns per row) to a file
  • I am looking for a more elegant and efficient way to do it.

    Thanks

    2
    • What's the search criteria?
      – Tariq
      Nov 19, 2013 at 19:51
    • I have added more exact description of the problem. Nov 21, 2013 at 9:17

    2 Answers 2

    0

    HBase will allow you to batch the Get operations, such that only a single network request is necessary to get the results for multiple rows.

    Note that HTable.get takes a List of Get objects, so just build your list of Gets and then invoke HTable.get a single time:

    List<Get> gets = new ArrayList<Get>();
    for (/* all rowkeys */) {
        Get g = new Get(row);
        g.addColumn(col1);
        g.addColumn(col2); 
        // ...
       gets.add(g);  
     }
    
    Result[] r = htable.get(gets);
    
    1
    • This is what i meant when I wrote to use get in a loop, thanks Nov 20, 2013 at 10:17
    0

    I feel you need something like DependentColumnFilter. DependentColumnFilter allows us specify a reference column that controls how other columns are filtered. It uses the timestamp of the reference column and includes all other columns that have the same timestamp.

    Constructors :

    DependentColumnFilter(byte[] family, byte[] qualifier)
    DependentColumnFilter(byte[] family, byte[] qualifier, boolean dropDependentColumn)
    DependentColumnFilter(byte[] family, byte[] qualifier, boolean dropDependentColumn, CompareOp valueCompareOp, WritableByteArrayComparable valueComparator)
    

    Further, based on the values you can select other columns.

    There is a catch though. This filter is not compatible with the batch feature of the scan operation.

    HTH

    Your Answer

    By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

    Not the answer you're looking for? Browse other questions tagged or ask your own question.