Questions tagged [hbase]
HBase is the Hadoop database (columnar). Use it when you need random, real time read/write access to your Big Data. This project's goal is the hosting of very large tables -- billions of rows X millions of columns -- atop clusters of commodity hardware.
6,921
questions
0
votes
0
answers
15
views
Spark3 Hbase connector issue (java.lang.NullPointerException at org.apache.hadoop.hbase.spark.HBaseRelation)
" spark-shell --jars
"/tmp/hbase-spark3-1.0.0.7.2.14.0-149.jar,/tmp/hbase-protocol-shaded-2.1.0.jar,/tmp/hbase-client-2.1.10.jar,/tmp/hbase-shaded-miscellaneous-2.1.0.jar,/tmp/hbase-...
0
votes
2
answers
36
views
hbase using google cloud storage (bucket)
I'm new to hbase.
New to explore few options.
Can I use gcs file system as backing store for hbase on google cloud?
I know there's a google's big table, assuming setting up our own cluster with gcs ...
0
votes
0
answers
23
views
I get some exception when I run a HBase program with Java API
I got an exception when I run my Hbase program with Java, but I don't know how to resolve it.
Hadoop and Hbase are running on a virtual machine (VMware). I run my HBase client program on my physical ...
0
votes
0
answers
56
views
Problem to read data from HBase on AWS EMR cluster using Java Spring boot client
I'm trying to write a simple API application to read data from HBase on an AWS EMR cluster. But I get an UnknownHostException when I try to send the request.
However, in the console log you can see, ...
0
votes
1
answer
43
views
Hbase | Hbase col qualifier hidden using Hbase shell cmds but visible via hbaserdd spark code
I am stuck in a very odd situation related to Hbase design i would say.
Hbase version >> Version 2.1.0-cdh6.2.1
So, the problem statement is, in Hbase, we have a row in our table.
We perform ...
1
vote
0
answers
15
views
Receive a Kafka message through Spark Streaming and delete Phoenix/HBase data
In my project, I have the current workflow:
Kafka message => Spark Streaming/processing => Insert/Update to HBase and/or Phoenix
Both the Insert and Update operation works with HBase directly or ...
0
votes
1
answer
17
views
How to pass hbase-site.xml to Google Cloud Dataflow template
We have a setup where we have a Hbase cluster running on Google cloud and using Dataflow I want to write into Hbase tables. For this, I want to pass my hbase-site.xml file in staging and then in prod, ...
0
votes
1
answer
13
views
How to resolve "Failed to load class "org.slf4j.impl.StaticLoggerBinder"" while start hbase
I try to start my hbase cluster by command start-hbase.sh. I got follow errors:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/michael/opt/modules/hbase/...
0
votes
1
answer
22
views
SQLException: No suitable driver found for jdbc:phoenix:(hosts)
I am running a spark-submit command that will do some database work via a Scala class.
spark-submit
--verbose
--class mycompany.MyClass
--conf spark.driver.extraJavaOptions=-Dconfig.resource=dev-...
0
votes
0
answers
10
views
HBase query using filter and sum
I am new to Apache Hbase.
I want to run the below query on the table having fields "Country" and "Infections"
How many times infections are reported in India (number of rows)?
Here ...
0
votes
0
answers
21
views
Bad return type in ProtobufUtil.toScan(scan) using Java, spark and hbase
I have the following error "bad return type" when doing this assignment
ClientProtos.Scan proto = ProtobufUtil.toScan(scan);
I use these packages
Scan hbase-client-2.4.11.jar
ProtobufUtil ...
0
votes
1
answer
18
views
Regions In Transition в java application
Please tell me what's wrong, I'm new to working with hbase. when creating regions in a java application for hbase, an error occurs below then.
Caused by: org.apache.hadoop.hbase.ipc....
0
votes
1
answer
37
views
Zookeeper java.lang.NoClassDefFoundError: org/eclipse/jetty/server/Connector
I am getting the below error when starting Apache HBase with Zookeeper on my mac. I am using openjdk11.
************* WARN [main] admin.AdminServerFactory: Unable to load jetty, not starting ...
-1
votes
0
answers
28
views
Is there an elegant, easy and fast way to move data out of HBase into MongoDB?
Is there an elegant, easy and fast way to move data out of HBase into MongoDB? I want to migrate HBase to mongoDB. I am new to mongoDB. Could someone please help me out on the same?
Is it possible to ...
1
vote
0
answers
23
views
Map reduce and Hbase on Dataproc
I am facing below error while executing Dataproc hadoop job. WIth some debugging I found that code is unable to get the HBASE Table.
Error: java.io.IOException: Cannot create a record reader because ...
0
votes
0
answers
13
views
How to map HBase native timestamp to phoenix non-pk column
The phoenix provides row_timestamp to map hbase native timestamp to phoenix column, however the column should be primary key. I want to handle the data which out of order, but it can't.
For example:
I ...
0
votes
1
answer
31
views
jps does not show hmaster but <no information available>
I configured HBase today and I configured it correctly at first. However, when I ran HBase use the code 'start-all.sh' again, I could not see 'Hmaster' anywhere. It just shows like:
[root@master bin]# ...
0
votes
0
answers
19
views
How to solve Hbase overlap regions
Hbase version - 1.2.6
Hadoop version -2.7.1
I'm trying to merge the two hbase regions with the same start key and same end key. When I checked the hdfs storage for those two regions one region has ...
0
votes
2
answers
24
views
Hbase restore table from snapshot after the table is truncated
I executed below statement in order :
hbase> snapshot 'table1', 'snap_table1'
hbase> disable 'table1'
hbase> truncate 'table1'
hbase> restore_snapshot 'snap_table1'
hbase> scan 'table1'...
0
votes
0
answers
40
views
Unable to establish connection. using DBeaver apache phoenix to connect to Apache Hbase database
I have a standalone Apache HBase Database installed on an ubuntu vm,
i want to be able to connect to the db from the outside using dbeaver (Apache Phoenix)
the Master is up and i am able to use the ...
0
votes
0
answers
16
views
Copy HBase table from one version to another
I have HBase table called "t" that is composed of 2 column families:
"BL" with one qualifier "BLQ"
"R" with multiple qualifiers "Q", "X", &...
0
votes
0
answers
33
views
How do I get the number of columns of a table in hbase with java
How can I get the number of columns of a table in HBASE using java
This is my code in java
HBASE Version
hbase(main):002:0> -version
2.0.0.3.0.1.0-187, re9fcf450949102de5069b257a6dee469b8f5aab3, ...
0
votes
1
answer
15
views
How to test Hbase Get using Mocha/Node.js
Hi guys I have a job that has a method that extracts data from HBase given a key in Node.js, something like follows:
findHbaseData(field1, field2) {
const key = generateKey(field1, field2);
const ...
0
votes
0
answers
10
views
In HBase2.1.0, Get query is slow when HFile is too large (about 100GB)
HBase2.1.0
After Major Compaction, querying a record is too slow by Get query when single hfile is too large(about 100GB).
Target table has more than 30 regions,and total size is about 4TB.
Reducing ...
1
vote
1
answer
34
views
Why aggregations are not possible in HBase?
I am wondering why aggregation operations such as SUM / AVG are not natively supported by HBase. I got to know that there are several workarounds for achieving the same, but while there are these ...
0
votes
0
answers
4
views
What will a HRegionServer do if the zookeeper node that connected to the HRegionServer goes down?
In HBase, when a HRegionServer starts up, it will create an ephemeral znode in the ZooKeeper cluster. When the HRegionServer crashes, the ephemeral znode will be deleted, and the HMaster can be ...
0
votes
0
answers
9
views
HBASE error: Could not determine the startup mode
i am using java version java 16.0.1, and hbase version 3.0.0-alpha-2
the problem is that when I try running HBASE in my command line using :
start-hbase.cmd
I get the following error message :
Error:...
0
votes
0
answers
12
views
Is there a way to only retrieve rows from happyBase python based on few column values?
b'29218fb4-fda7-47fb-a8df-689f67' {b'h:2ef25622-6167-441c-b0ad3@61638036045279': b'{"netid":"29218fb4-fda7-47fb-a8df","session_id":"~","scrollable":1,&...
0
votes
0
answers
16
views
How to do a bulk insert/ bulkload into Hbase through Glue
I am trying to do a bulk insert or bulk load into Hbase in EMR using Glue Scala (Spark 3.1). I got this using
table.put(List<Put>);
without a satisfatory performance. I tried to insert though ...
0
votes
0
answers
20
views
EMR Core nodes are not taking up map reduce jobs
I have a 2 node EMR (Version 4.6.0) Cluster (1 master (m4.large) , 1 core (r4.xlarge) ) with HBase installed. I'm using default EMR configurations. I want to export HBase tables using
hbase org.apache....
0
votes
1
answer
29
views
How to display timestamp while scanning hbase?
for key, data in table.scan():
print('Found: {}, {}'.format(key, data))
I have an HBase table that I need to scan and print the timestamp. I have written the above code but it only produces output ...
0
votes
1
answer
33
views
How to solve the "too many connection" problem in zookeeper when I want to query too many times in reduce stage?
Sorry for my stupid question and thank you in advance.
I need to replace the outputvalue in reduce stage(or map stage). However, it will case too many connection in zookeeper. I don't know how to deal ...
0
votes
0
answers
56
views
HBase Error: Could not find or load main class org.apache.hadoop.hbase.util.HBaseConfTool
I get the following two errors when I try to start hbase with start-hbase.sh:
Error: Could not find or load main class org.apache.hadoop.hbase.util.HBaseConfTool
Error: Could not find or load main ...
0
votes
0
answers
14
views
Why retrieving hbase region mapping from zookeeper is much faster on spark than from local jdbc
i want to understand why querying hbase from spark-job (cluster mode) is much faster than from local jdbc.
I know that data locality improve performance but my point is on region mapping retrieved ...
0
votes
0
answers
64
views
not able to write into hbase table from apache spark standalone cluster mode
Running spark cluster in standalone mode, have two workers
hbase configuration:
Configuration hbaseConfig = HBaseConfiguration.create();
hbaseConfig.set(HConstants.ZOOKEEPER_QUORUM, example.zookeekper....
0
votes
0
answers
11
views
While exporting the hbase snapshot will mappers use exiting hfile data for exporting or will they create duplicate data form these hfiles?
I have an HBASE table with 1TB of storage having 100 regions. I took a snapshot of the table. While I export this snapshot to remote HDFS cluster will the mappers create 1TB extra space in existing ...
0
votes
0
answers
42
views
Spring hbase-client : Zookeeper session created for each request
I'm doing multiple scans in HBase with a java/spring boot API. However I have a doubt about performances of the way hbase-client manages connections (I use hbase-client 1.3.0).
Here is the code I use ...
0
votes
0
answers
30
views
Spark read HFile without
We have some tables present on hbase (in TB's) which we have to migrate.
However, Hbase is fully utilize and we cannot run Export as it put too much pressure on hbase. As hbase use , HFile as its data....
0
votes
0
answers
46
views
Java compiler cannot find hadoop.hbase.mapreduce packages
I'm trying to run a MapReduce app which uses a HBase table. The code is provided by my university professor and we are supposed to run it. Here is the code:
package hbase;
import java.io.IOException;
...
0
votes
0
answers
10
views
Link for Hbase installtion in colab
I have installed and run hadoop in colab and execute python map reduce code.
I would like to install hadoop and work in colab. Can someone provide me the steps for installation
0
votes
0
answers
16
views
Hbase error when I init TableInputFormat : Can't get the locations
I try to use filnk.TableInputFormat<> to let the hbase data in, but the error was appeared at configure(): Can't get the locations. My env is docker(docker desktop) on windows11, and I can ...
0
votes
1
answer
154
views
HBase shell is starting but throwing error
I am trying to run the HBase shell. The shell get starts but when I put any command it gives me this error:
hbase:001:0> status
ERROR: org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: ...
0
votes
0
answers
60
views
Master is initializing error while creating table via hbase shell
hdoop@ubuntu:~/HBase/hbase-2.3.5-bin/hbase-2.3.5/bin$ hbase shell
/home/hdoop/hadoop-3.2.2/libexec/hadoop-functions.sh: line 2366: HADOOP_ORG.APACHE.HADOOP.HBASE.UTIL.GETJAVAPROPERTY_USER: bad ...
0
votes
0
answers
19
views
How to save inverted index RDD in HBase using Pyspark
I have created an inverted index RDD using pyspark which looks like this:
[('adage',
[('file:/notebooks/Data/shakespeare/tragedies/macbeth', 1),
('file:/notebooks/Data/shakespeare/histories/...
0
votes
0
answers
37
views
Hbase connections timing out after region server died
Whenever a region server dies/stops unexpectedly, our Hbase connections start timing out, and do not internally connect to the other alive region servers. I have to always restart our entire ...
-1
votes
1
answer
81
views
How to send a csv file from machine 1 to machine 3 using nifi in machine 2?
I have 3 machines for my scenario
Machine 1: has a .csv file
Machine 2: Nifi is installed and running
Machine 3: HDFS and Hbase is installed and running.
Now I have to send the .csv file from machine ...
0
votes
0
answers
21
views
setFilterIfMissing not working in hbase shell script
I have hbase shell query which works correctly in power shell using hbase shell:
scan 'table', {FILTER=>"SingleColumnFilter('ids','author_ids','=','regexstring:.*USER123.*',true,true)
I'm ...
0
votes
0
answers
37
views
Not able to upload full CSV file from hdfs to Hbase table
I am trying to upload csv file from HDFS to HBase, well i able to dump the csv file from hdfs to Hbase table but i am not able to get full csv file, CSV file has 53,292 but when i run count in my ...
0
votes
0
answers
26
views
Scan multiple versions from HBase with SHC
I have an HBase table with a versioned column and I need to be able to use the SELECT statement to have all versions retrieved as separate rows when accessing the table from pyspark via SHC.
I've seen ...
0
votes
1
answer
188
views
Spark 3.2.1 fetch HBase data not working with NewAPIHadoopRDD
Below is the sample code snippet that is used for data fetch from HBase. This worked fine with Spark 3.1.2. However after upgrading to Spark 3.2.1, it is not working i.e. returned RDD doesn't contain ...