Questions tagged [druid]
Druid is a column-oriented open-source distributed data store written in Java.
614
questions
0
votes
0
answers
7
views
How to load delimited file with apache druid
I like to ingest delimited file to druid. However, I cannot ingest anything because my data is not parsed. Can someone tell me what I did wrong.
My tsv file is like
1, 0
2.8736, 8.29
7.10, 8.83
My ...
0
votes
0
answers
18
views
Adding an input format in Druid via extension
I wish to parse an overly complicated json format in Druid by adding a custom extension. I followed the instructions in the documentation to create a module and implemented InputFormat and ...
-2
votes
0
answers
33
views
Springboot 15.21 cannot be linked to SQL Server 2012, is it a version incompatibility?
Java1.8, data source is alibaba.druid, spring boot 1.521 cannot connect to sqlserver2012, but can connect to sql2008. Is this version incompatible? It is normal to connect to SQL Server 2012 using the ...
0
votes
1
answer
16
views
Druid overwriting data [closed]
I have a druid datasource with column as ts, date, score
ts is the timestamp field, the data is structured in a way that there are rows with same ts value but different values in date and score, Druid ...
0
votes
1
answer
16
views
Insert python list of dict into Apache druid using sql
I am following Druid SQL based ingestion API https://druid.apache.org/docs/latest/api-reference/sql-ingestion-api#submit-a-query
In the reference python code the data is loaded from an external url ...
0
votes
0
answers
8
views
problem creating file file in apache druid
I am using apache-druid-28.0.0 single server micro_quickstart.
I submit the task spec with tutorial transform-index.json. however, I want to load data from /tmp/test directory instead of quickstart ...
0
votes
1
answer
16
views
I am trying to run Druid REPLACE Query but got following error in my java project
{"error":"Unknown exception","errorMessage":"Unexpected character ('_' (code 95)): was expecting comma to separate Object entries\n at [Source: (org.eclipse.jetty....
0
votes
0
answers
13
views
Flattenspec for nested json column in a csv file
I had a csv file in that there is a tag column for that column there is a nested json data, now I have to flatten those data and ingest it into druid data source.
Basically tags data are got like ...
0
votes
1
answer
48
views
Encountering HTTP/502 Error When Passing Emojis in Druid SQL Query via Scala Play Framework and JDBC
I'm facing an issue in my Scala Play Framework application that uses JDBC to send SQL queries to Druid. When I attempt to execute a Druid SQL query containing emojis (e.g., 🥰) using prepared ...
0
votes
1
answer
57
views
How to implement the DATE_PART function of Postgres in Druid sql?
I have a sql query that I use to do some modification on time series data. Here is my sql below (This is based on Postgres. I am just highlighting the date modification part)
## 6 days interval
"...
0
votes
2
answers
50
views
Column names do not appear using druiddb
I am running Apache Druid datastore locally. I am loading data from a Kafka stream.
On Druid, I can see the column names:
And then using druiddb (https://github.com/betodealmeida/druid-dbapi), I am ...
0
votes
2
answers
24
views
Druid - Failed to get storage slot due to error [Unable to pick a free slot, this should never happen]
Tasks are getting failed with the following error in Druid,
The tasks are not even scheduled and middle manager is getting restarted.
Failed to get storage slot due to error [Unable to pick a free ...
0
votes
1
answer
31
views
How to query Apache Druid's __time with milliseconds precision (efficiently)?
This is my Apache Druid query that I ran in the Druid’s web GUI:
SELECT *
FROM my_table
WHERE __time >= '2023-10-19T09:29:58.613Z'
and __time <= '2023-10-19T09:30:13.613Z'
and my_string_field = '...
0
votes
0
answers
32
views
Command[zk] failed while trying to run Apache Druid
I am very very new to Druid. I want to install it on my MacOS machine. I followed the instruction on this page.
When I run ./bin/start-druid, I get the output below:
Arat-MBP-2:apache-druid-27.0.0 ...
0
votes
0
answers
12
views
Hive materilize table- java.lang.NoClassDefFoundError: net/jpountz/lz4/LZ4Factory
I am trying to create a meterialized view for Druid storage handler from Hive.
Steps:
CREATE TABLE employee_trans2 (
`__time` TIMESTAMP,
id int,
name string,
age int,
gender string)
STORED AS ORC
...
0
votes
0
answers
3
views
How to access apache hive tables from Apache Druid.?
I am trying to access apache hive tables from apache Druid, as I dont see hive plugin/datasource from Druid router UI.
Curious to know where shall we see hive tables from Druid Router UI.?
accessing ...
1
vote
1
answer
21
views
How to use both mysql & postgresql extensions in Druid?
Druid only allows one of mysql-metadata-storage or postgresql-metadata-storage in the extensions loadList. If we try to add both, it fails.
But we have a use case where we are using mysql for metadata ...
0
votes
1
answer
24
views
Druid SQL for nested object throwing StringDictionaryEncodedColumn error
This is my query
SELECT JSON_VALUE("event.data", '$.favItems') as "fav.Items" FROM "db"
Where event.data is a nested object with a variety of values and essentially ...
0
votes
1
answer
168
views
spring boot:Failed to configure a DataSource: 'url' attribute is not specified and no embedded datasource could be configured
I am learning to use the spring boot framework to integrate MyBatis Plus and ShardingSphere to achieve MySQL master-slave read-write separation (one master and one slave)。
The master library is the ...
0
votes
1
answer
18
views
How to change Druid Historical service Tier
I have a configured Druid cluster with 6 historical services. All services belong to the default tier. I'm interested in the simplest way to make a change so that 2 of the Historicals are in the new (...
0
votes
1
answer
45
views
Handling updates in Apache Druid for data from a commerce system
I have a digital commerce system that we have built in-house and it has been working fine. We now want to implement OLAP to provide our customers with advanced reporting and BI features.
While looking ...
0
votes
0
answers
45
views
How to set value of an attribute as value of 'Maximum Rate' property in Control rate processor in nifi
I have a flow where I am using control rate processor to control the data flow, I will submit tasks in druid using those files, for example let's say I have a total of 10 files. I will first ask my ...
0
votes
0
answers
23
views
How to trigger Invokehttp processor only when flow files pass to downstream?
I have a workflow where I will be getting some number of flowfiles in a day, the flow files will be invoking a druid API to submit a task. I want to limit the flowfile rate that passes my InvokeHttp ...
1
vote
0
answers
18
views
Duid Kafka ingestion via Docker Compose : "No data returned from sampler" and "Bad Gateway" [duplicate]
I am new to druid and kafka.
I want a druid database to ingest a kafka topic.
Druid containers are running from a docker compose file (the one found on the doc), the kafka container is running from ...
0
votes
0
answers
69
views
Druid Deployment with Minio and PostgreSQL in Minikube
I'm trying to deploy Druid in Minikube with Minio and PostgrSQL.
I'm using Druid-Opearor and all druid-sds-cluster-brokers-0, druid-sds-cluster-coordinators-0,druid-sds-cluster-historicals-0, druid-...
0
votes
0
answers
60
views
Apache Druid occurs error code 405 with SQL insert
For the given data sample, I can ingest it using batch commit with an ingest spec, but I cannot insert it using SQL 'INSERT' statements, resulting in error code 405.
// sample data
{"time": &...
0
votes
0
answers
49
views
unable to connect to the stream in Druid ingestion
unable to connect to the stream in Druid ingestion
I am unable to ingest the data into druid due to above error
when i check bootstrap servers and topic all are existing and they were connecting
the ...
0
votes
1
answer
22
views
Csv headers mapping issue in nifi
I had a csv file, and in one csv file I'm getting 10 headers and in another l'm getting 15 headers. Like this, I have a number of csv files, all of which are related, but the only thing is that the ...
0
votes
0
answers
40
views
How to use tags of counter in Grafana?
I have such problem: i need to use tags of Counter
Here Im adding tags to counter:
Counter.Builder counterBuilder = Counter.builder("kafka_messages_total");
public void addMoreInfo(Map&...
0
votes
0
answers
22
views
Druid Security Group Setup in AWS
I have deployed druid latest(v26.0) version in a Clustered mode in 2 AWS EC2 instances ( 1 server for Master, Query & ZK, another dedicated for Data Server).
The master instance has the ports - ...
0
votes
1
answer
57
views
How to get latest value from Native Druid query?
There are duplicate records in my Druid DB, when we fire Druid Native Query, we get more than one records (duplicate records), my queryType is scan.
I need to see only one record/result through my ...
0
votes
1
answer
55
views
Apache Druid on MicroK8S
How to deploy Apache Druid on MicroK8S on a Linux Ubuntu VM and to configure its available resources? Most of the repositories seem to be deprecated.
I have tried to use helm and looked for the YAML ...
1
vote
1
answer
122
views
Can dimensions in Apache Druid be considered as a form of primary key?
In other word, in Druid, if rollup is disabled, will metric values with the same timestamp and dimension(value) overwrite former metric value?
0
votes
0
answers
259
views
Property 'sqlSessionFactory' or 'sqlSessionTemplate' are required
This error has been reported since the springboot3.0 upgrade. Please help me
Error message:
2023/06/16 12:34:04.225 WARN [x-transaction-id: ] [main] org.springframework.boot.web.servlet.context....
0
votes
1
answer
57
views
Druid `Latest` Performance implications
Druid supports Latest aggregation in newer releases. For example, following table stores user coordinates on different time stamps. I want to know how is latest user coordinate fetched via this query ...
1
vote
0
answers
98
views
An error occurred after the springboot3.0 upgrade. Procedure:Consider defining a bean of type javax sql DataSource in your configuration
After upgrading from springboot 2.7.9 to 3.0.0, I am unable to start. Below is my complete error message:
Exception encountered during context initialization - cancelling refresh attempt: org....
0
votes
2
answers
150
views
How can I execute druid query using cURL command in a Python script?
I need to run the Druid query in a Python script using the cURL command. I am able to get the result in the terminal like the one below. I am a novice in Python.
This is what I can do in the terminal.
...
0
votes
0
answers
249
views
com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure\n
Error Message:
[{"businessNo":"af7df42$1***$2a71be7a6","errMsg":"\n### Error querying database. Cause: com.mysql.cj.jdbc.exceptions.The program often: ...
0
votes
0
answers
20
views
Compile druid processing error with Intellij Idea
When i follow the steps to build druid project locally, it throws error [ERROR] No compiler is provided in this environment. Perhaps you are running on a JRE rather than a JDK? while other components ...
0
votes
0
answers
68
views
DRuid metadata is not having records and jobs in pending state
Druid Mysql metadata not showing segments and Jobs in Pending State
I completed a druid cluster setup and metadata in mysql. All services are showing. BUt Segments is showing "0 segments". ...
0
votes
0
answers
96
views
Druid Cannot Connect to mysql metadata database after reinitialization
I reconfigured my apache druid metadata store to mysql from derby.- But could not access the metadata database after.
Druid web UI is showing:
"Could not load SQL metadata".
Other druid log ...
1
vote
0
answers
85
views
Apache Druid throws "OSError: HTTP Error 400: Bad Request"
I encounter with an Error when I try to reach Apache Druid datasource. Here is the code sample I used to load Druid datasource into pandas dataframe:
from pydruid.client import *
from pydruid.utils....
0
votes
1
answer
65
views
Druid set maxAge in pac4j extension
I'm using the Druid Pac4j library for authentication for druid web console, which is deployed in kubernetes using terraform. Currently, I'm encountering an issue where the session token obtained ...
0
votes
0
answers
15
views
Druid implementation suggestions for Data analytics platform
I need to make some sort of analytical platform using druid. I would like to ask for community ideas on if druid would be a good fit and some sort of idea for the implementation of such.
I have a ...
0
votes
1
answer
174
views
Issue with Druid SQL and Curl
I making curl SQL queries against a druid broker and whenever I use a WHERE clause such as in
curl -kv -H "Content-Type: application/json" -X POST http://localhost:8082/druid/v2/sql?pretty --...
0
votes
0
answers
33
views
Production use of Druid materialized view - is there known issues?
Materialized views in Druid are marked as experimental. What's the caveat using them? Is there any known issues?
0
votes
1
answer
117
views
How to scale Druid's batch ingestion job?
I've setup a Druid cluster on 3 different machines with a master, data and query server. I've scaled the machines to be 48 vCPU and 200GB, but I'm still running into this error when running a batch ...
0
votes
1
answer
537
views
Load databases from PostgreSQL server to Apache Druid
I have a postgreSQL cluster that has a lot of databases inside it and
I want to migrate all of these databases to Apache Druid, is there any one who can help me ?
Thanks in advance
I have searched a ...
0
votes
1
answer
224
views
Ingest null value for INT and LONG type column in Druid
I am ingesting some data from JSON to Druid. The input data has null value for some of the INT and LONG fields. I want to store them as null, instead of storing them as 0. I am not able to figure out ...
0
votes
1
answer
318
views
transform epoch milliseconds in long to timestamp type while ingestion in Druid
My input data contains multiple fields which have epoch data in milliseconds in long datatype format. I want to transform format them into timestamp in Druid while ingestion. There are some time ...