Questions tagged [druid]

Druid is a column-oriented open-source distributed data store written in Java.

Filter by
Sorted by
Tagged with
0 votes
0 answers
7 views

How to load delimited file with apache druid

I like to ingest delimited file to druid. However, I cannot ingest anything because my data is not parsed. Can someone tell me what I did wrong. My tsv file is like 1, 0 2.8736, 8.29 7.10, 8.83 My ...
tayun 's user avatar
0 votes
0 answers
18 views

Adding an input format in Druid via extension

I wish to parse an overly complicated json format in Druid by adding a custom extension. I followed the instructions in the documentation to create a module and implemented InputFormat and ...
mohitmayank's user avatar
-2 votes
0 answers
33 views

Springboot 15.21 cannot be linked to SQL Server 2012, is it a version incompatibility?

Java1.8, data source is alibaba.druid, spring boot 1.521 cannot connect to sqlserver2012, but can connect to sql2008. Is this version incompatible? It is normal to connect to SQL Server 2012 using the ...
user23023045's user avatar
0 votes
1 answer
16 views

Druid overwriting data [closed]

I have a druid datasource with column as ts, date, score ts is the timestamp field, the data is structured in a way that there are rows with same ts value but different values in date and score, Druid ...
Shakun's user avatar
  • 59
0 votes
1 answer
16 views

Insert python list of dict into Apache druid using sql

I am following Druid SQL based ingestion API https://druid.apache.org/docs/latest/api-reference/sql-ingestion-api#submit-a-query In the reference python code the data is loaded from an external url ...
Shakun's user avatar
  • 59
0 votes
0 answers
8 views

problem creating file file in apache druid

I am using apache-druid-28.0.0 single server micro_quickstart. I submit the task spec with tutorial transform-index.json. however, I want to load data from /tmp/test directory instead of quickstart ...
tayun 's user avatar
0 votes
1 answer
16 views

I am trying to run Druid REPLACE Query but got following error in my java project

{"error":"Unknown exception","errorMessage":"Unexpected character ('_' (code 95)): was expecting comma to separate Object entries\n at [Source: (org.eclipse.jetty....
Rushikesh Wakchaure's user avatar
0 votes
0 answers
13 views

Flattenspec for nested json column in a csv file

I had a csv file in that there is a tag column for that column there is a nested json data, now I have to flatten those data and ingest it into druid data source. Basically tags data are got like ...
PM Ravi Teja's user avatar
0 votes
1 answer
48 views

Encountering HTTP/502 Error When Passing Emojis in Druid SQL Query via Scala Play Framework and JDBC

I'm facing an issue in my Scala Play Framework application that uses JDBC to send SQL queries to Druid. When I attempt to execute a Druid SQL query containing emojis (e.g., 🥰) using prepared ...
Jithin K J's user avatar
0 votes
1 answer
57 views

How to implement the DATE_PART function of Postgres in Druid sql?

I have a sql query that I use to do some modification on time series data. Here is my sql below (This is based on Postgres. I am just highlighting the date modification part) ## 6 days interval "...
Souvik Ray's user avatar
  • 2,939
0 votes
2 answers
50 views

Column names do not appear using druiddb

I am running Apache Druid datastore locally. I am loading data from a Kafka stream. On Druid, I can see the column names: And then using druiddb (https://github.com/betodealmeida/druid-dbapi), I am ...
ARAT's user avatar
  • 932
0 votes
2 answers
24 views

Druid - Failed to get storage slot due to error [Unable to pick a free slot, this should never happen]

Tasks are getting failed with the following error in Druid, The tasks are not even scheduled and middle manager is getting restarted. Failed to get storage slot due to error [Unable to pick a free ...
Optimus Prime's user avatar
0 votes
1 answer
31 views

How to query Apache Druid's __time with milliseconds precision (efficiently)?

This is my Apache Druid query that I ran in the Druid’s web GUI: SELECT * FROM my_table WHERE __time >= '2023-10-19T09:29:58.613Z' and __time <= '2023-10-19T09:30:13.613Z' and my_string_field = '...
Sergey Zakharov's user avatar
0 votes
0 answers
32 views

Command[zk] failed while trying to run Apache Druid

I am very very new to Druid. I want to install it on my MacOS machine. I followed the instruction on this page. When I run ./bin/start-druid, I get the output below: Arat-MBP-2:apache-druid-27.0.0 ...
ARAT's user avatar
  • 932
0 votes
0 answers
12 views

Hive materilize table- java.lang.NoClassDefFoundError: net/jpountz/lz4/LZ4Factory

I am trying to create a meterialized view for Druid storage handler from Hive. Steps: CREATE TABLE employee_trans2 ( `__time` TIMESTAMP, id int, name string, age int, gender string) STORED AS ORC ...
Basapuram Kumar's user avatar
0 votes
0 answers
3 views

How to access apache hive tables from Apache Druid.?

I am trying to access apache hive tables from apache Druid, as I dont see hive plugin/datasource from Druid router UI. Curious to know where shall we see hive tables from Druid Router UI.? accessing ...
Basapuram Kumar's user avatar
1 vote
1 answer
21 views

How to use both mysql & postgresql extensions in Druid?

Druid only allows one of mysql-metadata-storage or postgresql-metadata-storage in the extensions loadList. If we try to add both, it fails. But we have a use case where we are using mysql for metadata ...
Optimus Prime's user avatar
0 votes
1 answer
24 views

Druid SQL for nested object throwing StringDictionaryEncodedColumn error

This is my query SELECT JSON_VALUE("event.data", '$.favItems') as "fav.Items" FROM "db" Where event.data is a nested object with a variety of values and essentially ...
Dmns's user avatar
  • 11
0 votes
1 answer
168 views

spring boot:Failed to configure a DataSource: 'url' attribute is not specified and no embedded datasource could be configured

I am learning to use the spring boot framework to integrate MyBatis Plus and ShardingSphere to achieve MySQL master-slave read-write separation (one master and one slave)。 The master library is the ...
千葉幻凑's user avatar
0 votes
1 answer
18 views

How to change Druid Historical service Tier

I have a configured Druid cluster with 6 historical services. All services belong to the default tier. I'm interested in the simplest way to make a change so that 2 of the Historicals are in the new (...
Nenad Vujasinovic's user avatar
0 votes
1 answer
45 views

Handling updates in Apache Druid for data from a commerce system

I have a digital commerce system that we have built in-house and it has been working fine. We now want to implement OLAP to provide our customers with advanced reporting and BI features. While looking ...
brahmana's user avatar
  • 1,306
0 votes
0 answers
45 views

How to set value of an attribute as value of 'Maximum Rate' property in Control rate processor in nifi

I have a flow where I am using control rate processor to control the data flow, I will submit tasks in druid using those files, for example let's say I have a total of 10 files. I will first ask my ...
Ram's user avatar
  • 35
0 votes
0 answers
23 views

How to trigger Invokehttp processor only when flow files pass to downstream?

I have a workflow where I will be getting some number of flowfiles in a day, the flow files will be invoking a druid API to submit a task. I want to limit the flowfile rate that passes my InvokeHttp ...
Ram's user avatar
  • 35
1 vote
0 answers
18 views

Duid Kafka ingestion via Docker Compose : "No data returned from sampler" and "Bad Gateway" [duplicate]

I am new to druid and kafka. I want a druid database to ingest a kafka topic. Druid containers are running from a docker compose file (the one found on the doc), the kafka container is running from ...
Jeremy Tarrieu's user avatar
0 votes
0 answers
69 views

Druid Deployment with Minio and PostgreSQL in Minikube

I'm trying to deploy Druid in Minikube with Minio and PostgrSQL. I'm using Druid-Opearor and all druid-sds-cluster-brokers-0, druid-sds-cluster-coordinators-0,druid-sds-cluster-historicals-0, druid-...
Alexy Pulivelil's user avatar
0 votes
0 answers
60 views

Apache Druid occurs error code 405 with SQL insert

For the given data sample, I can ingest it using batch commit with an ingest spec, but I cannot insert it using SQL 'INSERT' statements, resulting in error code 405. // sample data {"time": &...
Xavier Z.'s user avatar
  • 179
0 votes
0 answers
49 views

unable to connect to the stream in Druid ingestion

unable to connect to the stream in Druid ingestion I am unable to ingest the data into druid due to above error when i check bootstrap servers and topic all are existing and they were connecting the ...
Ashok's user avatar
  • 1
0 votes
1 answer
22 views

Csv headers mapping issue in nifi

I had a csv file, and in one csv file I'm getting 10 headers and in another l'm getting 15 headers. Like this, I have a number of csv files, all of which are related, but the only thing is that the ...
PM Ravi Teja's user avatar
0 votes
0 answers
40 views

How to use tags of counter in Grafana?

I have such problem: i need to use tags of Counter Here Im adding tags to counter: Counter.Builder counterBuilder = Counter.builder("kafka_messages_total"); public void addMoreInfo(Map&...
Данил Матросов's user avatar
0 votes
0 answers
22 views

Druid Security Group Setup in AWS

I have deployed druid latest(v26.0) version in a Clustered mode in 2 AWS EC2 instances ( 1 server for Master, Query & ZK, another dedicated for Data Server). The master instance has the ports - ...
Santosh Kumar Sahoo's user avatar
0 votes
1 answer
57 views

How to get latest value from Native Druid query?

There are duplicate records in my Druid DB, when we fire Druid Native Query, we get more than one records (duplicate records), my queryType is scan. I need to see only one record/result through my ...
shamanth gowda's user avatar
0 votes
1 answer
55 views

Apache Druid on MicroK8S

How to deploy Apache Druid on MicroK8S on a Linux Ubuntu VM and to configure its available resources? Most of the repositories seem to be deprecated. I have tried to use helm and looked for the YAML ...
Hawke404's user avatar
1 vote
1 answer
122 views

Can dimensions in Apache Druid be considered as a form of primary key?

In other word, in Druid, if rollup is disabled, will metric values with the same timestamp and dimension(value) overwrite former metric value?
Xavier Z.'s user avatar
  • 179
0 votes
0 answers
259 views

Property 'sqlSessionFactory' or 'sqlSessionTemplate' are required

This error has been reported since the springboot3.0 upgrade. Please help me Error message: 2023/06/16 12:34:04.225 WARN [x-transaction-id: ] [main] org.springframework.boot.web.servlet.context....
user17453018's user avatar
0 votes
1 answer
57 views

Druid `Latest` Performance implications

Druid supports Latest aggregation in newer releases. For example, following table stores user coordinates on different time stamps. I want to know how is latest user coordinate fetched via this query ...
Mohit Jain's user avatar
1 vote
0 answers
98 views

An error occurred after the springboot3.0 upgrade. Procedure:Consider defining a bean of type javax sql DataSource in your configuration

After upgrading from springboot 2.7.9 to 3.0.0, I am unable to start. Below is my complete error message: Exception encountered during context initialization - cancelling refresh attempt: org....
user17453018's user avatar
0 votes
2 answers
150 views

How can I execute druid query using cURL command in a Python script?

I need to run the Druid query in a Python script using the cURL command. I am able to get the result in the terminal like the one below. I am a novice in Python. This is what I can do in the terminal. ...
Pankaj Kumar's user avatar
0 votes
0 answers
249 views

com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure\n

Error Message: [{"businessNo":"af7df42$1***$2a71be7a6","errMsg":"\n### Error querying database.  Cause: com.mysql.cj.jdbc.exceptions.The program often: ...
ChenYu's user avatar
  • 1
0 votes
0 answers
20 views

Compile druid processing error with Intellij Idea

When i follow the steps to build druid project locally, it throws error [ERROR] No compiler is provided in this environment. Perhaps you are running on a JRE rather than a JDK? while other components ...
AngryShark's user avatar
0 votes
0 answers
68 views

DRuid metadata is not having records and jobs in pending state

Druid Mysql metadata not showing segments and Jobs in Pending State I completed a druid cluster setup and metadata in mysql. All services are showing. BUt Segments is showing "0 segments". ...
Saberedo's user avatar
0 votes
0 answers
96 views

Druid Cannot Connect to mysql metadata database after reinitialization

I reconfigured my apache druid metadata store to mysql from derby.- But could not access the metadata database after. Druid web UI is showing: "Could not load SQL metadata". Other druid log ...
Saberedo's user avatar
1 vote
0 answers
85 views

Apache Druid throws "OSError: HTTP Error 400: Bad Request"

I encounter with an Error when I try to reach Apache Druid datasource. Here is the code sample I used to load Druid datasource into pandas dataframe: from pydruid.client import * from pydruid.utils....
ByUnal's user avatar
  • 100
0 votes
1 answer
65 views

Druid set maxAge in pac4j extension

I'm using the Druid Pac4j library for authentication for druid web console, which is deployed in kubernetes using terraform. Currently, I'm encountering an issue where the session token obtained ...
ragav ravi's user avatar
0 votes
0 answers
15 views

Druid implementation suggestions for Data analytics platform

I need to make some sort of analytical platform using druid. I would like to ask for community ideas on if druid would be a good fit and some sort of idea for the implementation of such. I have a ...
OpenSource's user avatar
0 votes
1 answer
174 views

Issue with Druid SQL and Curl

I making curl SQL queries against a druid broker and whenever I use a WHERE clause such as in curl -kv -H "Content-Type: application/json" -X POST http://localhost:8082/druid/v2/sql?pretty --...
Christian68's user avatar
0 votes
0 answers
33 views

Production use of Druid materialized view - is there known issues?

Materialized views in Druid are marked as experimental. What's the caveat using them? Is there any known issues?
supernova's user avatar
  • 1,840
0 votes
1 answer
117 views

How to scale Druid's batch ingestion job?

I've setup a Druid cluster on 3 different machines with a master, data and query server. I've scaled the machines to be 48 vCPU and 200GB, but I'm still running into this error when running a batch ...
Chiquelo's user avatar
  • 162
0 votes
1 answer
537 views

Load databases from PostgreSQL server to Apache Druid

I have a postgreSQL cluster that has a lot of databases inside it and I want to migrate all of these databases to Apache Druid, is there any one who can help me ? Thanks in advance I have searched a ...
hicham's user avatar
  • 1
0 votes
1 answer
224 views

Ingest null value for INT and LONG type column in Druid

I am ingesting some data from JSON to Druid. The input data has null value for some of the INT and LONG fields. I want to store them as null, instead of storing them as 0. I am not able to figure out ...
Rahul Kumar Gond's user avatar
0 votes
1 answer
318 views

transform epoch milliseconds in long to timestamp type while ingestion in Druid

My input data contains multiple fields which have epoch data in milliseconds in long datatype format. I want to transform format them into timestamp in Druid while ingestion. There are some time ...
Rahul Kumar Gond's user avatar

1
2 3 4 5
13