Questions tagged [clickhouse]
ClickHouse is an open-source column-oriented DBMS for real time analytical reporting which has Capability to store and process petabytes of data.
1,857
questions
0
votes
0
answers
4
views
Getting Table is in readonly mode error when trying to drop a partition on Clickhouse Replicated Table
ALTER TABLE dbName.rTable DROP PARTITION 20220107
I want to run this command on all underlying replicated tables.
when I run this command I get table is in read only mode error.
rTable's are created ...
0
votes
0
answers
18
views
Sending struct and nested structed spark columns to ClickHouse through JDBC
I want to send my spark dataframe to clickhouse table using JDBC.
I have multiple struct columns in my dataframe, structs with struct, array of structs etc.
|-- guest_third_party_id_arr: array (...
0
votes
1
answer
21
views
Is there a function to generate alternative +1s and -1s for a particular column in a table in Clickhouse database?
A little bit of background: I have a customer-data-lake which is like a repository having both old and new versions of each customer. I simply have to pick the latest record, but the materialized view ...
0
votes
1
answer
22
views
running dbt with clickhouse cluster: DB::Exception: Syntax error: failed at position 201 ('ON') (line 5, col 5): ON CLUSTER
I am trying to run dbt on a clickhouse cluster
the first run works just fine and dbt creates tables on all nodes with no issue, the second run however is causing the following error:
DB::Exception: ...
-2
votes
0
answers
26
views
How clickhouse-driver cursor works under the hood? [closed]
I use cickhouse-driver in my applications. Want to use cursor from "DB API 2.0".
query = select * from system.numbers limit %(param)s
I came across the fact that when execute cursor.execute(...
0
votes
0
answers
19
views
Getting frequent `No active replica has part` errors after upgrading clickhouse
I am getting frequent No active replica has part errors in clickhouse v23.8.2.7. I checked system.replication_queue. There is no record which might be stuck due to this error. The errors are getting ...
1
vote
0
answers
9
views
How to set labels in Grafana with ClickHouse
I use Grafana + ClickHouse to monitor logs. A have a "Logs" widget made from clickhouse sql query. When I switch on "Unique labels" option I see that no unique labels in the ...
0
votes
1
answer
15
views
Virtual columns in Kafka engine
I want to get virtual columns, but when I write the example code I get an error.
I'm following the Kafka engine documentation in Clickhouse:
[kafka-engine](https://clickhouse.com/docs/en/engines/table-...
0
votes
0
answers
36
views
Losing data while inserting in clickhouse
I want to insert data from one table to another. I use no filters except of date so my query look as follows:
insert into table1
select *
from table2
where dt = 'yyy-mm-dd'
But I lose ...
0
votes
0
answers
21
views
How to manage different data types in clickhouse
I'm new to Clickhouse and for the moment everything goes well. But I'm facing right now with a problem. I have to create data from a python script. But the values have different types. For the moment ...
0
votes
1
answer
29
views
IP to ASN lookup using ClickHouse
I have a table consisting of IP/CIDR and AS Number and a regular sFlow data table.
Since the sFlow data does not have an ASN, I want to add ASN into sFlow data table using IP/CIDR-ASN table.
All IP ...
0
votes
1
answer
18
views
How to reduce the memory usage of a multiple ASOF JOIN queries?
I have one query that results in ~100 rows. Not really much. This table is called FillsTable.
Then I have mulitple tables SignalTable1,2,3,4,5,6 that have millions of rows.
I want to ASOF JOIN the ...
1
vote
0
answers
47
views
'HttpClient' object has no attribute 'execute' clickhouse python
I'm trying to execute a command with Clickhouse client in Python as I have seen it in the document but it keeps give me an error
'HttpClient' object has no attribute 'execute'
from clickhouse_driver ...
-1
votes
0
answers
35
views
Represent nested structure in Clickhouse for optimal query performance
I need to map an Active Directory structure in a Clickhouse database. I have ~1000 Active Directory groups in a nested fashion, ~10 levels deep nesting (group A contains group B .... 10 times). I ...
0
votes
2
answers
59
views
Clickhouse creating columns base on value of other rows
I'm trying to know how much record a user has generated during the last x hours from when the record is created. In classic SQL what I've tried so far should be working.
I'm posting here because I ...
0
votes
1
answer
39
views
How do I update partial columns in clickhouse
I have a large table in clickhouse, and after adding a column with "alter table add column ed_code String", I want to refresh the column data according to the primary key. What should I do? ...
0
votes
1
answer
14
views
How to write ClickHouse OVER statement in DolphinDB?
I have a query in ClickHouse with the following syntax:
avg(ask) over(partition by sym order by time asc rows between 13 preceding and 0 following)
How can I write a DolphinDB script that can achieve ...
-1
votes
2
answers
43
views
Can't import CSV to ClickHouse
I'm trying to import CSV file to ClickHouse table:
INSERT INTO default.tags FROM INFILE '/home/sergey/test/tags_202309081557.csv' FORMAT CSV;
but got strange error:
SQL Error [78] [07000]: Code: 78. ...
0
votes
1
answer
25
views
How to deal with counter resets in ClickHouse?
I try to use the ClickHouse to store time series data. Precisely, accumulated counters. They behave the same as in Prometheus. Regrettably, I have been unable to locate any examples demonstrating how ...
0
votes
0
answers
26
views
Clickhouse kafka integration tables cleaning
I use kafka integration mechanism to transfer data via chain (kafka engine)->(materialized view with TO)->(destination table)
materialized view reads any data block coming to kafka table , and ...
0
votes
1
answer
26
views
How to add column to nested column
I have a table with nested column
...
a UInt64
b Nested (b1 String, b2 String)
c LowCardinality(String),
...
I want to update the table to be
...
a UInt64
b Nested (b1 String, b2 String, **b3 String**...
0
votes
1
answer
45
views
Can not access ClickHouse remotely on Ubuntu 20.04 DB::NetException Timeout: connect timed out
OS: Ubuntu 20.04.6 LTS
ClickHouse: 23.7.2.25 (official build)
Server: Intel(R) Xeon(R) CPU E5-2695 v4 @ 2.10GHz
I can not connect from outside remotely. Error:
$ clickhouse-client --user fridary --...
-2
votes
0
answers
22
views
The ClickHouse log error refreshing, leading to the filling of the entire storage space
here is log detail It will endlessly loop and print errors until the storage space is exhausted.
{"log":"11. void std::__function::__policy_invoker\u003cvoid ()\u003e::__call_impl\...
0
votes
2
answers
60
views
Get all dates between two dates with id
I have a table with sales like this:
Select Id, startdate, endate
From tabe
Output:
1 '01-01-2022' '06-01-2022'
1 '02-01-2022' '05-01-2022'
1 '13-01-2022' '17-01-2022'
2 '01-01-2022' '02-01-2022'
2 ...
0
votes
0
answers
21
views
Clickhouse asynchronous_insert_log table not visible
asynchronous_insert_log table is not there under system db in clickhouse. My config.xml contains below configuration by default:
<clickhouse>
<!-- Other settings -->
<...
0
votes
2
answers
22
views
Clickhouse WITH FILL STEP toIntervalHour(1) exception
I'm trying to do an aggregation using ORDER by date WITH FILL STEP toIntervalHour(1) but ClickHouse returns me an exception:
Received exception from server (version 23.5.3):
Code: 475. DB::Exception: ...
0
votes
0
answers
32
views
The clickhouse cluster cannot create any tables
Code: 107. DB::Exception: There was an error on [ck1:9000]: Code: 107. DB::ErrnoException: Cannot open file /data/clickhouse/store/76b/76b93482-a138-42fc-9969-e608641273be/zxc_sig.sql.tmp, errno: 2, ...
0
votes
1
answer
62
views
How to speed up Clickhouse query
I have a Clickhouse table defined as below:
CREATE TABLE default.my_table
(
`timestamp` DateTime CODEC(Delta(4), ZSTD(1)),
`service` LowCardinality(String) CODEC(ZSTD(1)),
`operation` ...
0
votes
2
answers
59
views
How do I add a primary key to a column, an existing data table?
What is the request to add the primary key to a certain column, an existing MergeTree data table?
I tried to change it with a standard query, but it didn't give any results:
ALTER TABLE table_name ADD ...
0
votes
1
answer
39
views
Insert data to table if subquery exists
What I want to do is check if a subquery exists, if exist then insert some data into a table, like:
if exists (select * from tableB)
update tableA set status=1 where id = 1001;
I cannot update the ...
1
vote
1
answer
96
views
How to integrate Rust UDF using C in Clickhouse?
Have a UDF in C
main.c
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
extern char *u_shaped_processor(char* camps_list_c);
void replacechar(char *s,char c1,char c2) {
...
0
votes
0
answers
44
views
Clickhouse aggregate page view user flow by sessions
The clickhouse database table looks somewhat like this:
| Timestamp | SessionId | PagePath | ReferrerPath | ... |
My desired outcome would be something along these lines in order to visualize the page ...
0
votes
2
answers
43
views
How to choose parameters for Buffer table in Clickhouse?
I want to create a buffer table to reduce the number of small inserts, but I don't understand how to select parameters for the table?
For example, I want the buffer table to flash the data to the ...
0
votes
2
answers
52
views
clickhouse extract several values by same key
I have a string which represent array of jsons like
'[{json}, {json}...]' in this jsons I have similar keys like metric: '[{"metric": "", }, {"metric": "", }]'.
...
0
votes
1
answer
39
views
How do I preserve the number of rows when using a CTE?
I have the following ClickHouse query that works fine:
WITH CTE AS (
SELECT
-- ClickHouse specific JSON function
JSON_QUERY(json, '$.projects[*].userId') as userIds
FROM $table
...
1
vote
1
answer
57
views
What is the best way to change a column type in ClickHouse table?
What is the best way to change a column type in ClickHouse table from Int32 to Int64? The table size is about 5Gb (handreds of millions lines). The column is NOT NULL, the engine is ReplacingMergeTree....
0
votes
1
answer
113
views
Clickhouse async insert settings implementation
How to correctly using async insert settings in clickhouse?
I am following the documentation on clickhouse website here: https://clickhouse.com/docs/en/optimize/asynchronous-inserts to use async ...
-2
votes
1
answer
72
views
HTTPDriver for http://clickhouse.prod.do. returned response code 400. Tried to run this code, but had the error
import numpy as np
import pandas as pd
import datetime as dt
import gspread
import clickhouse_connect
from oauth2client.service_account import ServiceAccountCredentials
client = clickhouse_connect....
0
votes
0
answers
23
views
extra query needed for resource not found?
I am writing this API /trace/{{id}}/spans?span_kind=server. This is gonna fetch all the spans which are having span_kind=server.
If particular {{id}} is not present in database, we should throw 404.
...
0
votes
0
answers
38
views
Outer table alias is not recognized in ClickHouse
I have a time series table in ClickHouse called kafkadata and I want to find out rows matching a certain range in past data with respect to incoming data timestamp.
For example, in past records, find ...
0
votes
1
answer
49
views
How to transfer 200 million clickhouse rows to another server?
There is a clockhouse database in which there are billions of rows of data an example of 200 million rows needs to be transferred and transferred to another physical server.
Writing a python script ...
0
votes
0
answers
61
views
Clickhouse database - backup - replica has part different from part on another replica - inconsistencies between the shards
While taking backup of clickhouse, it throws exception about inconsistencies between parts in different shards in different replica.
BACKUP DATABASE default
ON CLUSTER 'cluster-name'
TO S3(
'...
0
votes
3
answers
47
views
How to delete rows from Replicated&Distributed tables on Clickhouse?
I have a table which is a distributed table over replicated tables in Clickhouse.
Although I have tried different things I can not delete any rows from the table. What should I do?
Delete queries I ...
0
votes
0
answers
83
views
DB::ParsingException: Cannot parse input: expected ']' before: 'json'
I need to parse a message from kafka to clickhouse version 22.8.1.1
The message is a single json array
[
{
"partner_id":156,
"id":"bc238532-e400-786c-1e86-...
-1
votes
3
answers
60
views
Calculate Time Difference Between difference rows without lead/lag
I have a live-dashboard with a new row being added to the table every time a new status is recorded.
Status
Start_Time
1
2023-08-23 09:23:43
0
2023-08-23 08:23:34
1
2023-08-23 05:20:05
3
2023-08-...
1
vote
2
answers
37
views
aggregate Array of Tuples in Clickhouse
I created a ClickHouse Table using the following code
CREATE TABLE meetings
(
_id UInt64,
records Array(Tuple(a String, b UInt64, c String))
)
ENGINE = MergeTree()
ORDER BY (_id);
Describing ...
0
votes
1
answer
65
views
Reproducing ClickHouse Client CLI query result format using Golang ClickHouse library
Clickhouse CLI result
When performing a query using the clickhouse-client command line interface and setting the format as JSON, we not only obtain the result but also statistics.
Command
clickhouse-...
0
votes
1
answer
22
views
Why does ClickHouse not support ON CLUSTER option for MATERIALIZED views?
I have checked the documentation and the option is supported for normal views but not for materialized views.
I read around a bit and noticed people are saying that materialized views act locally. ...
1
vote
0
answers
91
views
ClickHouse SELECT returning empty rows. Query is correct and works on ClickHouse SQL Console
So I'm using a ClickHouse DB and trying to run the following query in a Go API.
SELECT (SELECT round(sum(a.amount),2) FROM activity a WHERE a.created_at >= FROM_UNIXTIME(?) AND a.created_at <= ...
0
votes
0
answers
27
views
How to handle apostrophe in an INSERT statement when inserting into Clickhouse using Python?
I am inserting into Clickhouse but an error occurs because of the apostrophe in the statement.
tmp_list = ['A\'BCD','12345','1a2a3a4a']
insert = f'INSERT INTO tmp_table (NAME) VALUES (\'{tmp_list[0]}\...