Questions tagged [clickhouse]

ClickHouse is an open-source column-oriented DBMS for real time analytical reporting which has Capability to store and process petabytes of data.

Filter by
Sorted by
Tagged with
0 votes
0 answers
4 views

Getting Table is in readonly mode error when trying to drop a partition on Clickhouse Replicated Table

ALTER TABLE dbName.rTable DROP PARTITION 20220107 I want to run this command on all underlying replicated tables. when I run this command I get table is in read only mode error. rTable's are created ...
reho's user avatar
  • 3
0 votes
0 answers
18 views

Sending struct and nested structed spark columns to ClickHouse through JDBC

I want to send my spark dataframe to clickhouse table using JDBC. I have multiple struct columns in my dataframe, structs with struct, array of structs etc. |-- guest_third_party_id_arr: array (...
Sajal 's user avatar
  • 89
0 votes
1 answer
21 views

Is there a function to generate alternative +1s and -1s for a particular column in a table in Clickhouse database?

A little bit of background: I have a customer-data-lake which is like a repository having both old and new versions of each customer. I simply have to pick the latest record, but the materialized view ...
praj's user avatar
  • 1
0 votes
1 answer
22 views

running dbt with clickhouse cluster: DB::Exception: Syntax error: failed at position 201 ('ON') (line 5, col 5): ON CLUSTER

I am trying to run dbt on a clickhouse cluster the first run works just fine and dbt creates tables on all nodes with no issue, the second run however is causing the following error: DB::Exception: ...
Atheer Abdullatif's user avatar
-2 votes
0 answers
26 views

How clickhouse-driver cursor works under the hood? [closed]

I use cickhouse-driver in my applications. Want to use cursor from "DB API 2.0". query = select * from system.numbers limit %(param)s I came across the fact that when execute cursor.execute(...
agm's user avatar
  • 1
0 votes
0 answers
19 views

Getting frequent `No active replica has part` errors after upgrading clickhouse

I am getting frequent No active replica has part errors in clickhouse v23.8.2.7. I checked system.replication_queue. There is no record which might be stuck due to this error. The errors are getting ...
Chirag Bansal's user avatar
1 vote
0 answers
9 views

How to set labels in Grafana with ClickHouse

I use Grafana + ClickHouse to monitor logs. A have a "Logs" widget made from clickhouse sql query. When I switch on "Unique labels" option I see that no unique labels in the ...
Dmitry  Skryabin's user avatar
0 votes
1 answer
15 views

Virtual columns in Kafka engine

I want to get virtual columns, but when I write the example code I get an error. I'm following the Kafka engine documentation in Clickhouse: [kafka-engine](https://clickhouse.com/docs/en/engines/table-...
sharp-0000ff's user avatar
0 votes
0 answers
36 views

Losing data while inserting in clickhouse

I want to insert data from one table to another. I use no filters except of date so my query look as follows: insert into table1 select * from table2 where dt = 'yyy-mm-dd' But I lose ...
Maria's user avatar
  • 47
0 votes
0 answers
21 views

How to manage different data types in clickhouse

I'm new to Clickhouse and for the moment everything goes well. But I'm facing right now with a problem. I have to create data from a python script. But the values have different types. For the moment ...
PsyKozZ09's user avatar
0 votes
1 answer
29 views

IP to ASN lookup using ClickHouse

I have a table consisting of IP/CIDR and AS Number and a regular sFlow data table. Since the sFlow data does not have an ASN, I want to add ASN into sFlow data table using IP/CIDR-ASN table. All IP ...
2sang's user avatar
  • 1
0 votes
1 answer
18 views

How to reduce the memory usage of a multiple ASOF JOIN queries?

I have one query that results in ~100 rows. Not really much. This table is called FillsTable. Then I have mulitple tables SignalTable1,2,3,4,5,6 that have millions of rows. I want to ASOF JOIN the ...
KamilCuk's user avatar
  • 122k
1 vote
0 answers
47 views

'HttpClient' object has no attribute 'execute' clickhouse python

I'm trying to execute a command with Clickhouse client in Python as I have seen it in the document but it keeps give me an error 'HttpClient' object has no attribute 'execute' from clickhouse_driver ...
ali heydarabadii's user avatar
-1 votes
0 answers
35 views

Represent nested structure in Clickhouse for optimal query performance

I need to map an Active Directory structure in a Clickhouse database. I have ~1000 Active Directory groups in a nested fashion, ~10 levels deep nesting (group A contains group B .... 10 times). I ...
dei's user avatar
  • 41
0 votes
2 answers
59 views

Clickhouse creating columns base on value of other rows

I'm trying to know how much record a user has generated during the last x hours from when the record is created. In classic SQL what I've tried so far should be working. I'm posting here because I ...
guana's user avatar
  • 3
0 votes
1 answer
39 views

How do I update partial columns in clickhouse

I have a large table in clickhouse, and after adding a column with "alter table add column ed_code String", I want to refresh the column data according to the primary key. What should I do? ...
rant S's user avatar
  • 1
0 votes
1 answer
14 views

How to write ClickHouse OVER statement in DolphinDB?

I have a query in ClickHouse with the following syntax: avg(ask) over(partition by sym order by time asc rows between 13 preceding and 0 following) How can I write a DolphinDB script that can achieve ...
molddd123's user avatar
  • 305
-1 votes
2 answers
43 views

Can't import CSV to ClickHouse

I'm trying to import CSV file to ClickHouse table: INSERT INTO default.tags FROM INFILE '/home/sergey/test/tags_202309081557.csv' FORMAT CSV; but got strange error: SQL Error [78] [07000]: Code: 78. ...
Sergey Lebedev's user avatar
0 votes
1 answer
25 views

How to deal with counter resets in ClickHouse?

I try to use the ClickHouse to store time series data. Precisely, accumulated counters. They behave the same as in Prometheus. Regrettably, I have been unable to locate any examples demonstrating how ...
Dustin Deus's user avatar
0 votes
0 answers
26 views

Clickhouse kafka integration tables cleaning

I use kafka integration mechanism to transfer data via chain (kafka engine)->(materialized view with TO)->(destination table) materialized view reads any data block coming to kafka table , and ...
Alexandr  's user avatar
0 votes
1 answer
26 views

How to add column to nested column

I have a table with nested column ... a UInt64 b Nested (b1 String, b2 String) c LowCardinality(String), ... I want to update the table to be ... a UInt64 b Nested (b1 String, b2 String, **b3 String**...
D.A's user avatar
  • 19
0 votes
1 answer
45 views

Can not access ClickHouse remotely on Ubuntu 20.04 DB::NetException Timeout: connect timed out

OS: Ubuntu 20.04.6 LTS ClickHouse: 23.7.2.25 (official build) Server: Intel(R) Xeon(R) CPU E5-2695 v4 @ 2.10GHz I can not connect from outside remotely. Error: $ clickhouse-client --user fridary --...
sirjay's user avatar
  • 1,779
-2 votes
0 answers
22 views

The ClickHouse log error refreshing, leading to the filling of the entire storage space

here is log detail It will endlessly loop and print errors until the storage space is exhausted. {"log":"11. void std::__function::__policy_invoker\u003cvoid ()\u003e::__call_impl\...
歪歪歪's user avatar
0 votes
2 answers
60 views

Get all dates between two dates with id

I have a table with sales like this: Select Id, startdate, endate From tabe Output: 1 '01-01-2022' '06-01-2022' 1 '02-01-2022' '05-01-2022' 1 '13-01-2022' '17-01-2022' 2 '01-01-2022' '02-01-2022' 2 ...
Irina's user avatar
  • 35
0 votes
0 answers
21 views

Clickhouse asynchronous_insert_log table not visible

asynchronous_insert_log table is not there under system db in clickhouse. My config.xml contains below configuration by default: <clickhouse> <!-- Other settings --> <...
Komal Bhadani's user avatar
0 votes
2 answers
22 views

Clickhouse WITH FILL STEP toIntervalHour(1) exception

I'm trying to do an aggregation using ORDER by date WITH FILL STEP toIntervalHour(1) but ClickHouse returns me an exception: Received exception from server (version 23.5.3): Code: 475. DB::Exception: ...
viniciusbo's user avatar
0 votes
0 answers
32 views

The clickhouse cluster cannot create any tables

Code: 107. DB::Exception: There was an error on [ck1:9000]: Code: 107. DB::ErrnoException: Cannot open file /data/clickhouse/store/76b/76b93482-a138-42fc-9969-e608641273be/zxc_sig.sql.tmp, errno: 2, ...
cnzxc's user avatar
  • 1
0 votes
1 answer
62 views

How to speed up Clickhouse query

I have a Clickhouse table defined as below: CREATE TABLE default.my_table ( `timestamp` DateTime CODEC(Delta(4), ZSTD(1)), `service` LowCardinality(String) CODEC(ZSTD(1)), `operation` ...
navinpai's user avatar
  • 955
0 votes
2 answers
59 views

How do I add a primary key to a column, an existing data table?

What is the request to add the primary key to a certain column, an existing MergeTree data table? I tried to change it with a standard query, but it didn't give any results: ALTER TABLE table_name ADD ...
Yuri's user avatar
  • 37
0 votes
1 answer
39 views

Insert data to table if subquery exists

What I want to do is check if a subquery exists, if exist then insert some data into a table, like: if exists (select * from tableB) update tableA set status=1 where id = 1001; I cannot update the ...
nick's user avatar
  • 39
1 vote
1 answer
96 views

How to integrate Rust UDF using C in Clickhouse?

Have a UDF in C main.c #include <stdio.h> #include <stdlib.h> #include <string.h> extern char *u_shaped_processor(char* camps_list_c); void replacechar(char *s,char c1,char c2) { ...
Sannidhya Malpani rev's user avatar
0 votes
0 answers
44 views

Clickhouse aggregate page view user flow by sessions

The clickhouse database table looks somewhat like this: | Timestamp | SessionId | PagePath | ReferrerPath | ... | My desired outcome would be something along these lines in order to visualize the page ...
Marvin's user avatar
  • 37
0 votes
2 answers
43 views

How to choose parameters for Buffer table in Clickhouse?

I want to create a buffer table to reduce the number of small inserts, but I don't understand how to select parameters for the table? For example, I want the buffer table to flash the data to the ...
Maria's user avatar
  • 47
0 votes
2 answers
52 views

clickhouse extract several values by same key

I have a string which represent array of jsons like '[{json}, {json}...]' in this jsons I have similar keys like metric: '[{"metric": "", }, {"metric": "", }]'. ...
Axel HK's user avatar
  • 97
0 votes
1 answer
39 views

How do I preserve the number of rows when using a CTE?

I have the following ClickHouse query that works fine: WITH CTE AS ( SELECT -- ClickHouse specific JSON function JSON_QUERY(json, '$.projects[*].userId') as userIds FROM $table ...
Foobar's user avatar
  • 7,540
1 vote
1 answer
57 views

What is the best way to change a column type in ClickHouse table?

What is the best way to change a column type in ClickHouse table from Int32 to Int64? The table size is about 5Gb (handreds of millions lines). The column is NOT NULL, the engine is ReplacingMergeTree....
doktr's user avatar
  • 406
0 votes
1 answer
113 views

Clickhouse async insert settings implementation

How to correctly using async insert settings in clickhouse? I am following the documentation on clickhouse website here: https://clickhouse.com/docs/en/optimize/asynchronous-inserts to use async ...
Shikhar Tyagi's user avatar
-2 votes
1 answer
72 views

HTTPDriver for http://clickhouse.prod.do. returned response code 400. Tried to run this code, but had the error

import numpy as np import pandas as pd import datetime as dt import gspread import clickhouse_connect from oauth2client.service_account import ServiceAccountCredentials client = clickhouse_connect....
EugeneSam's user avatar
0 votes
0 answers
23 views

extra query needed for resource not found?

I am writing this API /trace/{{id}}/spans?span_kind=server. This is gonna fetch all the spans which are having span_kind=server. If particular {{id}} is not present in database, we should throw 404. ...
kinnera kinnu's user avatar
0 votes
0 answers
38 views

Outer table alias is not recognized in ClickHouse

I have a time series table in ClickHouse called kafkadata and I want to find out rows matching a certain range in past data with respect to incoming data timestamp. For example, in past records, find ...
Muhammad Ahmad Mujtaba's user avatar
0 votes
1 answer
49 views

How to transfer 200 million clickhouse rows to another server?

There is a clockhouse database in which there are billions of rows of data an example of 200 million rows needs to be transferred and transferred to another physical server. Writing a python script ...
Andrey's user avatar
  • 3
0 votes
0 answers
61 views

Clickhouse database - backup - replica has part different from part on another replica - inconsistencies between the shards

While taking backup of clickhouse, it throws exception about inconsistencies between parts in different shards in different replica. BACKUP DATABASE default ON CLUSTER 'cluster-name' TO S3( '...
Sairam Krish's user avatar
  • 10.3k
0 votes
3 answers
47 views

How to delete rows from Replicated&Distributed tables on Clickhouse?

I have a table which is a distributed table over replicated tables in Clickhouse. Although I have tried different things I can not delete any rows from the table. What should I do? Delete queries I ...
reho's user avatar
  • 3
0 votes
0 answers
83 views

DB::ParsingException: Cannot parse input: expected ']' before: 'json'

I need to parse a message from kafka to clickhouse version 22.8.1.1 The message is a single json array [ { "partner_id":156, "id":"bc238532-e400-786c-1e86-...
Axel HK's user avatar
  • 97
-1 votes
3 answers
60 views

Calculate Time Difference Between difference rows without lead/lag

I have a live-dashboard with a new row being added to the table every time a new status is recorded. Status Start_Time 1 2023-08-23 09:23:43 0 2023-08-23 08:23:34 1 2023-08-23 05:20:05 3 2023-08-...
Tani1297's user avatar
1 vote
2 answers
37 views

aggregate Array of Tuples in Clickhouse

I created a ClickHouse Table using the following code CREATE TABLE meetings ( _id UInt64, records Array(Tuple(a String, b UInt64, c String)) ) ENGINE = MergeTree() ORDER BY (_id); Describing ...
kobe24's user avatar
  • 190
0 votes
1 answer
65 views

Reproducing ClickHouse Client CLI query result format using Golang ClickHouse library

Clickhouse CLI result When performing a query using the clickhouse-client command line interface and setting the format as JSON, we not only obtain the result but also statistics. Command clickhouse-...
calebeaires's user avatar
  • 1,972
0 votes
1 answer
22 views

Why does ClickHouse not support ON CLUSTER option for MATERIALIZED views?

I have checked the documentation and the option is supported for normal views but not for materialized views. I read around a bit and noticed people are saying that materialized views act locally. ...
atoMerz's user avatar
  • 7,542
1 vote
0 answers
91 views

ClickHouse SELECT returning empty rows. Query is correct and works on ClickHouse SQL Console

So I'm using a ClickHouse DB and trying to run the following query in a Go API. SELECT (SELECT round(sum(a.amount),2) FROM activity a WHERE a.created_at >= FROM_UNIXTIME(?) AND a.created_at <= ...
liam2682's user avatar
0 votes
0 answers
27 views

How to handle apostrophe in an INSERT statement when inserting into Clickhouse using Python?

I am inserting into Clickhouse but an error occurs because of the apostrophe in the statement. tmp_list = ['A\'BCD','12345','1a2a3a4a'] insert = f'INSERT INTO tmp_table (NAME) VALUES (\'{tmp_list[0]}\...
chapy's user avatar
  • 1

1
2 3 4 5
38