Questions tagged [citus]
Citus is a distributed database that lets you run SQL queries over very large data sets.
118
questions
0
votes
0
answers
14
views
citus:truncate records of a table
I want to use sql command truncate table to truncate all records in a distributed table of citus,but seems it doesn't work,so I have to login to each datanode to truncate each shard of the table. so,
...
1
vote
0
answers
28
views
postgresql citus: ctas with select on table with columnar storage runs in error, same select with select on non-columnar storage works
We run a postgresql 14 with citus_columnar in version 11.1.
Scenario 1: select on 3 tables, where 2 two tables are partitioned and "using columnar". This works pefectly
Scenario 2: the ...
0
votes
1
answer
81
views
Postgres-15.1 is restarting continuously on using shared_preload_libraries extension
Postgres is restarting continuously on using shared_preload_libraries extension.
https://postgresqlco.nf/doc/en/param/shared_preload_libraries/
I am running postgres-15.1 using a python-based daemon ...
1
vote
0
answers
33
views
Resolve debug symbols for postgres
I am working on an adding a new extension on citus and am in the process of collecting and analyzing the stack traces for the extension, postgres, citus and other libs by running ycsb workloads ...
0
votes
0
answers
55
views
How to query JSONB column with nested array & objects in Citus postgres
I have a Postgres Table "Organization" containing "email_details" column of JSONB data.
This column will hold data in below format.
{
"emails":
[
{
...
0
votes
1
answer
53
views
citus: Can I add one more replica for a distributed table
I have a distributed table,but this table only has one replica,only one replica doesn't have
ha, I want and one more replica for the table,can i? how to do?
I have search online help docs,but didn't ...
2
votes
1
answer
49
views
How can I stop an active query on a postgres coordinator when the worker node has crashed
I have a postgres select query that can not be stopped using the standard pg_cancel/pg_terminate commands.
Both commands return true, but do nothing.
The query has access share locks on hundreds of ...
1
vote
1
answer
79
views
Query against worker nodes omitting the coordinator in Citus
Existing documentation for Citus11 explicitly points out that external clients should interact with the Citus cluster through the coordinator node, which is supposed to route request among workers.
...
2
votes
1
answer
127
views
Cannot create distributed table with foreign key on Citus 11.1
I want to migrate from Citus 11.0 to 11.1.
I'm using the docker-compose file from Citus' github.
The following script works fine on 11.0, but not on 11.1:
create table tmp_table1 (
id varchar(36) ...
0
votes
2
answers
188
views
add worker node and rebalance distributed shards problem with PostgreSQL + Citus
The cluster had 1 coordinator and 6 worker nodes. (environment: PostgreSQL 15 + Citus 11.1)
We want to add more worker nodes to it, so we executed
SELECT * from citus_add_node('node-name', 5432);
and
...
1
vote
1
answer
300
views
Update PostgreSQL 14 to 15 with active citus extension
I am trying to use the official way
to upgrade from PG 14 to 15 with Citus. If I would follow these instructions, I would have upgraded PG, but not citus, since nowhere the citus111_15 rpm is being ...
0
votes
1
answer
181
views
Delete in Postgres citus columnar storage alternatives
I am planning to use citus to store system logs for upto n no of days after which they should be deleted.Citus columnar store looked like the perfect database for this until I read
this where its ...
0
votes
2
answers
226
views
Postgresql doesn't working with Citus and pg_stat_statements at the same time
So, I built the PostgreSQL with citus extension in docker.
I use the official documentation in citus, then I run this command in the terminal.
docker run -d --network citus-network --name ...
0
votes
0
answers
45
views
ERROR: attempted to read an unexpected stripe while reading columnar table daily_p2022_10_11, stripe with id=281134 is not flushed
I am getting this error in postgres alert log:
ERROR: attempted to read an unexpected stripe while reading columnar table daily_p2022_10_11, stripe with id=281134 is not flushed.
While trying vacuum, ...
0
votes
0
answers
157
views
'another command is already in progress' returned when trying to rebalance shards
There are two Postgres instances running in a Citus cluster:
analysis=# select * from citus_check_cluster_node_health();
from_nodename | from_nodeport | to_nodename | to_nodeport | result
-------------...
0
votes
1
answer
33
views
Citus - Possible to undo citus's create_time_partitions results?
I'm working on a database with Citus and I would like to know if it's possible to undo the results made by executing create_time_partitions. I know that it is possible to "undistribute" a ...
0
votes
1
answer
179
views
Citus data - How to query data from a single shard in a query?
We are evaluating Citus data for the large-scale data use cases in our organization. While analyzing, I am trying to see if there is a way to achieve the following with Citus data:
We want to create ...
0
votes
0
answers
206
views
Citus support the HA? ( High Availability )
i have one simply question: citus 11 support the HA?
This is the structure that i have implemented with 7 node ( replication with pg_autofailover):
The problem occurs when i disable a worker1_primary ...
1
vote
1
answer
470
views
Citus sharding, replication, and replication
I setup the citus 11 cluster using the official docker and docker compose instruction. I scaled the cluster to 5 worker nodes and created a distributed table, with replication factor 3.
I expected the ...
0
votes
1
answer
182
views
How to promote metadatasynced worker to coordinator in Citus, if coordinator completely lost
I have a citus cluster with 1 coordinator and 3 workers.
Recently, the server which hosted coordinator is completely down which cannot be recovered.
Now the cluster only does Query and Some DML, but ...
3
votes
1
answer
153
views
Citus shard rebalancing does not progress
I have a Citus cluster with 3 worker nodes, added a new worker recently then started a rebalance on a table. Everything is OK until here but rebalancing does not end. When I look at ...
0
votes
1
answer
58
views
pg_auto_failover Installation fails on Rocky linux
I want to install pg_auto_failover(https://github.com/citusdata/pg_auto_failover) on rocky linux 8.5 .
But when I key in the following install command:
(1) curl https://install.citusdata.com/community/...
2
votes
1
answer
415
views
PostgreSQL Citus Shard Rebalancing after adding new node -> connection to the remote node localhost:5432 failed with the following error
Here I have a problem while applying Citus rebalancing after distributing a specific PostgreSQL table and adding new nodes to scale my database.
You can take a look at this useful article if you would ...
1
vote
1
answer
145
views
Sharding with replication
Sharding with replication]1
I have a multi tenant database with 3 tables(store,products,purchases) in 5 server nodes .Suppose I've 3 stores in my store table and I am going to shard it with storeId .
...
0
votes
0
answers
116
views
Poor performance when distributing a table in Azure Citus
Currently using Citus in Azure for a high-volume transactional application. 1 coordinator node, 3 worker node. This application performs inserts, but also a lot of queries like the one below. The ...
0
votes
1
answer
144
views
Create a PostgreSQL Hyperscale Citus DB from Azure CLI (not ARC)
How do I create a hyperscale PostgreSQL DB from the Azure CLI?
Please note, I am not asking about Arc-enabled Kubernetes clusters.
The current documentation is for Portal only.
Is there a way to do ...
0
votes
0
answers
214
views
Could not access file “citus”:no such file or directory;
when i start postgres-14, then tell me "Could not access file “citus”:no such file or directory;"
i have insetlled citus. but i still don't start postgres-14.can you help me?
0
votes
1
answer
142
views
curl Unfortunately,your operating system distribution and version are not supported by this script
my operating system is rocky8.5。when i execute “curl https://install.citusdata.com/community/rpm.sh | sudo bash”,tell me “Detected operatiog system as rocky/8 Unfortunately,your operating system ...
0
votes
0
answers
95
views
Stopping Citus Data postgres container corrupting the data with following error?
I have set up an rails app with citus data using docker-compose.
version: "3"
services:
master:
container_name: "${COMPOSE_PROJECT_NAME:-citus}_master"
image: "...
0
votes
0
answers
49
views
select from returns nothing on citus columnar storage
i've create a table named tbl1 with USING columnar in citus then inserted dump file successfuly :
SELECT pg_total_relation_size('tbl1')
+--------------------------+
| pg_total_relation_size |
|------...
0
votes
1
answer
402
views
Postgres Citus, immutable date conversion
Trying to update some dates programmatically on Citus I always get
[0A000] ERROR: STABLE functions used in UPDATE queries cannot be called with column references
From a query like
UPDATE ...
0
votes
1
answer
414
views
Citus Sharding and PostgreSQL table partitioning on the same column
We are starting a project with PostgreSQL and will need to use Citus in the near future for a multi tenant application so we are aiming at preparing our PostgreSQL database schema appropriately for
an ...
0
votes
1
answer
92
views
citus, which column can be treated as distribution column
I'm a newbie of citus and need an advice that which column will be treated as distribution column when create_distributed_table.
Exp 1, the snapshot table of order, every row containing an unique ...
3
votes
1
answer
599
views
Primary Keys (UUID, Sequence) strategy on Citus
What is the best approach for primary keys on Citus?
UUID:
No lock required, as opposed to the Identity/Serial. But expensive in storage and eventually on queries + causes fragmentations.
Sequence - ...
3
votes
1
answer
768
views
citus rebalance_table_shards() -> fe_sendauth: no password supplied
Version: postgresql 14 and citus 10.2
When I execute
select rebalance_table_shards('tbl_name');
in the CN node, postgresql gives an error message: ERROR: connection to the remote node localhost:5432 ...
1
vote
1
answer
154
views
Can we use Citus with Gorm?
We are trying to do distributed DB using PostgreSQL.
We came across Citus (https://github.com/citusdata/citus)
but we are using GORM as ORM for our backend.
Is it possible to combine both??
2
votes
1
answer
94
views
Best Approach for Developing a SaaS-based Event Analytics
My company is developing a SaaS service to store events and provide analytics through dashboards.
Since we won't have deletes or updates, the idea is to create a columnar-based, OLAP architecture to ...
2
votes
2
answers
1k
views
How to set Postgres tables to use columnar storage using dbt?
We are using dbt to manage our data models in MSSQL, and we are considering moving to PostgreSQL Citus.
I'm not clear on how I could use Citus's features via dbt? I'm thinking specifically of data ...
3
votes
2
answers
9k
views
AWS DMS Task failed with error: Error executing source loop; Stream component failed at subtask 0
I want to migrate my PostgresDB hosted in Citus cloud service to AWS RDS Aurora Postgres.
I am using AWS DMS service. Have created task but getting following errors:
Last failure message Last Error ...
0
votes
1
answer
89
views
Increasing shards in Citus
How can we add shards to existing colocated tables in citus?
I found out that the default is 32, which is definitely lower than expected. Is there a easy way of resizing without downtime?
0
votes
0
answers
110
views
testing replication from Citus to my RDS Aurora Postgres on subscriber no data is coming
I am testing replication from Citus(Cloud Hosted) to my RDS Aurora Postgres with ref https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.Replication.Logical.html#...
0
votes
2
answers
345
views
Citus 10.2 not starting an instance of PostgreSQL
I'm using RHEL 8.4 and I followed the installation instruction at Set up a single-node Citus cluster on your own Linux machine from RPM packages..
Step 1 and 2 (Install PostgreSQL 14 and the Citus ...
-1
votes
1
answer
216
views
I am unable to install citus version 10.2 on my ubuntu 21.04
I am unable to install citus version 10.2 on my ubuntu 21.04
with the following errors :
E: Unable to locate package postgresql-14-citus-10.2
E: Couldn't find any package by glob 'postgresql-14-citus-...
2
votes
1
answer
541
views
Migrate data from Citus to RDS
Since Citus is not going to be available as a Managed Service in AWS, I am trying move the database to RDS (not the whole history but only the transactional portion as an OLTP). The migration from ...
0
votes
1
answer
160
views
measure tpc-ds benchmark on citus
I'm trying to do some measurement on citus (extension of postgres). for that mission I'm running tpc-ds query on citus. the citus that I'm using is a containers of master, workers and manager that ...
0
votes
1
answer
162
views
Citus "... is a metadata node, but is out of sync HINT: If the node is up, wait until metadata gets synced to it and try again."
I've got a Citus (v10.1) sharded PostgreSQL (v13) cluster with 4 nodes. Master node address is 10.0.0.2 and the rest are up to .5 When trying to manage my sharded table, I've got this error:
ERROR: ...
0
votes
1
answer
376
views
citus can I use join if I use citus?
citus can I use join if I use citus ?
And yes are they performant ?
........................................................................................................................................
0
votes
1
answer
431
views
How to get aggregate row count for sharded tables in citus postgres
Requirement is to get sum of reltuples for all distributed tables in Citus Postgres database with a certain format which is explained below.
When i run the below command, the query propagates to ...
0
votes
1
answer
521
views
Citus: How can I add self referencing table in distributed tables list
I'm trying to run create_distributed_table for tables which i need to shard and almost all of the tables have self relation ( parent child )
but when I run SELECT create_distributed_table('table-name',...
0
votes
1
answer
472
views
pg_autofailover setup with password
I have created the postgres with pg_auto failover on one primary and one secondary node .
I am followed this link. https://www.citusdata.com/blog/2019/05/30/introducing-pg-auto-failover/ I tested the ...