Questions tagged [questdb]
QuestDB is a high-performance, open-source SQL database used in financial services, IoT, machine learning, DevOps and observability.
233
questions
2
votes
0
answers
14
views
How to fix errno=24 in QuestDB?
I run QuestDB and sometimes I have my WAL tables suspended with the error message
2023-12-04T13:32:29.490424Z C i.q.c.w.ApplyWal2TableJob job failed, table suspended [table=..., error=could not open ...
0
votes
1
answer
10
views
How to fine-tune QuestDB to ingest into 1,000 tables or more?
I'm trying to ingest data into 1,000 tables in QuestDB. But my machine runs out of free disk space quickly as soon as I start my InfluxDB Line Protocol client. Occasionally, I'm also getting Linux' ...
3
votes
1
answer
18
views
How can I drop all tables in QuestDB?
I tried to drop all tables but received following error:
failed to drop tables ['telemetry': [-1] could not lock 'telemetry' [reason='telemetry']]
How can I drop all tables?
2
votes
1
answer
7
views
How to configure alerting for critical QuestDB errors based on logs?
I'm configuring monitoring for my QuestDB instance and I'd like to have alerts when a critical error in the database. How do I do that?
2
votes
1
answer
40
views
How can I use lag(offset) window function on QuestDB?
I know QuestDB has some window functions, but lag is still not supported. I have been using LT JOINS to get the value from the previous row. This is not ideal (query more verbose than needed and join ...
1
vote
1
answer
28
views
questdb Column names must be unique on web console
what is wrong at my define of a table ?
Thanks for your help.
Defines in the web-Console:
Table name Partition by WAL
testtable hour disabled
columns
Column names must be unique
datetime ...
2
votes
1
answer
37
views
QuestDB stopped writing data because of the lack of the disk space, how to recover?
I had QuestDB v 7.3.4 working for quite a while but at one point the data took up all the disk space and QuestDB stopped.
I have added more disk space and expanded the file system. I see QuestDB ...
3
votes
1
answer
32
views
How to generate a candle stick chart with QuestDB?
I have a table with market data. The table structure looks like this:
symbol
side
price
amount
timestamp
BTC-USD
buy
35831.56
0.00051151
2023-11-15T10:42:11.754626Z
I need to aggregate prices for a ...
0
votes
0
answers
25
views
File logging in questDb docker container doesn't work
I have file logging enabled in the mountedDir/conf/log.conf:
# list of configured writers
writers=file,stdout
# file writer
w.file.class=io.questdb.log.LogRollingFileWriter
w.file.location=log/...
1
vote
1
answer
38
views
How to back up QuestDB
I'm running QuestDB 7.3.3 on Linux and I'd like to back up my database. BACKUP SQL statement seems deprecated, so I probably should use SNAPSHOT, right?
0
votes
0
answers
14
views
How to enable log rotation in QuestDB [duplicate]
How do I enable log rotation in QuestDB to avoid disk space issues? It looks like QuestDB logs are never rotated or deleted on their own. I could not find anything about log rotation in QuestDB ...
3
votes
2
answers
46
views
Sum of column containing nan and inf as values in SQL
I am using QuestDB and one column has values double e.g. 10.5, 5.2 etc, but few rows come like -nan, inf for not a number and infinity.
What is the best way to sum this column. If I do normal sum it ...
4
votes
1
answer
32
views
Automate partition removal on QuestDB
I have several time-series tables with daily partitioning on a QuestDB database. I would like to remove every day all the partitions older than 21 days from my tables. What would be the best way to do ...
2
votes
1
answer
29
views
QuestDB - ingest data from Telegraf
When using JSON Web Key as authentication method for ILP, is there any way to ingest data into QuestDB from Telegraf?
The authentication part is not covered in the docs.
https://questdb.io/docs/third-...
2
votes
1
answer
41
views
Restoring QuestDB backup [closed]
Was going through backup and restore doc for QuestDB here: https://questdb.io/docs/operations/backup/
Is restoring a database backup a manual process as I can't always have a backup dir mounted from ...
2
votes
1
answer
43
views
How can I change the designated timestamp in QuestDB
I have a table with a designated timestamp col A. How can I change the designated timestamp from col A to col B? Is it possible at all?
1
vote
0
answers
25
views
QuestDB cloud ILP connection error: Failed to read authentication challenge (timed out?): Resource temporarily unavailable (os error 11)
I am following the code example from QuestDB and migrating my data from InfluxDB. I have files exported in ILP format, so I read and send them one by one using:
try:
with Sender(host, port,...
2
votes
1
answer
25
views
How can I load data into QuestDB using the InfluxDB Client Library?
I have a lot of code written using the InfluxDB Java Client, but want to migrate my database to QuestDB without having to change my code. The InfluxDB client libraries use HTTP as a transport method ...
1
vote
1
answer
41
views
QuestDB: LineTcpConnectionContext [151] buffer overflow
I'm experiencing a problem with QuestDB ingestion via the Influx Line Protocol.
The setting is the following: from time to time I create a JSON value (for one column in a table) that has two lists of ...
2
votes
1
answer
29
views
Column names with dots in QuestDB?
I'd like to use the Jolokia Telegraf plugin to collect JMX metrics and push them to QuestDB. However, the Jolokia plugin creates tags with dots and QuestDB does not support dots in column names. Is ...
1
vote
3
answers
126
views
Read UTC timestamp from questdb with pg module. Date object has wrong UTC time
All the timestamps in the questdb are UTC times but when I read them with the nodejs questdb client it will give me the correct time but with my local timezone.
For example, in the database a time of ...
0
votes
1
answer
27
views
QuestDb Web Console Path for Application Load Balancer
I have set up QuestDb pod in AWS EKS. I have ALB to EKS mapped.
I want to know what to use for QuestDb's ALB Listener path
I tried searching it on the internet but all I found was just /
It's not ...
1
vote
1
answer
26
views
QuestDB Use result of SELECT MIN (...) in WHERE
I would like to do something like this:
select *
from some_table t
where t.ts >= (select min(ts) from other_table)
and t.ts <= (select max(ts) from other_table)
Is this somehow possible? I ...
0
votes
2
answers
45
views
Select distinct by one column ordered by time
I have this table:
ID
ParentNumber
ObjectNumber
Start
Component
...
4b8cf664-0ef9-44b2-a266-6c27079b90ed
2
1
2023-08-10T12:50:03.716000Z
1
...
92847bf9-dbfc-46ff-a9bb-b2210df66507
2
1
2023-08-11T17:...
0
votes
1
answer
111
views
How to increase query performance in QuestDB? Will qdb index help or should I use something instead WITH keyword?
I have a QuestDB database with 3 tables.
Table Foo - 15 columns
Table Bar - 30 columns
Table Baz - 23 columns
Most columns are string or long data types. All tables are partitioned by DAY.
40 days ...
0
votes
1
answer
53
views
How to stream large results sets from QuestDB to java application?
I have an issue with PG Driver in Java. Usually I need to stream around 50-60 million rows from db to my java application, however (internally) the PG Driver creates a lot of garbage and all the ...
1
vote
3
answers
143
views
Is symbol type compressed or indexed in Kdb+?
The official doc already showed that symbol is an atomic, but does this feature utilized for compressing or indexing ?
What we can know is that, in QuestDB, the symbol type is compressed and indexed, &...
1
vote
2
answers
73
views
In questDB retrieve the record which is having the closest to range of timestamps
I am new to QuestDB, and I think basic SQL commands run on it.
I want a scenario where input is 5 timestamps (say T1,T2,T3...T5) in a single query I want to get the result from DB where timestamp is ...
0
votes
0
answers
95
views
QuestDB ILP - An existing connection was forcibly closed by the remote host
I'm entering data on questDB via ILP, from time to time the application stops entering data, giving me this error:
System.IO.IOException: Unable to write data to the transport connection: An existing ...
1
vote
1
answer
152
views
Is it possible to SAMPLE BY any timestamp column in QuestDB?
All SAMPLE BY examples at https://questdb.io/docs/reference/sql/sample-by/ use the designated timestamp column of a table.
Is it possible to use a different column ?
1
vote
1
answer
55
views
questDB 7.1.1 java.lang.UnsupportedOperationException
I am trying one of the latest features of QuestDB (v7.1.1).
Connecting to questdb using Npgsql in C#: the request SELECT * FROM table_partitions('F-WWCT_adc')
does not work and the log report i.q.c.p....
1
vote
2
answers
120
views
Does QuestDB cache query results?
Are query results cached in QuestDB ?
What does i.q.c.p.PGConnectionContext query cache used [fd=34248] appearing in logs mean ?
When I'm running
SELECT colX FROM (SUBQUERY)
for the first time it ...
0
votes
1
answer
93
views
Read-only user for Grafana when connecting to QuestDB
I have a Grafana dashboard connected to QuestDB to visualise low-latency data from cryptocurrency.
I would like to set the user as read-only, but it seems QuestDB is postgresql compatible only for ...
0
votes
1
answer
109
views
SAMPLE BY and FILL to generate intervals between two arbitrary dates [closed]
I have a table with some data in QuestDB. If I want to downsample by day and interpolate empty rows for the days where I have no data I can do it via
SELECT timestamp, AVG(energy) AS energy FROM ...
1
vote
1
answer
192
views
How do I tune symbol column capacity in QuestDB?
Say, I have a symbol column with 1M unique values and 10M rows per day and I want to add an index for that column.
How do I tune symbol and index capacity to make sure that QuestDB performance is ...
2
votes
1
answer
141
views
How do I import single-line CSV into QuestDB?
I'm trying to import a table into QuestDB via the /imp REST endpoint. The file contains a single row one with no column row:
a,10.5,True
I'm trying to import it with cURL:
curl -v -F [email protected] '...
1
vote
1
answer
49
views
Questdb docker StaticContentProcessor [106] not found
I'm running questdb via docker on Linux (Ubuntu)
sudo docker run -p 9000:9000 questdb/questdb
and want to record some price data in python: from cryptofeed I use the example
https://github.com/...
0
votes
1
answer
70
views
saving questdb database on an external disk
I don't have enough space on my computer so I am trying to create a questdb on an external hard drive. Is there a way I can do this?
I am looking at the documentation and going through ~/.questdb/conf ...
-1
votes
1
answer
146
views
Create a table by putting data from psycopg2 on QuestDB
Can QuestDB create a table by putting data from psycopg2 in it?
I want to initial table using psycopg2
0
votes
1
answer
61
views
How To Use REINDEX in QuestDB?
I'm testing REINDEX operation of QuestDB in the localhost:9000 Web Console.
But there continuously comes error:
[23:12:14] Cannot lock table: /usr/local/var/questdb/db/my_table
Here is my code:
drop ...
1
vote
1
answer
298
views
How to get data from questdb and get it in a js file? [closed]
Description:
Implement a ZoomingAndScrolling chart with infinite scrolling capability.
Set up a new project on CodeSandbox, using the template from the following link
https://js.devexpress.com/Demos/...
1
vote
1
answer
99
views
What's the best way to upload an ILP file into QuestDB? [closed]
I have a text file in ILP format that I would like to import into QuestDB.
I know QuestDB can ingest data in ILP format, but it seems you can do this only line by line. I have tried using the HTTP ...
1
vote
1
answer
91
views
Finding the initial sampled time window after using SAMPLE BY again
I can't seem to find a perhaps easy solution to what I'm trying to accomplish here, using SQL and, more importantly, QuestDB. I also find it hard to put my exact question into words so bear with me.
...
0
votes
1
answer
99
views
"Not Found" problem with QuestDB behind NGINX
I am running a QuestDB 6.6.1-server. Now I want to increase the protection of this server and put the web gui behind an NGINX reverse proxy as described in QuestDB blog post where setting up basic ...
1
vote
1
answer
122
views
How can I output values for time intervals with no data in QuestDB
I am using QuestDB to get the amount of events we are receiving every 500 milliseconds. Everything works as expected and I can use SAMPLE BY 500T to aggregate in half a second intervals.
However, for ...
1
vote
1
answer
80
views
Why is QuestDB using more memory than allocated?
We've been looking at memory for capacity planning and have a Helm-deployed 8GB-limited QuestDB instance running on one of our k8s clusters.
We recently began scraping metrics off of it. I'm trying to ...
2
votes
1
answer
201
views
Why is QuestDB not showing me the data I just ingested?
I am streaming data into QuestDB using the ILP protocol with one of their official clients. I would expect to see the data available immediately after sending, but that's not the case.
If I go to the ...
1
vote
1
answer
137
views
Can we use Tortoise ORM (Python) for QuestDB?
I want to use QuestDB with Tortoise ORM in my python app. Are the connection params same as a basic PostgreSQL connection?
I want to connect DB like this -
await Tortoise.init({db_url=DATABASE_URL, ...
0
votes
0
answers
56
views
Backup of QuestDB failing
After having failed doing a 'backup database' in QuestDB because of storage shortage I am not able to run backups at all anymore, despite now having enough storage for it.
Logfile shows this:
2022-12-...
1
vote
1
answer
98
views
Error "io.questdb.cairo.CairoException: [2] could not open read-write [file=<dir>/_tab_index.d]"
Currently, I am testing QuestDB in a Apache Camel / Spring Boot scenario for our project. I set up a custom Camel component and a configuration bean holding the connection properties. As far as I can ...