Questions tagged [database-performance]
Questions about database performance and tuning, ranging from files organization and configuration to benchmarking and system optimization, passing through management systems and the fastest ways to operate a database.
2,493
questions
0
votes
0
answers
52
views
Postgres slow in Docker [closed]
We have a relatively large Postgres database (1.5Gb sql dump file) which we're spinning up in a Docker container. Running a specific query takes about 1000ms within the container, compared to <...
0
votes
0
answers
33
views
SQL server - way to execute 10 million insert/update command [closed]
I have a sql server and .sql file containing 10 million update statements that I need to execute against the database.
It is taking hours to update 100,000 records.
looking forward to a faster way of ...
1
vote
0
answers
40
views
Why are my apache ignite cache put operations inconsistent in speed?
I am running a local apache ignite docker container with in the default configuration, storing all data into RAM. Writing larger base64 strings (~200kb) with the pyignite cache.put() operation into ...
0
votes
1
answer
33
views
How to make a multi column JSON index in Postgres?
I have a query where I query by a normal text column and a value in a JSON column. What I'm wondering is how to create the best index for the query?
This is the query:
explain select * from "tags&...
2
votes
1
answer
109
views
First query or connection to AWS RDS is very very slow
I have a product built with laravel, with multi-tenancy.
Deployed on EC2 instance and using AWS RDS as the database server.
I am currently having around 100 databases on the production.
Laravel's hyn ...
1
vote
1
answer
53
views
Why SQLite queries run slower in Windows [server] machine compared to Ubuntu & MacOS?
We have 3 machines: One has Windows server OS 2012-r2 installed with decent specs (12 GB RAM, 3.6 GHz, 4 cores, 600 GB hard disk). The others are home laptops with regular specs of Ubuntu 20.04 & ...
0
votes
0
answers
36
views
Snowflake and Bulk Collect
I have been using Oracle DB for a while now and really liked the "Bulk Collect" feature. It made processes faster. Now, I have started using Snowflake. Is there anything similar to "...
0
votes
1
answer
74
views
Move rows older that x days to archive table or partition table in Postgres 11
I would like to speed up the queries on my big table that contains lots of old data.
I have a table named post that has the date column created_at. The table has over ~31 million rows and ~30 million ...
0
votes
0
answers
89
views
Performance regression in PostgreSQL 14.3 compared to PostgreSQL 13.7 on the same query
I migrated a database from Postgresql 13.7 to Postgresql 14.3 recently, and I saw a x2 execution time in a specific case which I can't understand.
The issue has been seen first on Postgres on Linux, ...
0
votes
0
answers
28
views
Oracle on AWS RDS: is performance tuning no longer required?
My customer recently migrated from on-premises to the AWS Cloud. With SYSDBA locked down, I am aware that the Cloud has taken over many of the manual responsibilities -- if not all. When I review the ...
1
vote
2
answers
63
views
PostgreSQL - 100 million records transfer from archive to a new table
I have a requirement to transfer data from 2 tables (Table A and Table B) into a new table.
I am using a query to join both A and B tables using an ID column.
Table A and B are archive tables without ...
0
votes
1
answer
33
views
Dynamically getting data from a oracle table
Need your help in the below scenario.
Table1
Columns
Unique_id number
Name varchar2(20)
desc varchar2(20)
Column_Name varchar2(20) /* ...
0
votes
0
answers
8
views
How much traffic between Foglight and monitored SQL Server?
I have been trying to find the traffic volume that would result from having a SQL Server monitored by Foglight. Foglight servers and my SQL Servers are in separate data centers and the Foglight team ...
1
vote
1
answer
84
views
Postgres select query making sequential scan instead of index scan on table with 18 Million rows
I have a postgres table which has almost 18 Million rows and I am trying to run this query
select * from answer where submission_id = 5 and deleted_at is NULL;
There is an partial index on the table ...
0
votes
0
answers
11
views
Slow Query Occurs During MariaDB Multiple Concurrent Client Bulk Insert
I have a problem to multiple concurrent client bulk insert occur Slow Query.
Can this problem be solved by mariadb tuning?
1
vote
1
answer
33
views
What is the most efficient Data type to store categorical string variable in MySQL
I have a table with about 50k rows and multiple columns.
Some columns have the data type VARCHAR but the store a unique set of values, Categorical strings.
I'm having some performance issues with this ...
1
vote
1
answer
120
views
MySQL query very slow on live db but fast locally and on staging?
execution time
local : 0.0097 seconds
prod : 10 seconds
local and prod have exactly the same amount and value of data
this is my server specification
16 VCPUS / 32 GB RAM / 200 GB SSD Disk
...
1
vote
2
answers
76
views
MySQL is not using composite index with ORDER BY clause
I have a table user that looks like this
id | first_name | last_name | org_id
This table has few million entries.
I want to run the below query with an exact match and an order by clause
...
0
votes
0
answers
27
views
enq: TA - Contention issue in Oracle Database
When there is a peak load in my application several of requests are failing or slowing down due to the above error in Oracle database. When I try to check the top queries ordered by Buffer Gets, I see ...
-1
votes
1
answer
48
views
SQL Server odd Performance results when querying/not querying checking for NULL values
I have to modify a query that is checking that a given date falls between the StartDate and EndDate fields of some View result set.
Since there is no row with null values in those fields, the ...
-2
votes
1
answer
90
views
T-SQL -- faster way to do match for NOT LIKE '%[^0-9]%' used in CASE/WHEN statement
I am looking for a potentially faster way to do this check:
NOT LIKE '%[^0-9]%'
This checks to ensure all characters are numbers (see description of T-SQL pattern)
Is there a faster way to do this in ...
0
votes
2
answers
79
views
Oracle Redo Logs All Active
First, I know very little next to nothing about being an OracleDba. I am responsible for managing an application using Oracle 11g written by a third party. Our application/database has been stable for ...
1
vote
1
answer
88
views
Postgres - same query two different EXPLAIN plans
I have two postgres databases (one for development, one for test). Both have the same structure.
I'm running this query on both (merges one table into another)
WITH moved_rows AS
(
DELETE FROM ...
0
votes
0
answers
17
views
Foglight for Database
I need to evaluate database performance monitoring tool for which we have chosen the product Quest - Foglight for databases.
But looking at their overwhelming website I'm confused which is the actual ...
0
votes
1
answer
146
views
Elastic search, experiencing very low search speed
We have a cluster consisting of 3 masters (4 core, 16 GB RAM each), 3 hot(8 core, 32 GB RAM, 300 GB SSD each), and 3 warm nodes(8 core, 32GB RAM, 1.5TB HDD each).
We have one index for each month of ...
1
vote
1
answer
84
views
Snowflake to SQL Server data load performance
I am trying to improve performance of a SQL query used to move data from snowflake view to SQL Server table. Query uses linked server created using a ODBC driver to connect to Snowflake view from SQL ...
0
votes
0
answers
111
views
Select query hangs when a large number of rows have an update lock
I am designing a program that will read a queue table. I am using locking to make sure that multiple instances do not pull the same row.
I am locking rows like this:
BEGIN TRANSACTION
UPDATE ...
0
votes
0
answers
12
views
Cache a view or some other method to speed up query results
So, I have a table "Data" with several million records from a ERP system based on Maria DB.
There are several text fields and an indexed integer of "Period" of which the numbers 1-...
1
vote
1
answer
57
views
Is pg_stat_user_indexes replicated?
We are running Postgres 9.6.23 on Google Cloud SQL, with the Primary Instance being used for the (transactional) application and a Read Replica being used for analytical purposes.
Querying ...
0
votes
1
answer
33
views
Are Mongoose arbitrary queries as slow compared to DynamoDB Scan?
I am reading in this blog: DynamoDB is bad for analytics. It is true that because you cannot run arbitrary queries against DynamoDB tables (technically speaking you can but it's extremely inefficient ...
0
votes
1
answer
86
views
Problem with Recursive CTE very long query plan
When I execute below query SQL run this plan and it took a long time to run it and it will not be over.
QueryPlanLink
I have 3 million records in #T table.
myCode:
;WITH cte1 AS (
SELECT ...
0
votes
0
answers
18
views
Where to store group entity when shard database by userId in SQL
I always confuse with the entity that has One-to-Many relationship with Sharding key. For example, I have an entity User and another entity Group in a chat system. Then I decide to shard my database ...
0
votes
0
answers
42
views
Optimize a simple sorting mysql query to reduce CPU usage
Below query is taking a very long time to run ( more than 20 seconds) due to sorting which leads to more CPU usage. This is just a straightforward 'select' query with no joins/subqueries. Please let ...
0
votes
0
answers
41
views
Optimize a Group By that is done on a joined table in Azure SQL Data Warehouse
I've been working at optimizing a query that I've been given, I'm curious if there is any way to speed up the following query that I have.
This isn't the exact query that I am using but I think it ...
0
votes
0
answers
53
views
Why calling the Oracle from ODP.Net decrease the throughput of my asp.net webapi core by 1525%?
I am doing some research in order to increase the performance of my asp.net webapi core (5.0) when working with an oracle (12g) database.
I have made two endpoints for tests purpose in this API :
Raw ...
0
votes
0
answers
48
views
Mongodb is slow compared to MySQL on massive update
I'm trying to migrate some MySQL code into mongodb (intermediate to senior in MySQL, beginner to intermediate in MongoDB). I need to perform severals updates an hour that imply all documents in the ...
0
votes
0
answers
43
views
geoNear with Pagination and sorting on created_at
I´m having an App with the capability of listing articles within a user specified location + radius surrounding. See ruby code with mongo query below.
For Web App, it uses pagination based on "...
0
votes
0
answers
62
views
Postgres batch upsert and insert to versions table
I am using Spring Data JPA, together with Spring Boot, and trying to insert rows in my MAIN table, but in case of CONFLICT, old values should be inserted in my VERSIONS table, and row in main table ...
0
votes
0
answers
81
views
Article Listing with Pagination and filters - overall performance - database and code design
I´m a technical project manager coordinating developers. I´ve gone through a - what seems to be - challenging task of having queries for
listing articles
within specified search point + radius
...
1
vote
0
answers
55
views
varchar column is causing distinct order by to be very slow
I have a simple table and query where I am doing:
SELECT DISTINCT ON (security) security, value_date, value_currency, mid_price
FROM prices
WHERE value_date <= date '2022-02-18'
ORDER ...
-1
votes
1
answer
96
views
Poor performance when joining SQL view
I have a table -
TABLE Entries(
[Id] [uniqueidentifier] NOT NULL PRIMARY KEY NONCLUSTERED,
[Timestamp] [datetime2](7) NOT NULL CLUSTERED INDEX,
[PersonId] [uniqueidentifier] NOT NULL,
[...
2
votes
1
answer
150
views
Postgres gin Index for two words search
Aurora Postgres 12.8
Wildcard searches are working perfectly using gin index as under on multiple tables having millions of rows each.
CREATE INDEX index_name ON table_name USING gin (column_name ...
0
votes
0
answers
60
views
SAP / DB2 for LUW 11.1 Finding out process that is creating high number of TLOG entries
I have problem with replicating data from DB2 LUW (that is DB for SAP ECC app).
Replication procedure utilize DB2 low level API DB2 Log Reader.
https://www.ibm.com/docs/en/db2/11.1?topic=apis-...
1
vote
1
answer
56
views
All else held equal, which is the faster querying option: Milvus, RocksDB, or Apache HBase
I have a requirement to store billions of records (with capacity up to one trillion records) in a database (total size is in terms of petabytes). The records are textual fields with about 5 columns ...
1
vote
1
answer
51
views
Best performance in filtering large datasets: = ANY() or @> on an array column; XOR on a bitmapped binary column; join a m2m table or something else?
The General Question:
This question pertains to tables with a large number of rows (say millions) that has a many to many relationship with a relatively small set of data (say tens). For example you ...
0
votes
0
answers
35
views
How to improve SQL execution query plan for script below
I have a slow SQL query which execution plan run mainly on "index scan". I had added index to it which have help improve it to be "index seek". But out of 2 of the index (...
1
vote
1
answer
241
views
How to "PERFORM" a CTE query returning multiple rows/columns?
As a follow-up to this question:
How to "PERFORM" CTE queries in PL/pgSQL?
I try:
perform (with test_as_cte as(select * from myTable) select * from test_as_cte);
But get the following ...
0
votes
1
answer
82
views
NOT EXISTS vs NOT attribute= ANY vs NOT IN
After I read this post, I found a third way to achieve the query with NOT ANY. Does it really makes a difference?
Which of these queries run the faster?
NOT EXISTS
SELECT DISTINCT U.no FROM U
INNER ...
0
votes
1
answer
81
views
High query response time for find operation in mongo 4.0.27 with mmap storage engine
Scenario:
We have application which is doing CRUD operations on database. For past 2 years we have been using mongo 3.6.9 and system has been working fine without any issues. When we upgraded the ...
0
votes
0
answers
52
views
How to detect the cpu-peak-inducing transaction/statements (mostly read/select) from mon$... data?
I can see that the Firebird 2.1 process (on Linux) (for our program) reaches 97% CPU load, the load may be distributed, e.g. the server can have 4 cores and 2 cores are consumed with 97% load and the ...