Questions tagged [cluster-computing]

A computer cluster is a set of connected systems that work together so that in many respects they can be viewed as a single system.

Filter by
Sorted by
Tagged with
1 vote
0 answers
12 views

What does the Aedes packages "aedes-persistence" and "mqemitter" do?

I am using the aedes MQTT broker. I read on this page that to use clustering, you need to mqemitter and aedes-persistence. But I am not sure what those packages do. My guess is that aedes-persistence ...
true_north_404's user avatar
0 votes
0 answers
40 views

Inconsistent PCA Results with Different Multiprocessing Settings in sklearn

I'm encountering an issue with PCA in sklearn while using multiprocessing. Specifically, the reconstruction error in PCA varies significantly based on the number of processes set in Pool. For instance,...
cat's user avatar
  • 59
0 votes
0 answers
14 views

Regarding the consumption of Kafka under the multi-process model of Node.js cluster

I encountered a problem. I used the node.js cluster model to register kafka consumers, but the multi-process did not work. Instead, only one process was continuously consuming. What is the reason for ...
user2774860's user avatar
1 vote
0 answers
26 views

R: Why do I get a Negative CH Index?

So, I'm trying to analyse my data following this tutorial : https://enterotype.embl.de/enterotypes.html At some point I need to determine the optimal number of clusters in my data calculating the CH ...
esti's user avatar
  • 11
0 votes
1 answer
26 views

Run eclipse-mosquitto MQTT broker docker image and custom paho-mqtt Python clients inside minikube Kubernetes cluster

I want to run an MQTT broker and two clients inside a Kubernetes cluster and be able to send messages back and forth. Therefore I wanted to use the eclipse-mosquitto Docker image and two custom ...
Juvojic's user avatar
0 votes
0 answers
18 views

How to use TLS for secure communication in clusters ?

I'm implementing secure communication between clusters. However, there is very little information on this. It is only known that TLS can be used as a protocol for encrypted communication. But how do ...
hpd's user avatar
  • 3
0 votes
0 answers
19 views

How to further partition/split an already partitioned dask dataframe

I have a very large DataFrame, that I partition based on values in one column "A" using the dask.DataFrame.set_index() method. Such N partitions are still too large to fit into memory when ...
Ste's user avatar
  • 53
0 votes
1 answer
63 views

Incorrect Hostname of pods in same stateful set in an AKS cluster

I am at my wits' end here. I have been trying to solve this issue that all of a sudden started happening with my recent deployments to AKS to no solution. I have gone through a lot of different ...
Brian Lui's user avatar
0 votes
0 answers
13 views

Getting com.hazelcast.core.OperationTimeoutException

We are using Hazelcast as embedded in our spring boot application and our application is hosted on 12 tomcat servers forming a single cluster. com.hazelcast.spi.properties.ClusterProperty....
user3702237's user avatar
1 vote
1 answer
34 views

SLURM/NFS cluster shared filesystem locked permissions on some users

Background I used slurm to set up a 3-node cluster, one control node and 3 compute nodes (the control node doubles as a compute node). All of the nodes are on linux mint and they all communicate ...
floating_protein's user avatar
-1 votes
0 answers
27 views

Flutter Cluster Manager per region

In my flutter app I use map service. I have places around let's say Europe. I'm using Cluster Manager for flutter in order to group them. When I zoom out a lot (in country level) they all merged in ...
skalogirou's user avatar
0 votes
0 answers
8 views

Can Apache Mesos Be Used to Cluster Servers Across Departments Without Altering Application Code?

I'm working on an infrastructure project where our organization has several physical servers across different departments, each running its own set of applications and websites. I'm considering using ...
Prama Aditya's user avatar
2 votes
1 answer
38 views

Problem running single-node job on computing cluster in conda environment for all processors in R

I am trying to run a single-node job on my university computing cluster, submitted via qsub, within a conda environment "myenv". As soon as I start my parallelization within my R script, the ...
peony's user avatar
  • 23
0 votes
0 answers
27 views

Cluster configuration test failed in Vivado

l open settings options, and use cluster Configuration. When clicking test Configuration button frequently, it errors occasionally. Prompting content as follows: Job <502467>: job has already ...
kai chen's user avatar
0 votes
0 answers
22 views

Slurm Error: Error in saveRDS(fit, paste("outputs3/fit_", mod, ".rds", sep = "")) : error writing to connection

I have a batch script that I am trying to run. I've made some changes to the original script (which ran and saved successfully) so I wanted to rerun the models and save to a different output folder. I ...
megsruppUNBC's user avatar
0 votes
0 answers
20 views

When I make custom cluster marker on google map via swiftUI, create class CustomClusterRenderer: GMUDefaultClusterRenderer

To override the renderer GMUDefaultClusterRenderer use the following code: class CustomClusterRenderer: GMUDefaultClusterRenderer { override func renderer(_ renderer: GMUClusterRenderer, ...
wenchao zhang's user avatar
0 votes
1 answer
25 views

TLS/SSL certification error when intalling pip packages over SSH HPC clutser in virtual enviroment

I'm trying to install python packages in a virtual venv over HPC cluster. (metaTS_venv) [xxxuser@hpc metaTS]$ pip3 install pandas I get the TLS/SSL error below. It works fine if I install on the main ...
Sadeq Al-Ahdal's user avatar
0 votes
0 answers
8 views

How should I distribute cores for inter and intra parallelization in TensorFlow?

I have been trying to run my neural network training on a cluster computer system. I am given a single node with 128 cores to use. I wanted to run 4 trainings in parallel, each using 32 cores at a ...
fahd's user avatar
  • 141
-2 votes
0 answers
16 views

Cluster and Crop cancer segments from images of the dataset

The dataset is split into train,test,val and each one have 7 cancer subtypes. What i want to do is to cluster each image and save important segments that contains cancer images from the each image. ...
ergun's user avatar
  • 1
-1 votes
0 answers
19 views

Accessing a remote minikube from a local computer

I'm trying to follow this guide in order to set up a cluster that I can connect to from another local machine: https://faun.pub/accessing-a-remote-minikube-from-a-local-computer-fd6180dd66dd#:~:text=...
Adi Jian's user avatar
0 votes
0 answers
9 views

Changing the Active Directory on an existing install of Windows HPC Pack 2019

I have an existing Windows HPC Pack 2019 install that is currently only working with an Administrator login. I want to change it to work with a newly set up Active Directory. Is there a way to do this,...
Bob Moore's user avatar
0 votes
1 answer
17 views

Percona XtraDB Cluster: Automatic Failover Between Nodes in Java Spring Boot Application

I've set up a Percona XtraDB Cluster with two nodes, and my Java Spring Boot application is currently communicating with node1. The challenge is that when node1 goes down, the application doesn't ...
Muhammad Umar's user avatar
1 vote
1 answer
38 views

How to hold jobs for a user so that a total cpu number isn't reached in slurm?

I am submitting a bunch of array jobs 4 sets of 5 with 8 cpus each, so 4x5x8=160 cpus total. I would like to keep the running amount to below 100>2x5x8+1x2x8 cpus total, because I need to let ...
Jason M Gray's user avatar
0 votes
0 answers
15 views

Clustering Words into given Categories in R?

I have a vector of words and two predefined categories: cat1 = c("Money, Commercial, Facility") and cat2 = c("Debt, Liability, Loans"). I'm interested in clustering the words in ...
george1994's user avatar
0 votes
1 answer
35 views

Snowflake - Warehouse Node Utilization data?

I vaguely recalling someone stating that there is at least an "estimated" node usage of a Warehouse in Snowflake. Not a cluster. The nodes of a cluster. For instance a Large Warehouse in ...
user45867's user avatar
  • 899
2 votes
0 answers
61 views

Scala TrieMap.computeHash fails unexpectedly on Spark Cluster, but HashMap works with same code

I am trying to refactor some existing code to run fully in cluster-mode and have been running into an unexpecteded NullPointerException under Scala 2.12.10 when trying to use the TrieMap, see TrieMap....
AlexHB's user avatar
  • 21
0 votes
0 answers
16 views

Wildfly-29 clustering mode Logout Not Working - Session is Still Valid

i am running 2 nodes of Wildfly 29 in clustering mode, all works fine but when i tried to logout in the app, logout not working, the session remains alive. I think the problem is the session cache ...
Juan Santamaría's user avatar
0 votes
1 answer
27 views

Vert.x SharedData - can it reliably store an LRU?

I'm trying to store a list of things in the Vert.x cluster shared data, where the list if kind of like a queue, but things drop off of the queue based on a limit of number of elements - basically the ...
Guss's user avatar
  • 30.9k
0 votes
1 answer
27 views

Redis streams and multi/exec(transaction) in Redis Cluster

I want to publish a message to the Redis stream as part of Redis transaction(multi/exec). In a clustered environment, all data that is part of a redis transaction must be in same shard, so the ...
zoran's user avatar
  • 835
0 votes
1 answer
37 views

Databricks cluster policy reference oracle jar

How can I set a Databricks Cluster policy to reference an oracle jar file? I'm creating a policy via JSON (In the UI I'm not allowed to view the Libraries tab while creating the policy). I've done ...
adrihernandez's user avatar
0 votes
0 answers
27 views

How do I connect to a cluster with remote-ssh on vscode?

I am trying to connect to a ovh cluster using the ssh connection. I can connect but when im connected, the server crash immediatly with this log errors : [13:00:38.785] Failed to parse remote port ...
Emma Kopp's user avatar
0 votes
0 answers
13 views

Spark cluster - Stuck at start-all.sh and stop-all.sh

I am trying to configure a spark cluster, where I have one driver and one executor. After having done all the necessary setup, I am trying to start my cluster using the scripts in .../spark/sbin, ...
Sherwin R's user avatar
0 votes
0 answers
18 views

Problem with Clusters in NodeJS app + Socket.io

I was planning to optimize my Node.js application to distribute the load across all CPU cores, as a Node.js application can only utilize a single core. However, by using clusters, we can make use of ...
mkrdev's user avatar
  • 17
0 votes
0 answers
29 views

Python optimisation library to use on Databricks Spark cluster

I am running a Apache Spark cluster on Azure Databricks, on which I want to run an optimisation algorithm in Python. Are there any optimisation libraries built to work specifically with Spark? I have ...
Emil Doyle's user avatar
0 votes
0 answers
27 views

Cassandra cluster reset the modified value to initial cassandra.yaml configuration value when restart docker container

I'm running docker container with cassandra image. But running on vm occurs changing bridged-ip. I want to modify cassandra.yaml configuration. I tried modify cassandra.yaml configuration with docker ...
Sanghyeon.Jeon's user avatar
0 votes
0 answers
16 views

Spark cluster with different relative path on Dockerfile both for local and remote reference

I forked a spark cluster work. On file below, step ADD build/workspace/ ${SHARED_WORKSPACE}/ works fine on building action master.yml. https://github.com/trouchet/sparkles/blob/master/build/docker/...
Tarzan Django's user avatar
0 votes
1 answer
33 views

failed to find version script file when linking a dynamic library by using bazel cluster

I was building a shared/dynamic library by using bazel. The rules in Build file are like below cc_library( name = "a", srcs = ["a.cc"] ) filegroup( name = "vis_ld&...
James's user avatar
  • 25
0 votes
0 answers
9 views

Docker tomcat scale with loadbalancer

I would like to implements docker-compose for my tomcat application where I can scale all tomcat containers that I need. Does anyone can share with any docker-compose.yml example in order to see how ...
aivdev's user avatar
  • 69
0 votes
1 answer
50 views

Databricks Cluster Spark configuration

How to run code in a Cluster's Spark Configuration. I've tried several settings to get Python to set the default database. Idea is that users don't have to specify the database when writing SQL ...
adrihernandez's user avatar
1 vote
1 answer
87 views

Automatically shift rows with same spatial coordinates into a different cluster

I have a dataframe df: import pandas as pd data = { "latitude": [49.5659508, 49.568089, 49.5686342, 49.5687609, 49.5695834, 49.5706579, 49.5711228, 49.5716422, 49.5717749, 49.5619579, 49....
PParker's user avatar
  • 1,461
0 votes
0 answers
39 views

Snakemake: Handling Preempted Jobs on Slurm: Issues with Job Status and Cleanup

This message follows up on the one posted two years ago (Snakemake: Job preemption can interrupt running jobs on clusters, how to make sure that the task is not considered as failed?). I no longer ...
Thomas Bigot's user avatar
0 votes
0 answers
20 views

How to set resource limits to accounts for each partition in accounting file in slurm

We have SLURM deployed on our cluster with several partitions (part_1, part_2, part_3). We have created several accounts in the accounting file and several users are part of each account. For each ...
Anas's user avatar
  • 1
0 votes
0 answers
17 views

Jupyter Lab Remote Tunnel Link

Is there a way to launch a Jupyter Lab server that is accessible through a direct tunnel link (NOT via SSH) in a similar way that is available in VSCode Remote Tunnels? The use case is when running ...
Guy Gaziv's user avatar
0 votes
0 answers
28 views

Automatic authentication with VSCode Remote Tunnel on a cluster compute node

I came across this helpful post on VSCode Remote Tunnel on a cluster compute node. Is there a way to also automate the authentication (similar to ssh-key) such that one doesn't need to follow the ...
Guy Gaziv's user avatar
0 votes
0 answers
54 views

Segmentation Fault When Sending Arrays Over a Certain Size with Open MPI

I am writing a program to run with an ifiniband and intel-based cluster using openmpi, pmix, and SLURM scheduling. When I run my program on the cluster with an input matrix over 38x38 on each node, I ...
Another Shrubbery's user avatar
0 votes
2 answers
66 views

Dask Distributed - Stateful global parameters shared by worker methods

I am using Dask to setup a cluster. For now I am setting up both the scheduler and the workers on localhost. cluster = SSHCluster(["localhost", "localhost"], ...
Jurgen Cuschieri's user avatar
1 vote
1 answer
219 views

Move everything from a k8s cluster to a new one

Is it possible to move every deployment on a running k8s cluster to another running k8s cluster? I don't need to move cluster configurations, just every deployment, its pods, secrets, services, ...
simple-thomas's user avatar
0 votes
0 answers
19 views

How to divide the indices to determine the ideal number of clusters in 3 groups?

How to divide the following indices to determine the ideal number of clusters in 3 groups: internal cohesion (intracluster similarity), distance between cluster (intercluster dissimilarity) or both? ...
César Macieira's user avatar
1 vote
1 answer
26 views

killing child process in Node.js

If the child process is caught in an infinite loop, the 'exit' event is never handled. How do you force a worker to stop in this case? "use strict"; (async () => { const cluster = ...
user3515941's user avatar
0 votes
0 answers
18 views

How to color branches of dendrogram [duplicate]

I am trying to color the branches for a dendrogram plot in order to better visualize similar and dissimilar years for my counts. This is my current code I have: library(dendextend) BBbray_root.cluster ...
user390865's user avatar

1
2 3 4 5
111