Questions tagged [ray]

Ray is a library for writing parallel and distributed Python applications. It scales from your laptop to a large cluster, has a simple yet flexible API, and provides high performance out of the box.

Filter by
Sorted by
Tagged with
0 votes
0 answers
8 views

ProcessPoolExecutor and ray

I have written some code that follows the following structure. Basically I am accessing some files from the disk and I am doing some image processing operations on them. The program below makes a pool ...
Aenaon's user avatar
  • 3,261
0 votes
0 answers
8 views

Only 40% of CPU usage with Ray / modin.pandas

I am manipulating some data from dataframes(650MB) so I am using parallel processing in python. The problem is that I thought that it would require CPU usage of around 80-90% but it is requiring only ...
GioTTV_PyBrazil's user avatar
-1 votes
0 answers
27 views

Ray Dataset loading HDFS: Failed to close HdfsReadableFile: IOError: HDFS CloseFile failed. Detail: [errno 255] Unknown error 255

I am using Ray Dataset read parquets on the HDFS cluster. import ray test_dataset = ray.data.read_parquet('hdfs://[email protected]:9000/path/to/file.parquet') test_dataset.take(1) When reading ...
David Lee's user avatar
0 votes
0 answers
44 views

Synchronize many ray.remote functions in python

I'm working on a project where multiple nodes (let's say 4, arranged in a grid) are engaged in parallel computations across a time span from tau = 0 to 2. Each node reads its own and its neighbors' ...
Vii's user avatar
  • 1
0 votes
0 answers
12 views

Checking if TorchTrainer is using the available GPUs

I am trying to train on a given dataset using distributed training in ray. I ran a sample code: import os import tempfile import torch from torch import nn from torch.nn.parallel import ...
Prabhjot Singh Rai's user avatar
0 votes
0 answers
29 views

pandas value_counts in ray

I am pretty new to Ray and trying to use it for some data analysis operations on very large datasets Lets say I have data in the following format score_A score_B 1 2 2 3 1 2 2 3 7 2 5 11 pd[&...
Ash's user avatar
  • 396
0 votes
1 answer
10 views

Ray tranformation with column dtype list<element: double not null>

My dataset schema is the following in ray: Column Type ------ ---- x_1 list<element: double not null> x_2 list<element: double not null> y int32 Right now if I try to ...
Prabhjot Singh Rai's user avatar
0 votes
0 answers
55 views

How can I launch a ray server in a linux/amd64/x86_64 docker container running on a macOS arm64/M1/Apple Silicon - without crashing?

I have a ray server running successfully inside a docker (docker engine 24.0.6) [docker desktop: 4.25.2 (129061)] container on my M1 (Sonoma 14.1) as arm64. On the M1, I need to create - and ...
jtlz2's user avatar
  • 7,857
0 votes
0 answers
31 views

What is the correct technique for incorporating access to large machine learning data sets in a Ray Tune Tuner() object?

I'm attempting to use Ray Tune 2.8.0 to optimize the hyperparameters of a pytorch neural network model--a fairly standard use case as far as I can understand. Pytorch is distributed with several ...
stachyra's user avatar
  • 4,463
0 votes
0 answers
5 views

How to vectorize an RLlib MultiAgentEnv similar to Gym/Gymnasium's VectorEnv?

I have a multi-agent RL environment that's implemented as an RLlib MultiAgentEnv. I would like to do training and analysis with this environment but with my own custom algorithm code and without the ...
Yuerno's user avatar
  • 773
1 vote
0 answers
11 views

Ray tune and non-CPU-bound tasks

I like the interface and functionality of Ray Tune. However, I've observed a large per-task overhead when I use non-CPU-bound tasks (the actual compute is done on a 3rd party service). I understand ...
Grachev Alexey's user avatar
2 votes
0 answers
42 views

Attribute Error initialising Ray in Python

My python environment details are below. This is a conda virtual env that I created just to test ray core as on this Ray getting started tutorial. Initialising Ray throws the error: "...
Zhubarb's user avatar
  • 11.6k
0 votes
0 answers
21 views

Ray task is not cancelled

I am running task in Ray but I am unable to cancel it; the task is still running from Ray dashboard, noting that run_ray_task and cancel_ray_task are called in separate APIs and no errors. or ...
Mais's user avatar
  • 1
1 vote
0 answers
108 views

"protocol version mismatch -- is your shell clean?" when using docker with ray

I am getting the following error when using docker with ray: Digest: sha256:2c0f86c6209e02d1f9e3d0a611ac04d73c3ed132b8b9319a28e2069f1503fb53 Status: Downloaded newer image for us-west1-docker.pkg.dev/...
Prabhjot Singh Rai's user avatar
0 votes
0 answers
32 views

Ray head node fails to connect when running ray up

I am trying to configure ray cluster with the following cluster.yaml configuration. For GCP, I am required to input the image. With 50gb images, the head and worker nodes start easily but with 10GB ...
Prabhjot Singh Rai's user avatar
1 vote
1 answer
29 views

PermissionError when ray.init()

I cloned a conda environment to new computer, and run a ray project, but face the error. I find this error occurs when ray.init(), the details codes and info are follows: >>> import ray >&...
BobCheng's user avatar
0 votes
0 answers
25 views

Ray inferencing not happening in streaming way

I am using ray for inferencing. scale = ActorPoolStrategy(size=100) scale_gpu = ActorPoolStrategy(size=4) preprocessed_ds = ds.map_batches(CPUPreprocess, compute=scale, num_gpus=0, batch_size=...
humble's user avatar
  • 2,086
1 vote
1 answer
98 views

python ray out of memory (OOM) with RAY_memory_monitor_refresh_ms set to 0

I'm using Modin to use pandas code that can't fit in memory. It works very well locally with dataset (30GB) and my RAM(16GB). Now I want to speed this up and decided to run a cluster on GCP with modin ...
tsh's user avatar
  • 897
0 votes
0 answers
43 views

Function Parameters in Customizing Models in Ray RLLib

How severe does this issue affect your experience of using Ray? High: It blocks me to complete my task. I started using Ray RLLib (version 2.7.1) to solve a RL problem with customized environments ...
Yiming Dong's user avatar
0 votes
1 answer
36 views

Python Ray - explanation of await queue.get_async(block=True, timeout=None)

Ray.util.queue.Queue.get_async has block argument. Why async method blocks as async means return without blocking, in my understanding. Looked for an explanation in the user guide but not found a clue ...
mon's user avatar
  • 19.8k
-1 votes
1 answer
86 views

Issue with Multi-Agent Environment Observation Space in RLlib

I'm working on a multi-agent environment with two agents (nodes), where each agent has to decide whether to transmit or not based on their observations of their own capacity and the capacity of the ...
Laura's user avatar
  • 1
0 votes
0 answers
38 views

Ray-tune log results in two folders

I using ray-tune to fine-tune parameter. Based on the docs, ray-tune log the results of each trial to a sub-folder under a specified local dir, which defaults to ~/ray_results. To change ray_results ...
LearnToGrow's user avatar
  • 1,678
0 votes
1 answer
47 views

Is it necessary to use object store in python ray.serve?

In python's ray.core, you can use an object store that can be shared by each Actor as follows. import numpy as np import ray @ray.remote def Actor(a): return a = np.zeros((5000, 5000)) ...
user60150's user avatar
0 votes
0 answers
95 views

Ray Tune: Load a checkpoints and continue to train PPO

I train a ppo in the following way, with my config, custom Env and action masking: config = ( ppo.PPOConfig() .environment(env=CustomEnv)) tuner = tune.Tuner( "PPO",...
aondra's user avatar
  • 1
1 vote
1 answer
55 views

In Ray, does worker node system logs get streamed back to header node or only stay in worker node by default

I noticed system logs on worker node become unavailable when the node is gone, suspecting the log only stays in worker node. Is that correct? If so, how does the worker node becomes available on Ray ...
zyxue's user avatar
  • 8,040
0 votes
0 answers
28 views

Python Ray Cluster, tell Node A to never take LAST Task in Job

I have a 3 node cluster for Ray. Node A - 1 cpu Node B - 1 cpu Node C - 1 cpu And 25 tasks. If all things were equal, then A,B,C runs roughly 8 tasks and the last one is picked up by whoever finishes ...
user1639926's user avatar
0 votes
0 answers
62 views

Can not use ray datasets map_batches function to update database (a class object obtained by langchain)?

I want to update the database (a class object obtained by langchain.FAISS) with new data use ray. But I can not obtain the updated database even if it has been updated in the function. This is a demo: ...
waking's user avatar
  • 1
0 votes
0 answers
208 views

Ray for numpy multiprocesing

I am trying to use Ray for multiprocessing on CPU and GPU Unfortunately, i cannot share the whole dataframe but here is a replica of my dataframe. import pandas as pd import numpy as np issues_df=pd....
ricardo's user avatar
  • 166
0 votes
0 answers
40 views

Ray data read does not seem to run in parallel

When I materialize this, dataset = ray.data.read_csv(s3_bucket_name+'/'+s3_folder_path_prefix+'/',partition_filter=partition_filter,filesystem=s3,convert_options=convert_options) it works but the read ...
Dhruva Kartik's user avatar
1 vote
1 answer
68 views

TypeError: buffer is too small for requested array

I am using Ray 2.6.3. I would like to rename a column in a dataset. I am able to run map properly without my data column of images. However, the buffer is too small for the array when I include data. ...
jameszp's user avatar
  • 57
0 votes
0 answers
75 views

Transformers pipeline with ray does not work on gpu

I am following the documentation for offline batch inferencing using huggingface on gpu https://docs.ray.io/en/latest/data/batch_inference.html#using-gpus-for-inference I get a warning You seem to be ...
humble's user avatar
  • 2,086
0 votes
0 answers
216 views

Using ray on Databricks I got error ArrowInvalid: URI has empty scheme: '~/ray_results'

On a databricks notebook I'm running %pip install jinja2 --upgrade %pip install mlflow --upgrade %pip install pycaret==3 catboost optuna lightgbm hyperopt tune-sklearn ray[tune] %pip install pandas==1....
Carlo's user avatar
  • 395
1 vote
0 answers
13 views

Using one Neural Network model inside of ray.pool

I am concerned about using Pool() from ray library. This is simplified structure of my question: import torch import multiprocessing as mp from ray.util.multiprocessing import Pool def acquire_data(...
Vukasin S's user avatar
0 votes
0 answers
44 views

Issue with parallel estimation of Transformer embeddings

I am attempting to compute BERT embeddings for a list of sentences using the Ray parallelization framework. The objective is to speed up this computation by running each sentence list (item contained ...
akd's user avatar
  • 25
0 votes
0 answers
76 views

Scaling Pytorch training on a single-machine with multiple CPUs (no GPUs)

What is the best way to accelerate PyTorch training on a single machine with multiple CPUs (NO GPUs)? We need to speed up training for a customer, because the training dataset grew substantially ...
movingabout's user avatar
1 vote
0 answers
38 views

Ray task in nested loop results in out of memory

I'm trying to run a ray task in nested loop. Each loop takes dataA which has 5000 rows of list. In ray task, it takes dataB and dataC that are 5000 x 256, 5000 x 768 respectively. Whenever ray task ...
Chan's user avatar
  • 31
1 vote
1 answer
169 views

Parallelize DeepFace on multiple GPUs

I am trying to use the DeepFace python library to do face recognition and analysis on long videos: https://github.com/serengil/deepface. Using the library out of the box, I am able to get desired ...
jameszp's user avatar
  • 57
1 vote
0 answers
154 views

Error while running ray function - The task's local raylet died. Check raylet.out for more information

I am trying to run Ray from a docker container on Mac M1 (ubuntu:20.04, built via the command docker buildx build --platform linux/x86_64 -t dev_container -f Dockerfile.dev .) ray.init() is working ...
Dipu Cs's user avatar
  • 23
0 votes
0 answers
65 views

Ray Tune Hyperparamter Optimization with XGBoost - TuneNoNextExecutorEventError| RayActorError

I simply would like to tune my XGBoost with Ray Tune locally. However, my "actor" always dies unexpectedly before the ray trail could even start. I tried different code variants and also ...
Carsten's user avatar
  • 35
0 votes
2 answers
133 views

Running Ray in docker

I'm trying to run ray in a docker image that looks like this: FROM python:3.10.5-bullseye RUN pip install -U "ray[default]==2.6.3" ... CMD ["ray", "start", "--head&...
pcauthorn's user avatar
  • 360
1 vote
0 answers
40 views

Error while testing keras_custom_model.py of ray rllib

I'm encountering a persistent issue while working with the Tune library in Ray RLLIB with Python on a Windows system. I'm attempting to run a test script from the library titled: custom_keras_model.py ...
Nadir Bakyac's user avatar
0 votes
0 answers
16 views

How train model using tensorflow and rayTune then upload best model and its log to S3

I'm using raytune to do hyperparameter optimization with tensorflow model. What I want to do is after training, upload model.h5 and its training log upto S3. My code is something like this: tuner = ...
haneulkim's user avatar
  • 4,608
0 votes
0 answers
88 views

How to use Ray Clusters to read a parquet and run a computation

I'm using Dask to read two parquet files, compute them, then merge them. However, I'd like to use Ray Clusters to achieve this, in hopes of speeding up the computation; how can I change the code to ...
Akham01's user avatar
1 vote
0 answers
172 views

Implement PPO agent using RAY on custom Gym environment

I've been interested in Reinforcement Learning for a while. I've made a simple PPO agent, using Stable Baselines 3, PyTorch and Gym to trade Bitcoin using Binance data. This was relatively easy and ...
Max van kekeren's user avatar
1 vote
2 answers
158 views

Running Ray on top of AWS Batch multi-node?

I am interested in running Ray on AWS Batch multi-node. This is a pattern that hasn't been done before on Ray, and thus, there's no documentation on it. But, I'd really like to try it since Ray can be ...
Riley Hun's user avatar
  • 2,613
0 votes
0 answers
34 views

Large Scale Hierarchical Agglomerative Clustering With Custom Distance Function/Similarity Matrix

I’m working on a project where I need to run hierarchical agglomerative clustering on between 1 million and 10 million data points. I also need to use a custom distance function (I cannot use ...
Michelle_J's user avatar
1 vote
1 answer
208 views

How to end episodes after 200 steps in Ray Tune (tune.run()) using a PPO model with torch

I'm using the following code to import a custom environement and then train on it: from ray.tune.registry import register_env import ray from ray import air, tune from ray.rllib.algorithms.ppo import ...
Dawid's user avatar
  • 13
0 votes
0 answers
29 views

XGBoost RayTaskError

I am trying to do model training and inferencing in Databricks clusters using the Ray framework. I have the following example with the xgboost-Ray example. I have generated ray dataset from parquet ...
Vusal's user avatar
  • 11
0 votes
0 answers
64 views

How to configure Grafana dashboard uid for Ray dashboard?

The default uid is rayDefaultDashboard (e.g. https://github.com/ray-project/ray/blob/ray-2.4.0/dashboard/client/src/pages/metrics/Metrics.tsx#L201-L202), is that configurable? E.g. I'll have two ...
zyxue's user avatar
  • 8,040
0 votes
0 answers
82 views

How to use ray to efficiently run inference onnx run time?

I am running my inference scripts with multiprocessing currently which takes up a lot of memory with loading models with onnxruntime so I am trying to use ray run inference but getting issue with ...
Abhinandan 's user avatar

1
2 3 4 5
15