Questions tagged [reinforcement-learning]
Reinforcement learning is an area of machine learning and computer science concerned with how to select an action in a state that maximizes a numerical reward in a particular environment.
2,641
questions
-1
votes
0
answers
9
views
Why is this PPO agent not able to learn its task?
I am currently trying to learn the leg coordination of a hexapod robot, meaning it should learn when to lift its legs so that an efficient gait(tripod, wave, etc...)emerges.
I am relatively new to RL ...
0
votes
0
answers
4
views
Reinforcement learning dqn algorithm does not forge to target value
I am working on a study project with stable baselines3 and webots. I let a vehicle drive in a simple round course. The vehicle has sensors like a camera, distance sensor, the current speed and the ...
0
votes
0
answers
11
views
DQN reinforcement learning for function optimization
I am trying to write a pytorch DQN Reinforcement Learning python script to optimize a simple function of two variables. I am failing in making it converge.
Can you tell me what is wrong in my code ?
...
0
votes
0
answers
12
views
How to modify the MuJoCo environment (InvertedPendulum)?
I am working on a project that involves modifying Mujoco environments for reinforcement learning. I want to create a new environment that is similar to the InvertedPendulum environment, but with a ...
0
votes
0
answers
14
views
How can I understand whether my coded SARSA Algorithm works?
'''
Q-Table:
State (0, 0, 0): Action (0, 0, 0) -> Q-value: -1377.5596445225826
State (0, 0, 1): Action (0, 0, 0) -> Q-value: -1620.7411224452567
State (0, 0, 2): Action (0, 0, 0) -> Q-value: -...
0
votes
0
answers
11
views
Hugging Face Deep RL Course: Huggy Environment on M1 Mac
I am starting the Hugging Face Deep Reinforcement Learning Course. I am following this tutorial locally on my M1 Mac instead of in Google Colab. I am able to do all of the steps until the training ...
0
votes
0
answers
20
views
Tensorflow tf.random.normal() causes RAM usage to keep growing
Issue: Running TF 2.13.0, Python 3.8.3, I'm getting what looks like a memory leak from running the following simple code, in eager mode.
Context: I'm running RL algorithms using a TF-addon NoisyDense ...
0
votes
0
answers
4
views
RLlib RecurrentNetwork and Trajectory View API
I've been reading over the AttentionNet file for both the GTrXL and AttentionNetWrapper implementations located at ray/rllib/models/torch/attention_net.py at master · ray-project/ray (github.com).
A ...
0
votes
0
answers
5
views
How to create a 2D environment using Utility and deploy Reinforcement learning agents onto it?
I'm currently working on a project where I aim to create a 2D ant colony simulation within a gaming engine using Utility Hub. My goal is to implement reinforcement learning techniques for swarm ...
0
votes
0
answers
8
views
StableBaselines3 - How to create callback on env.reset()
I am training an agent on a custom environment using the PPO implementation from stable_baselines3. Is there a way to create a custom callback that is executed after every reset of the environment ...
0
votes
0
answers
11
views
I need finrl 0.3.1 version repo
There is a finrl=0.3.1 which is used by quantconnect platform. I want to check that file and contents repository. Where should I get all this file?
found on quantconnect docs
I searched on github but ...
-1
votes
0
answers
9
views
Input parameters for training in tensorflowjs
I'm trying to train a model using tensorflowjs. The goal is to play a simple tic tac toe. I'm adapting a code from a tutorial but some things are unclear to me.
I play a move, then I let the agent ...
0
votes
1
answer
29
views
Node: 'sequential/dense/MatMul' In[0] and In[1] has different ndims: [4] vs. [4,24]' error of keras.model.predict() using cartpole exgample
I try to train a cartpole, an example of reinforcement learning, using DQN.
But I got a running error.
At this point
target = self.model.predict(states)
I can't figure out why this is happening, even ...
-1
votes
0
answers
13
views
A n choose k combinatorial optimization problem
I have a n-by-m boolean matrix A (whose elements are either 0 or 1).
I want to select k rows from A to form a submatrix B (size of B is k-by-m).
I compute the elementwise-product for each column of B ...
-1
votes
0
answers
17
views
Reinforcement Learning SARSA Temporal Difference Method is not converging
I've been trying to study Reinforcement Learning and ran into the concepts of model-free methods like SARSA and Q-Learning
I tried to make a Sarsa application using google colab. It doesn't give any ...
-3
votes
0
answers
9
views
How to formulate a reinforcement learning model for human feedback text summarization [closed]
I want to propose an approach for text summarization in which user gives feedback to each of the sentences of the generated abstractive summary and through this interaction the model learns user ...
-1
votes
0
answers
28
views
Debugging the execution of reinforcement learning code in python
I am getting below warning during executing the code:
C:\Users\learner\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.1
0_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-
packages\gym\...
0
votes
0
answers
5
views
Is there a way in Stable baseline 3 to introduce decay parameter in the gaussian noise used in DDPG implementation?
Is there a way in Stable baseline 3 to introduce decay parameter in the gaussian noise used in DDPG implementation ?
I want the noise whether the OU Noise or Gaussian noise to decay with the episodes ...
0
votes
0
answers
20
views
shape mismatch in deep learning model
I have the following code :
import tensorflow as tf
from keras.layers import Dense, Flatten
import gym
from rl.agents.dqn import DQNAgent
from rl.policy import BoltzmannQPolicy
from rl.memory import ...
0
votes
0
answers
16
views
Building task scheduling environment for training DQN agent
description:
I have a dataset which includes task requirements, nodes specifications, and resources usage. I need to use this data to train DQN agent to get optimal task scheduling. How can I build ...
0
votes
0
answers
19
views
RuntimeError: Index tensor must have the same number of dimensions as self tensor
I am running a deep reinforcement learning code and I got this error:
RuntimeError: Index tensor must have the same number of dimensions as self tensor
The code snippet that is related to this error ...
0
votes
0
answers
24
views
OpenAI Gymnasium vectorized approach works slower than non-vectorized
I'm getting started with OpenAI Gymnasium. My task is to speed-up trajectory generation. So to create N trajectories, I want to use a single parallel multi-env instead of running the same env N times ...
0
votes
0
answers
14
views
Error when converting CartPole Numpy arrays for PyTorch Tensors
Reposting this since I think my previous post rambled and did not get to the point right away.
I am studying Deep Q-Learning algorithms and I am following along with a Udemy instructor who is showing ...
0
votes
0
answers
23
views
How to use actor-critique method in reinforcement learning to calculate slope when there are multiple behavioral probabilities, and more questions
I am trying to run reinforcement learning with actor-critic methods
Link: https://github.com/keras-team/keras-io/blob/master/examples/rl/actor_critic_cartpole.py
The agent takes as input a map of size ...
-2
votes
0
answers
7
views
Double Q learning convergence
Can we expect that, in Double q learning, the two q tables converge together? which means that max(|Q1-Q2|) per step gradually converge to zero, Can we say that?
-2
votes
0
answers
19
views
How to make a RL environment with a datafram
I'm trying to learn implementation of reinforcement learning.
my data set is NSL-KDD and I already have normalized it in a pandas dataframe. now I want to use it as my RL environment but I have no ...
-1
votes
0
answers
12
views
Stable baselines 3 explore observation space before starting
I have a really huge obs space like a big room maybe 20 by 20 meters! In this room there are blocks which my agent shall pickup and put on defined locations! I am aware of framestacking but as the ...
0
votes
1
answer
20
views
Stop Stable baselines learn method when execution is terminated
I am using the satble-baselines3's "learn" method to train a model. The thing is that the rewards in my agent are negative and I want the agent to have the urge of stoping as soon as ...
0
votes
1
answer
42
views
How to record and save video of Gym environment
I have trained a DQN agent and I want to record and save a video of a single episode using gym.wrappers.RecordVideo. Here's my code so far:
from gym.wrappers import RecordVideo
env = gym.make("...
-1
votes
0
answers
37
views
How to create a responsive environment for a reinforcement learning problem when you don't have the model?
I want to solve a scheduling optimization problem using reinforcement learning. I don't have the model of the environment and as far as I know reinforcement learning does not need one. My question is: ...
0
votes
0
answers
16
views
keras-gym library has assertion issue
I have posted already two question about frameworks for reinforcement learning and did not get an answer. This indicates two things:
either on this website there are few people who knows materials ...
-1
votes
0
answers
12
views
How to get a dataset for more complex environments for trying to train a DQN and find what hyperparameters/ optimizers to use?
In videos like this, it talks about how you need to find the right fit for your data which is shown on a scatter plot. I understand how this works when you have a dataset for something but how does it ...
0
votes
0
answers
18
views
Can't save a dqn model using a custom environment
I was trying to make a reinforcement learning ai for a chrome dino game. I was following a guide on this link: https://www.youtube.com/watch?v=vahwuupy81A, but changed something when I had errors. The ...
1
vote
0
answers
15
views
Can you explain to me how to apply Policy updates in a Policy Gradient algorithm ? (Python)
I code a Policy Gradient algorithm in python with Tensorflow and I would like to know how to modify the neural network (updating the Policy).
My current code:
import tensorflow as tf
from random ...
0
votes
0
answers
29
views
How to check if a model is being correctly learnt in stable baselines
I am trying to create a model using stable baselines.The thing is that the model is not apparently improving, so I am concerned I may not be storing the model as I should. What I do is the following:
...
0
votes
0
answers
14
views
Issue with ACME's Distributional Reinforcement Learning Implementation in TensorFlow
I've been using DeepMind's ACME library for some of my Reinforcement Learning projects and I've come across an issue that I'm hoping someone can shed light on.
Libraries and Environment
Python 3.9
...
1
vote
0
answers
25
views
How can I write the output stream of sddp.jl into an excel file?
I am using sddp.jl for my research project in which I am developing a state-of-the-art actor critic algorithm which I am going to benchmark with SDDP but for it I need to plot graphs which require ...
-1
votes
0
answers
22
views
Is there anything wrong with my update policy network function or my DQN in my deep-q-learning
I want to train deep q-learning to solve a Rubik's cube given 10 possible moves (I am implementing this for an Arduino project and don't have access to every side of the cube which is why I only ...
0
votes
0
answers
23
views
Contextual Bandit vowpal_wabbit training dataset validation
I am currently using the Vowpal Wabbit package in order to implement a Contextual Bandit use case.
My use case is to provide categories(L1/L2/L3/L4/L5) considered action here with personalized ranking ...
-1
votes
0
answers
24
views
model-based reinforcement learning algorithms for continuous states and continuous actions
Are there model-based reinforcement learning algorithms for continuous states and continuous actions? I was trying to find such algorithms but couldn't find any. I know dynamic programming is for ...
0
votes
0
answers
8
views
eMBB and uRLLc services
`I have one base station runs two services:embb and urllc services. I want to desgin this case in Reinforcement learning. I have the following enviroment but i failed to slice it into two independent ...
0
votes
0
answers
21
views
How to perform gradient descent on a keras neural network with two outputs
I have a network that takes two inputs, the first input is passed through a series of convolutional layers, before being flattened and concatenated with the second input. The result is then passed ...
0
votes
1
answer
26
views
Stable Baselines - model.predict: how to select the GPU
I'm using Stable-Baselines3 and I'd like to test a model I've just trained.
How can I assign a specific GPU to the execution of the model.predict method?
0
votes
0
answers
17
views
Sensor shape must match. ml-agents, Unity
I am using unity and ml-agents, and have started modifying this project: https://github.com/CoderOneHQ/ultimate-volleyball
I have a added more observations to my agents, where it used to have 11 ...
0
votes
0
answers
16
views
Error while testing keras_custom_model.py of ray rllib
I'm encountering a persistent issue while working with the Tune library in Ray RLLIB with Python on a Windows system. I'm attempting to run a test script from the library titled: custom_keras_model.py
...
1
vote
2
answers
63
views
Exception encountered when calling layer and 'KerasTensor' object is not callable
I'm newbie to reinforcement learning.
I'd like to see and understand the code that predicts Keras actor critic value, and then run it with some changes.
Example code: https://github.com/keras-team/...
0
votes
0
answers
35
views
SB3 - AttributeError: 'DummyVecEnv' object has no attribute 'get_action_meanings'
When I try to combine the SB3 vec_env with AtariWrapper, I get an error -
import gymnasium as gym
from stable_baselines3 import PPO
from stable_baselines3.common.env_util import make_vec_env
from ...
0
votes
0
answers
24
views
What can I do to fix my "ValueError" in my Deep-Learning Mario project
I have been following a tutorial for how to make a RL Mario agent. I am using the libraries, gym_super_mario_bros and nes_py, and when I run my code I received an error.
In the tutorial, the guide ...
0
votes
0
answers
14
views
Compatibility Issues with CityLearn 2.0b4, Gym 0.26.1, and Stable Baselines 3 2.0.0
I am currently working on a project involving CityLearn (version 2.0b4), Stable Baselines 3 (version 2.0.0), and Gym (version 0.26.1), and I have encountered an issue that I'm struggling to resolve.
...
0
votes
0
answers
11
views
Is 'observation' method in 'ObservationWrapper' class of gym lib implicitly called?
I found a problem during debugging the code of reinforcement learning model based on large language model (LLM). Generally, majority of plugins or other software for research on reinforcement learning ...