Questions tagged [stable-baselines]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
7 views

Observation Space when using Custom State Objects in stable-baselines

I am attempting to define a custom environment for my RL problem in stable-baselines. Each state is represented by a custom data type, with the space consisting of trillions of these objects, and the ...
0 votes
1 answer
40 views

Stable Baselines 3 support for Farama Gymnasium

I am building an environment in the maintained fork of gym: Gymnasium by Farama. In my gym environment, I state that the action_space = gym.spaces.Discrete(5) and the observation_space = gym.spaces....
  • 687
0 votes
0 answers
14 views

Masking Invalid Actions when using Stable Baselines

I am attempting to set up an environment for an RL problem using stable-baselines. Assume that the action space consists of several groups of actions as follows. Group 1: 30 actions Group 2: 30 ...
0 votes
0 answers
10 views

How to make the rollout of a stable baseline scroll up and eval appear

I'm doing some agent training with stable baseline3, but rollout doesn't scroll up and eval doesn't appear, I'm new to this, please advise more. I want the program to work.
  • 1
0 votes
1 answer
25 views

Create Custom Policy using stable baseline3

I am trying to create a custom lstm policy. It seems that BasePolicy is missing. How can we create a custom LSTM policy to pass to PPO or A2C algorithm. Also, if not, can modify the layer of lstm in ...
0 votes
1 answer
44 views

A2C and stable_baselines3

I´m trying to use this code from a repo in GitHub (https://github.com/nicknochnack/Reinforcement-Learning-for-Trading-Custom-Signals/blob/main/Custom%20Signals.ipynb) in Point 3: model = A2C('...
-1 votes
0 answers
40 views

Baseline Wandering in ECG Graph

Hello I am an iOS Developer. We have an application to plot ECG graph using signals from our ECG device. In my application we plot an ECG Graph but it causes the problem of baseline wandering which is ...
0 votes
1 answer
55 views

stable_baselines3 PPO won't learn

I've written code for moving a target object towards a specific goal position on a 2D surface. However, when I run it, the cart doesn't move towards the object or the goal position at all. It just ...
  • 1
0 votes
0 answers
41 views

How to implement a _train_step method for RecurrentPPO in SB3-Contrib to perform Continual Learning?

I want to add a _train_step method to RecurrentPPO from SB3-Contrib (see here : https://github.com/Stable-Baselines-Team/stable-baselines3-contrib). This method was available in hIll-a Stable ...
  • 51
0 votes
0 answers
49 views

How to schedule learning rate decay based on metrics for stable baselines 3?

I want to write a learning rate schedule based on logged metrics. The API's doc provides example for linear decaying (https://stable-baselines3.readthedocs.io/en/master/guide/examples.html#learning-...
  • 51
0 votes
0 answers
18 views

How to get Q value for each action in stable-baselines 2

Assuming I have a well-trained DQN model and observations. How can I get the Q-value for each potential action by input the current observations. I am expecting something like the following: model = ...
0 votes
1 answer
60 views

Resume training for stable baseline model PPO

I wonder if I can resume training from checkpoints after saving logs in stable baseline as from what I understood from the documentation that CheckpointCallback can work as snapshots (yet not sure ...
0 votes
1 answer
27 views

Stable Baselines 3: Default parameters

I am running some simulations using PPO and A2C algorithms from Stablebaselines3 with openai-gym. I know that i can customize all of them, but i was wondering which are the default parameters. ...
  • 3
0 votes
0 answers
33 views

How to make sense of stable baselines 3 utils and vectorization regarding shapes

I want to setup and pre-process a gym environment with stable baselines 3 and then write my own code for the agent but I have trouble making sense of stable baselines documentation. This leads to not ...
0 votes
1 answer
126 views

Openai gym window wont close on env close

I am trying to implement simple cart pole code but pygame window doesnt close on env.close and freezes. When i try to manually close, it is restarting kernel. This is my code : env = gym.make("...
0 votes
1 answer
133 views

Custom Network and Policy in Stable-Baselines3

I am attempting to create a small working example of how to use MultiDiscrete actions spaces together with a Box observation space. One of the problems that I have run into is that the dimension ...
  • 61
0 votes
1 answer
54 views

Comparison of trained model in stable baseline 3 with another policy

I want to compare a trained model in stable-baseline 3 (SB3) with another one ( a base algorithm) and see how it performs on the same episode. However I am having issues with the evaluate_policy ...
  • 38
0 votes
0 answers
23 views

stable_baselines3 callback on each step

I am training a stable_baselines3 PPO agent and want to perform some task on every step. To do this, I'm using a callback CustomCallback with _on_step method defined. But it appears that _on_step is ...
0 votes
0 answers
163 views

Why using GPU in Stable Baselines 3 is slower than using cpu?

When training the "CartPole" environment with Stable Baselines 3 using PPO, I get that training the model using cuda GPU is almost twice as slow as training the model with just the cpu (both ...
  • 1
0 votes
1 answer
36 views

StableBaselines3 / PPO / model rollsout but does not learn?

When a model learns there is: A rollout phase A learning phase My models are rolling out but they never show a learning phase. This is apparent both in the text output in a jupyter Notebook in ...
1 vote
1 answer
72 views

Do I need to retrain reinforcement model from scratch each time I want to use it in practice?

This seems like it should be obvious but I can't find resources on it anywhere. I am building a reinforcement learning model with openai's gym any_trading environment and stablebaselines3. There are a ...
0 votes
0 answers
49 views

How to add metrics to EvalCallback in StableBaselines 3

I want to add some metrics like loss to evaluation metrics in Stable Baselines 3. The goal is to make an callback that stops training on loss no decreasing instead of provided OnNoImprovement stop ...
  • 51
0 votes
0 answers
33 views

What is missing in Gym registration?

I get ValueError: xxx not found in gym registry, you maybe meant when trying to register a custom environment in stable baselines 3. I tried the following command: python train.py --algo tqc --env ...
2 votes
0 answers
541 views

Can't install stable-baselines3[extra]

I am having trouble installing stable-baselines3[extra]. Not sure if I missed installing any dependency to make this work. Machine: Mac M1, Python: Python 3.10.9, pip3: pip 23.0 !pip3 install 'stable-...
0 votes
1 answer
44 views

How can I improve this Reinforced Learning scenario in Stable Baselines3?

In this scenario, I present a box observation with numbers 0, 1 or 2 and shape (1, 10). The odds for 0 and 2 are 2% each, and 96% for 1. I want the model to learn to pick the index of any 2 that comes....
  • 715
0 votes
1 answer
61 views

stable-baseline3, gym, train while also step/predict

With stable-baselines3 given an agent, we can call "action = agent.predict(obs)". And then with Gym, this would be "new_obs, reward, done, info = env.step(action)". (more or less, ...
  • 964
0 votes
2 answers
109 views

Accessing training metrics in stable-baselines3

Is it possible to access the A2C total loss and whether the environment truncated or terminated within a custom callback? I'd like to access truncated and terminated in _on_step. That would allow me ...
0 votes
1 answer
101 views

Stable-Baselines make_vec_env() not calling the wrapper kwargs as expected

I have a custom gym environment where I am implementing action masking. The code below works well from toyEnv_mask import PortfolioEnv # file with my custom gym environment from sb3_contrib import ...
0 votes
0 answers
35 views

Problem registering custom gym: gym.error.UnregisteredEnv

I created a custom gym called BazEnv for use with stable_baselines3 but I cannot seem to register it properly. I get the error gym.error.UnregisteredEnv: No registered env with id: foo_bar_envs/BazEnv-...
0 votes
0 answers
20 views

How to rescale the output of PPO model to the range of the action space in PPO?

Stable baselines3 PPO implementation using MLP policy uses two hidden layers with 64 nodes each. On setting my gym environment, I had set my action space in the range [-50,50]. However, there seem to ...
0 votes
0 answers
56 views

StableBaselines3 / steps vs. total_timesteps vs. number of times environment is solved during training

model.learn(total_timesteps=20) takes much longer than I expected, so I'm trying to understand if I should: Be frugal with time steps Speed up my environment env.step(action) Train even more time ...
0 votes
1 answer
126 views

Why `ep_rew_mean` much larger than the reward evaluated by the `evaluate_policy()` fuction

I write a custom gym environment, and trained with PPO provided by stable-baselines3. The ep_rew_mean recorded by tensorboard is as follow: the ep_rew_mean curve for total 100 million steps, each ...
0 votes
1 answer
63 views

Use MiniGrid environment with stable-baseline3

I'm using MiniGrid library to work with different 2D navigation problems as experiments for my reinforcement learning problem. I'm also using stable-baselines3 library to train PPO models. but ...
0 votes
0 answers
59 views

Optuna HyperbandPruner not pruning?

My study is setup to use the Hyperband pruner with 60 trials, 10M max resource and reduction factor of 2. def optimize_agent(trial): # ... model = PPO("MlpPolicy", env, **params) ...
0 votes
1 answer
44 views

Can't set the frame rate when recording a video with VecVideoRecorder

I have a working RL model and set up that produce a video for me - however becuase the model is reasonably good, the videos are very short (reach a desitination therfore better = shorter) Is there a ...
1 vote
0 answers
96 views

Stable Baselines 3 DQN Model refuses to use CUDA even though it recognizes my GPU

as the title states. My DQN Model refuses to use the GPU for a custom environment with Stable Baselines 3. model = DQN("MlpPolicy", env, device="cuda") My GPU is an RTX 2070 Super ...
  • 11
1 vote
1 answer
64 views

Get the current step number in a gym.Env

Is there a way to access the current step number of a gym.Env from inside its step method? I'm using a model from stable_baselines3 and want to terminate the env when N steps have been taken.
0 votes
0 answers
61 views

Can´t use Stable Baselines 3 in Anaconda Environment

I try to use Stable Baselines 3 in my project. Still I can't use it, even after installing it in my Anaconda environment. Python Script from stable_baselines3.common.env_checker import check_env from ...
  • 1
0 votes
1 answer
151 views

Is it possible to use a stable-baselines model as the baseline for another model?

I recently trained a stable-baselines PPO model for a couple of days and it is performing well on test environments. Essentially, I am trying to iterate on this model. I was wondering if it is ...
0 votes
1 answer
165 views

Pre-Train a Model using imitation learning with Stable-baselines3

I have been trying to figure out a way to Pre-Train a model using Stable-baselines3. In the original documentation for Stable-baseline (the version which runs on Tensorflow 1.X), this seems to be an ...
1 vote
1 answer
136 views

gym 0.21 + stable_baseline3 TypeError: tuple indices must be integers or slices, not str

I'm trying to train a stable_baseline3 model on my custom gym environment. The training crashes with a TypeError on the first step. Using cuda device Traceback (most recent call last): File "...
0 votes
0 answers
77 views

Hey, I am trying to build a custom environment and training an agent on that

at this line: model = PPO("MlpPolicy", env, verbose=1, tensorboard_log=log_path) It raises an error: AttributeError Traceback (most recent call last) Cell In[200],...
0 votes
0 answers
24 views

Teaching Reinforcement learning Agent on many custom Envionments that perform steps asynchronously

Based on old adobe flash game, I've created custom OpenAI Gym, that's interacting with the game, by reading it's process memory, and performing actions with mouse clicks (game window needs to be ...
1 vote
1 answer
133 views

No GL context; create a Window first

When I ran the Stable Baselines3 RL Colab Notebooks, an error occurred. stable_baselines_getting_started.ipynb record_video('CartPole-v1', model, video_length=500, prefix='ppo-cartpole') GLException ...
0 votes
1 answer
69 views

Evaluate_policy records much higher mean reward then stable baselines 3 logger

As the title says, I am testen PPO with the Cartpole Environment using SB3, but if I look at the performance measured be the evaluate_policy function I reach a mean reward of 475 reliable at 20000 ...
0 votes
1 answer
24 views

Is it possible to set the exploration rate to 0, and turn off network training for a Stable Baselines 3 algorithm?

After training a stable baselines 3 RL algorithm (I am using mainly PPO) I want to set the exploration rate to 0, and turn off network training so I always get the same output (action) from the model ...
1 vote
0 answers
100 views

Stablebaselines3 and Pettingzoo

I am trying to understand how to train agents in a pettingzoo environment using the single agent algorithm PPO implemented in stablebaselines3. I'm following this tutorial where the agents act in a ...
  • 151
0 votes
0 answers
72 views

Error while rendering in the gym environment

I'm trying to work for a RL project where I need to test the ideas in some Atari environement and I choose Pong to test the initial ideas. I'm using stable-baselines3 version 1.6.2 for ease of ...
0 votes
1 answer
90 views

StableBaselines creating a model segmentation fault

I am getting a segmentation fault when trying to create a stable_baselines3 PPO model on a CartPole-v1 OpenAI Gym environment. So far what I've tried is running a short example code on Python 3.10 as ...
  • 311
0 votes
1 answer
405 views

AssertionError: The algorithm only supports <class 'gym.spaces.box.Box'> as action spaces but Box(-1.0, 1.0, (3,), float32) was provided

So basically I tried converting this custom gym environment from https://github.com/Gor-Ren/gym-jsbsim to use farama foundation's gymnasium api. This is my repo whih I am working on: https://github....
  • 1

1
2 3 4 5