Questions tagged [fb-hydra]

Questions about Hydra, an open source Python framework by Facebook

Filter by
Sorted by
Tagged with
0 votes
1 answer
12 views

Python Hydra - Is this possible to list config and yaml file?

I have below file hirarchy from which i want to list config path and yaml files Main_dir: conf: project: test_project1: env: source: src.yaml dest: ...
Hari's user avatar
  • 87
0 votes
0 answers
9 views

Python Hydra- Is there any way to print config path and file using hydra compose api?

I need to get the list of config directory/path name and files inside the config path from below directory Main_dir: conf: project: test_project1: env: source: src....
Hari's user avatar
  • 87
1 vote
0 answers
14 views

Use Hydra to populate nested dataclass objects

I have a yaml config file that has the following structure paths: log: ./runs data: ./data data: downloads: [ { name: "1", url: "...
Dan Jackson's user avatar
1 vote
1 answer
60 views

How to use hydra compose API to read yaml from a nested directory?

I am trying to print the src.yaml from a nested directory below Main_dir: conf: project: test_project1: env: source: src.yaml src.yaml table: ...
Hari's user avatar
  • 87
0 votes
1 answer
10 views

Hydra defaults using YAML: How to handle optional user configuration?

I'm distributing an application which includes a several commands. Some commands require no user configuration as sane defaults can be packaged with the app Some commands require additional user ...
Mike Richards's user avatar
0 votes
0 answers
23 views

Easiest way to override PosixPath in hydra

Consider the following yaml file for hydra config: a: b: !!python/object/apply:pathlib.PosixPath - /my/path/to/dir How would I override a.b so that is stays PosixPath after providing a new ...
Proko's user avatar
  • 1,841
0 votes
1 answer
19 views

refined selection of log-level of modules in Hydra

I'm trying to debug a Hydra application, setting hydra.verbose=true will set the logging level of all modules to Debug. Is there a way to get rid of some of the modules debug messages that produce ...
itamar kanter's user avatar
0 votes
0 answers
26 views

How to gather several datasets (dataset configurations) in a list with hydra?

I am using hydra for the configuration of deep learning projects. I want to put together several datasets for the training. As the number of datasets is a priori not defined, I want to get the ...
nemo's user avatar
  • 11
0 votes
0 answers
13 views

Running Hydra from IDE

That may seem a naive question, but is there a way to run use Hydra from an IDE? I am interested in using it within anaconda spyder, but if there is way to run experiments with hydra from another IDE ...
can't stop me now's user avatar
0 votes
1 answer
25 views

Where does hydra find its keys in Python?

I'm working in the nuPlan devkit (https://github.com/motional/nuplan-devkit/tree/master/tutorials), specifically in the nuplan_framework tutorial, and I reach this error: MissingConfigException: In '...
ROS's user avatar
  • 305
0 votes
1 answer
109 views

Hydra omegaconf resolvers on dictionaries and lists

I'm trying to use omegaconf resolvers on dictionaries and lists but I keep getting an error. for example the following code gives a hydra.errors.OverrideParseException (taken from here Hydra combine ...
Ori Noked's user avatar
0 votes
2 answers
159 views

OmegaConf - how to delete a single parameter

I have a code that looks something like this: def generate_constraints(c): if c.name == 'multibodypendulum': con_fnc = MultiBodyPendulum(**c) where c is an OmegaConf object containing a ...
Tue's user avatar
  • 371
0 votes
1 answer
111 views

Overriding Hydra config with experiment

I'm working through this Hydra doc example to override the main config with an experiment config. I'm trying to override main with experiment and I'm not sure where I'm going wrong. The error I get is:...
KSHMR's user avatar
  • 729
1 vote
1 answer
51 views

How can I organize Hydra 1.3 configuration files and import them into each other?

I am trying to use Hydra 1.3 contrive a simple, but apparently not trivial, configuration that maps endpoints of a given API to their corresponding processing functions. So far, I came up with a ...
Índio's user avatar
  • 539
0 votes
0 answers
108 views

Is there a way to use Ray Tune in combination with ML Flow and Hydra in Python?

I want to do hyperparameter tuning for a neural net, created with keras. For this project I handle my config.yaml files with hydra, use mlflow to store the metrics and parameters from the optimization ...
Patrick's user avatar
0 votes
0 answers
89 views

How to customize hydra run dir and sweep subdir based on values of a sweep?

I am using hydra to organize my configurations and output folders for a project. This project involves first specifying a particular objective that can differ in a few parameters. Then that objective ...
Nathaniel Imel's user avatar
0 votes
1 answer
75 views

Key 'optuna_config' not in 'OptunaSweeperConf' full_key: hydra.sweeper.optuna_config object_type=OptunaSweeperConf

I tried this tutorial but gives me this error Key 'optuna_config' not in 'OptunaSweeperConf' full_key: hydra.sweeper.optuna_config object_type=OptunaSweeperConf why is this error? this is my config....
Jorge Armando Navarro Flores's user avatar
0 votes
1 answer
95 views

In hydra, can I interpolate config from a file without using the defaults list?

Say I have a directory utility_configs that has a bunch of different configurations for different things that are useful in different situations. And then I want to be able to use these different ...
gmr's user avatar
  • 682
0 votes
1 answer
130 views

Hydra - accessing parent of parent directory

is it possible to have a 2-degree sibling file as a default configuration? Files hierarchy: dir_1 ├── dir_1_1 │ └── dir_1_1_1 │ └── conf.yaml └── dir_1_2 └── dir_1_2_1 └── ...
Ilan Geffen's user avatar
1 vote
1 answer
163 views

Set Hydra enviroment with uvicorn/fastapi

I have a FastAPI application that is going to production soon, however i am facing some problems with hydra integration. First i could not run the @hydra.main() decorator on fastAPI endpoints. This ...
Showertime's user avatar
1 vote
1 answer
207 views

extend list from default config in Hydra

I have a list that is defined in my defaults configuration file base_list: list: - 1 - 2 I know I can override the list values in the config file: defaults: - base_list list: - 3 - 4 which ...
itamar kanter's user avatar
1 vote
1 answer
573 views

How to access the Hydra config object at runtime

I need to change the output/working directory of the hydra config framework in such a way that it lies outside of my project directory. According to my understanding and the doc, config.yaml would ...
Autumn's user avatar
  • 41
1 vote
2 answers
72 views

How can I access to a help subsection from a hydra config file in a command line?

I am currently using the hydra config file as a part of a deep learning framework, in order to declare several parameters. I wrote a help section in my config.yaml file to declare all the parameters ...
ofares's user avatar
  • 33
0 votes
1 answer
354 views

Hydra config: using another config as an input to multiple keys within a config

Suppose I have the following directory structure for my hydra configs: config |_config.yml operations |_subconfig.yml subconfig.yml is param_1: foo param_2: bar and config.yml is: ...
Alberto A's user avatar
  • 1,140
0 votes
1 answer
360 views

Access Hydra configuration parameters from different files

I'm trying to integrate hydra into my reinforcement learning code and I would like to access configuration parameters from different files, containing various functions and classes. Short Example: ...
Davide Ferrari's user avatar
0 votes
1 answer
117 views

Hydra - Specifying config group with a keyword other than the directory

Say I have the following file structure: |--configs | |--config.yaml | |--A | |--conf1.yaml | |--conf2.yaml | |--conf3.yaml |--main_app.py and my base config.yaml looks like defaults: ...
E9101's user avatar
  • 1
0 votes
0 answers
243 views

AttributeError: __aenter__ on asynchronous websocket python

I try to resolve this problem about asynchronous WebSocket using python. I don't know why when I used the asynchronous process I get the error log bellow. But my code is totally fine when I didn't ...
Moch. Chamdani M's user avatar
0 votes
1 answer
71 views

Schema validation in Hydra not working when configuration path is parent folder

I have the following project setup: configs/ ├── default.yaml └── trainings ├── data_config │ └── default.yaml ├── simple.yaml └── schema.yaml The content of the files are as ...
Gros Lalo's user avatar
  • 1,068
1 vote
1 answer
217 views

is there a way to modify log directory in hydra?

I am trying to modify log dir in hydra. In default, output dir in hydra is like below. outputs └── 2022-11-12 ├── 18-17-28 │ └── train.log ├── 18-18-37 │ └── train.log └── 18-...
이준혁's user avatar
  • 257
2 votes
0 answers
48 views

Hydra: disable permutation of few hyperparameters

I am trying to do a basic hyperparameter tuning. By default Hydra creates a permutation of each hyperparameter. hydra: mode: MULTIRUN sweeper: params: +n: 5,10,15 +...
Wonder HD's user avatar
1 vote
1 answer
333 views

Conditional intializations of parameters in hydra

I'm pretty new to hydra and was wondering if the following thing is was possible: I have the parameter num_atom_feats in the model section which I would like to make dependent on the feat_type ...
James Arten's user avatar
0 votes
2 answers
94 views

Using multiple models with repeated keywords in `hydra`

I'm pretty new to hydra and I'm trying to better understand the config.yaml file. I'm undertaking a deep learning experiment where I have two separate models, an embedding network and a simple fully ...
James Arten's user avatar
1 vote
1 answer
356 views

How to dynamically define project root path in config file?

I want to define project root path in Hydra config file without hardcoding it. 'til now couldn't find any feature of Hydra that can imply the path! E.g., suppose we have the following project ...
mbrb's user avatar
  • 363
0 votes
0 answers
282 views

RuntimeError: only batches of spatial targets supported (3D tensors) but got targets of size: : [50] Unet integration

I modified this repository to use hydra and add U-net model https://github.com/kuangliu/pytorch-cifar I run this: python3 main.py --config-name=unet_train params.epoch_count=5 I got this error, but ...
Jorge Armando Navarro Flores's user avatar
3 votes
2 answers
2k views

What are Hydra advantages vs using a regular configuration file

I wonder what are the advantages of using Hydra to manage my configuration files, versus loading .yaml configuration file directly (using import yaml)?
Ilan Geffen's user avatar
1 vote
1 answer
165 views

How to create cli to access multi cli commands using hydra?

I want to package the repo and provide a cmd-line entry point for all the separate tasks managed by hydra. Let me explain with an example. I have multiple tasks that are all powered by hydra. Each ...
Ayush Chaurasia's user avatar
0 votes
1 answer
162 views

How to access all file names in hydra config

I have a directory contains a bunch of txt files: dir/train/[train1.txt, train2.txt, train3.txt] I'm able to read a single file, if I define following in a config.yaml file_name: ${paths.data_dir}/...
abhijit dhakane's user avatar
1 vote
0 answers
3k views

ModuleNotFoundError: No module named 'hydra'

I'm trying to run a Python script but it gives an error immediately at the first line of code while trying importing hydra library via: import hydra The error i get is the following: Traceback (most ...
Mario's user avatar
  • 137
1 vote
1 answer
118 views

Handling high number of configuration parameters in python

In my Machine Learning project I have a high number of parameters that are loaded from a configuration file, e.g. a YAML file. I wonder, is there any best practice on how to integrate them in the ...
Haydnspass's user avatar
1 vote
1 answer
131 views

Python - Hydra - Obtaining configuration in sub-module

Question I am currently having trouble with getting hydra to work when not using @hydra.main in the main.py script (refer to error section). I have tried the compose API, but that prevents me from ...
Raymond C.'s user avatar
0 votes
1 answer
158 views

How to use a config group multiple times, while overriding each instance

Here is my current config structure hydra/ pipeline/ common/ feature.yaml stage/ train.yaml with the following files: train.yaml # @package _global_ defaults: - _self_ - ../...
wc250's user avatar
  • 119
0 votes
1 answer
722 views

How to use Hydra Config alongside user-defined parameters?

Is there a way to use Hydra Config alongside other parameters? I want some arguments to be defined in the code, and others to be read from the config file. @hydra.main(config_path="conf", ...
Aadam's user avatar
  • 345
0 votes
2 answers
271 views

Using SageMaker with Hydra

I have a question about SageMaker and Hydra. TL;DR Is there a way to pass arguments from SageMaker estimator to a Hydra script? Currently it passes parameters in a very strict way. Full Question I use ...
David Lasry's user avatar
1 vote
1 answer
363 views

define a value dynamically using hydra for yaml files

Let's say I have an app.py like this class myClassA : def __init__(self): self.id = 100 class myClassB : def __init__(self, objA, id): pass Is there a way to use hydra to have a ...
Aydin Abiar's user avatar
2 votes
2 answers
191 views

Is there any way to log 'git hash' in hydra?

I want to control version of experiment configuration files with hydra and dvc without uploading original config files to git. Hydra does control config, and dvc controls version. But Hydra does not ...
이준혁's user avatar
  • 257
0 votes
1 answer
319 views

Can't create a multiprocessing Manager when using hydra

I am developing a code where I need to create a Manager to synchronize a Queue. In the project I am using hydra and everything works fine until I create a multiprocessing Manager Object. This is the ...
Alex Ferrando's user avatar
2 votes
1 answer
119 views

How to allow the user to override a subset of the configuration using their own yaml file?

Let's say I have this basic app: from dataclasses import dataclass import hydra from hydra.core.config_store import ConfigStore @dataclass class MyAppConfig: req_int: int opt_str: str = &...
talz's user avatar
  • 964
1 vote
1 answer
2k views

Hydra install on python 3.10 fails due to VS build tools

I'm trying to install Hydra 2.5 on a Windows 10 system. I have Visual Studio Build Tools 2022 installed with the desktop C++ development option. When I use pip I get the error attached below. I've ...
NatePhysics's user avatar
2 votes
2 answers
279 views

Passing argument/key pair to vscode Python debugger separated with an equal sign

For management of my config files, I'm using Hydra which requires passing additional arguments using a plus and then an equal sign between the argument and its value, e.g. python evaluate.py '+...
chronosynclastic's user avatar
2 votes
2 answers
880 views

Use a parameter multiple times in hydra config file

I am currently trying to replace the usage of argparse with hydra files to set the hyperparameters of a deep learning neural network. I succeeded in using a config.yaml file linked to a hydra main ...
ofares's user avatar
  • 33