4

When starting a project in Python, I want to save some environment variables in a file called environment_variables and source this file in the bashrc.

The file looks something like this:

username=$(whoami)

# project root path
export PROJECT_DIR='/home/'$username'/nuclei_segmentation/'

# project data path
export DATA_DIR=$PROJECT_DIR"data/"

# location of models 
export MODEL_DIR=$PROJECT_DIR"models/"

# project output data
export OUTPUT_DIR=$PROJECT_DIR"output/"

I would like to change the PROJECT_DIR path so it is platform/name independent. So this environment_variables file will always be in the root directory of the project and I want to set PROJECT_DIR to always be the location of the environment_variables file.

I thought I could do this with PWD but when called from bashrc this creates an error, I also thought of a solution using find to search for the file from the root directory but this seems to complex and think there must be a better way?

14
  • 1
    This sounds very much like an XY Problem. Why do you want to do these things? Yours is perhaps not a good approach to the underlying problem you are attempting to solve.
    – tripleee
    Feb 11, 2018 at 15:41
  • Perhaps you are correct! Maybe the X part of my project is, what is a good way to setup environment variables in a python data science project, that is reproducible and platform independent? Feb 11, 2018 at 16:14
  • Honestly, I would just edit the file when I first copy it to the project directory to hard-code the value of PROJECT_DIR.
    – chepner
    Feb 11, 2018 at 16:34
  • 1
    What do you mean by "platform/name independent"? Should this be portable to POSIX platforms, or e.g. Windows too? What exactly is the scenario where $PWD doesn't work, and what's the error you get?
    – tripleee
    Feb 11, 2018 at 18:26
  • 1
    @WilliamGrimes, if you want to find the path to a source file, that's what $BASH_SOURCE is for. We have lots of preexisting questions that cover that ground; ie. getting the source directory of a bash script from within. Also see BashFAQ #28. Feb 12, 2018 at 5:03

2 Answers 2

2

If I am understanding at all where you want to go with this, my recommendation would be

  • Make the code work out of the current directory as far as possible.
  • Don't litter the user's environment with multiple variables if it can be avoided.
  • If you have to have environment variables, give them a name which clearly communicates what they are related to.

In some more concretion, a single variable which names the project root should be all you really need. Don't require it to be set if the expected files are in the current directory.

export NUC_SEG_DIR=$HOME/nuclei_segmentation

In your Python code, put in reasonable defaults. In this particular case, expect os.path.join(os.environ['NUC_SEG_DIR'], 'data') to point to your data directory, etc. If users want to override this, that's easy enough with symbolic links. Make sure this is parametrized in the code so that it's easy to override in a single place if you should want to make this configurable in the future. Maybe something like

def nuc_seg(root_dir=os.environ['NUC_SEG_DIR'], data_dir='./data', model_dir='./models', output_dir='./output'):
    if root_dir == '':
        root_dir='.'
    ... your code here

and perhaps later on make your command-line interface let the user override these values by way of command-line options or a configuration file.

Again, don't require the user to edit their .bash_profile or similar - what if they want to run multiple experiments in different directories, or whatever? They can figure out how to make the environment variable permanent, or you could even document this as an option if your users can't be expected to be familiar with the basics of the shell.

-1

Use dirname $0 inside script file to find the directory where current script file resides

6
  • for example export PROJECT_DIR=$(dirname $0) Feb 11, 2018 at 16:58
  • not sure exactly of syntax? Feb 11, 2018 at 16:58
  • 1
    Use quotes to cope with irregular filenames; PROJECT_DIR="$(dirname "$0")" ... Not sure if the export is really necessary; many beginners needlessly use it for variables which are not going to be exported to subprocesses.
    – tripleee
    Feb 11, 2018 at 17:05
  • thanks @tripleee I've just tried that and get the following error : dirname: invalid option -- 'b' Feb 11, 2018 at 17:39
  • 2
    Then your $0 is -bash (or -sh) which means you are sourcing the script. This answer is specifically about finding the source of the currently running script, but if you are not running a script, then of course that is moot.
    – tripleee
    Feb 11, 2018 at 18:22

Not the answer you're looking for? Browse other questions tagged or ask your own question.