97

I am hoping to run a simple shell script to ease the management around some conda environments. Activating conda environments via conda activate in a linux os works fine in the shell but is problematic within a shell script. Could someone point me into the right direction as to why this is happening?

Example to repeat the issue:

# default conda env
$ conda info | egrep "conda version|active environment"
     active environment : base
          conda version : 4.6.9

# activate new env to prove that it works
$ conda activate scratch
$ conda info | egrep "conda version|active environment"
     active environment : scratch
          conda version : 4.6.9

# revert back to my original conda env
$ conda activate base 

$ cat shell_script.sh
#!/bin/bash
conda activate scratch

# run shell script - this will produce an error even though it succeeded above
$ ./shell_script.sh

CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run

    $ conda init <SHELL_NAME>

Currently supported shells are:
  - bash
  - fish
  - tcsh
  - xonsh
  - zsh
  - powershell

See 'conda init --help' for more information and options.

IMPORTANT: You may need to close and restart your shell after running 'conda init'.
3
  • 2
    One thing you can do is goto your .bashrc using nano ~/.bashrc and copy all the conda related stuff in the beginning of your shell script that you are creating. That might help. Commented Apr 5, 2019 at 16:37
  • Conda related things you can find simply using finding #Added by Anaconda comment. Commented Apr 5, 2019 at 16:38
  • 1
    this command conda shell.bash activate ENV_NAME worked for me
    – M Rostami
    Commented Mar 12 at 15:02

13 Answers 13

70

I use 'source command' to run the shell script, it works:

source shell_script.sh
5
  • 20
    Please make sure to explain your answer in detail. Visti stackoverflow.com/help/how-to-answer on how to write a good answer
    – DjSh
    Commented Jul 2, 2019 at 12:57
  • 1
    well this was the most useful answer so far! took me a whole day and I could not figure it quite out, but this answer really solved my problem :) Thanks.
    – Prelude
    Commented Sep 5, 2019 at 14:59
  • 2
    an edit to my previous comment, actually in the end it set me in a rabit hole where I had to delete my whole linux and reinstall it after mistakes which followed using this. So if you're planning to use it do it carefully!
    – Prelude
    Commented Dec 2, 2019 at 9:55
  • 7
    @Prelude Definitely, sourcing your script and running it are not the same (see here for more info). Basically, sourcing will run the lines in your script in the current shell, rather than enveloping them in a subshell. Sometimes, this is what one wants, but not always! Commented Jan 11, 2020 at 19:44
  • This also resolved the issue for me with a minimum of fuss.
    – A Burns
    Commented Aug 25, 2022 at 20:26
66

The error message is rather helpful - it's telling you that conda is not properly set up from within the subshell that your script is running in. To be able to use conda within a script, you will need to (as the error message says) run conda init bash (or whatever your shell is) first. The behaviour of conda and how it's set up depends on your conda version, but the reason for the version 4.4+ behaviour is that conda is dependent on certain environment variables that are normally set up by the conda shell itself. Most importantly, this changelog entry explains why your conda activate and deactivate commands no longer behave as you expect in versions 4.4 and above.

For more discussion of this, see the official conda issue on GitHub.


Edit: Some more research tells me that the conda init function mentioned in the error message is actually a new v4.6.0 feature that allows a quick environment setup so that you can use conda activate instead of the old source activate. However, the reason why this works is that it adds/changes several environment variables of your current shell and also makes changes to your RC file (e.g.: .bashrc), and RC file changes are never picked up in the current shell - only in newly created shells. (Unless of course you source .bashrc again). In fact, conda init --help says as much:

IMPORTANT: After running conda init, most shells will need to be closed and restarted for changes to take effect

However, you've clearly already run conda init, because you are able to use conda activate interactively. In fact, if you open up your .bashrc, you should be able to see a few lines added by conda teaching your shell where to look for conda commands. The problem with your script, though, lies in the fact that the .bashrc is not sourced by the subshell that runs shell scripts (see this answer for more info). This means that even though your non-login interactive shell sees the conda commands, your non-interactive script subshells won't - no matter how many times you call conda init.

This leads to a conjecture (I don't have conda on Linux myself, so I can't test it) that by running your script like so:

bash -i shell_script.sh

you should see conda activate work correctly. Why? -i is a bash flag that tells the shell you're starting to run in interactive mode, which means it will automatically source your .bashrc. This should be enough to enable you to use conda within your script as if you were using it normally.

8
  • 2
    Thanks for your response but after viewing both referenced links, it is still not clear to me how to resolve the issue. I have tried a combination of conda init bash in my shell script as well as sourcing . $CONDAHOME/etc/profile.d/conda.sh neither of which worked. Note that conda info works fine in the shell script. Commented Apr 5, 2019 at 2:17
  • @user9074332 What version of conda do you have? And have you verified that $CONDAHOME/etc/profile.d/conda.sh actually exists and contains what you expect? For example, the changelog suggests /opt/conda/etc/profile.d/conda.sh. Also, what does which conda give you? Commented Apr 5, 2019 at 4:14
  • 3
    Thanks for the additional information. This seems to have done the trick. $cat shell_script.sh #!/bin/bash which conda conda activate scratch conda info|grep "active environment" Run as follows: bash -i shell_script.sh. And the output: ~/miniconda3/bin/conda active environment : scratch Commented Apr 7, 2019 at 20:27
  • 2
    I like the solution here as it can help run the bash file without the need of source or -i option.
    – SaTa
    Commented Jul 3, 2019 at 16:57
  • 1
    bash -i, works but it has the side effect that your history becomes polluted with the commands inside the bash script. using source ~/miniconda3/etc/profile.d/conda.sh works best and calling the script with bash script.sh alone.
    – raygozag
    Commented Dec 16, 2022 at 1:24
64

Using conda activate or source activate in shell scripts does not always work and can throw errors like this. An easy work around it to place source ~/miniconda3/etc/profile.d/conda.sh above any conda activate command in the script:

source ~/miniconda3/etc/profile.d/conda.sh # Or path to where your conda is
conda activate some-conda-environment

This is the solution that has worked for me and will also work if sharing scripts. This also gets around having to use conda init as on some clusters I have worked with the system is initialised but conda activate still won't work in a shell script.

4
  • 2
    +1. As mentioned in this GitHub issue Commented Mar 7, 2020 at 4:59
  • +1 and, in my case, my bash script had a & hanging at the end of the line 'conda activate some-env' so I could return command to the terminal but it was preventing the script from running successfully.
    – rearThing
    Commented Oct 22, 2020 at 17:18
  • for /bin/sh use something like . /opt/conda/etc/profile.d/conda.sh Commented Dec 9, 2021 at 9:19
  • this is also mentioned in the official miniconda guide docs.conda.io/projects/conda/en/latest/user-guide/install/…
    – red-o-alf
    Commented Sep 12, 2023 at 14:24
39

Quick solution for bash: prepend the following init script into your Bash scripts.

eval "$(command conda 'shell.bash' 'hook' 2> /dev/null)"

Done.


For other shells, check the init conf of your shell, copy the following content within the shell conf and prepend it into your scripts.

# >>> conda initialize >>>
...
# <<< conda initialize <<<

You can also use

conda init --all --dry-run --verbose

to get the init script you need in your scripts.

Explanation

This is related with the introduction of conda init in conda 4.6.

Quote from conda 4.6 release log

Conda 4.4 allowed “conda activate envname”. The problem was that setting up your shell to use this new feature was not always straightforward. Conda 4.6 adds extensive initialization support so that more shells than ever before can use the new “conda activate” command. For more information, read the output from “conda init –help”

After conda init is introduced in conda 4.6, conda only expose command conda into the PATH but not all the binaries from "base". And environment switch is unified by conda activate env-name and conda deactivate on all platforms.

But to make these new commands work, you have to do an additional initialization with conda init.

The problem is that your script file is run in a sub-shell, and conda is not initialized in this sub-shell.

References

2
  • Thanks. I am working on a cluster, than there are several conda, but I wanted a specific one installed on my $HOME, that had its "init" in my .bashrc. I tried the method to copy the block "# >>> conda initialize >>>" and it worked on my bash script, my environments can be activated. Thanks.
    – Gildas
    Commented Jan 14, 2020 at 8:34
  • 8
    Could you explain the purpose of putting command before conda? Is there some particular case where the shell function for conda gets in the way?
    – Ben Mares
    Commented Sep 15, 2020 at 15:25
12

This answer from Github worked for me (I'm using Ubuntu so it's not for Windows only):

eval "$(conda shell.bash hook)"
conda activate my_env
3
  • Fine, but this is a less robust version of this older answer.
    – merv
    Commented Jan 5, 2022 at 1:34
  • @merv obviously I tried all the answers from here before writing down my answer
    – Oresto
    Commented Jan 12, 2022 at 12:50
  • This also worked for Ubuntu on wsl for me
    – Rahul K.
    Commented Jun 3 at 17:56
9

if you want to use the shell script to run the other python file in the other conda env, just run the other file via the following command.

os.system('conda run -n <env_name> python <path_to_other_script>')
2
  • Thanks a lot, this solution avoid a lot of unwanted issues with all the conda init and conda activate
    – SJU
    Commented Aug 14, 2021 at 14:56
  • Use --live-stream to also see stdout/stderr as it is produced
    – winni2k
    Commented Jul 24, 2023 at 17:41
4

I didn't find any of the above scripts useful. These are fine if you want to run conda in non-interactive mode, but I'd like to run it in interactive mode. If I run:

conda activate my_environment 

in a bash script it just runs in the script.

I found that creating an alias in .bashrc is all that is required to change directory to a particular project I'm working on, and set up the correct conda environment for me. So I included in .bashrc:

alias my_environment="cd ~/subdirectory/my_project && conda activate my_environment"

and then:

source ~/.bashrc

Then I can just type at the command line:

my_environment

to change to the correct project and correct environment everytime I want to work on a different project.

1
  • Saved my day. This is precisely what I wanted
    – srswat
    Commented Jun 11, 2023 at 7:50
3

What is the problem with simply doing something like this in your shell:

source /opt/conda/etc/profile.d/conda.sh

(The conda init is still marked as Experimental, and thus not sure if it is a good idea to use it yet).

0
3

I also had the exact same error when trying to activate conda env from C++ or Python file. I solved it by bypassing the conda activate statement and using the absolute path of the specific conda env.

For me, I set up an environment called "testenv" using conda.

I searched all python environments using

whereis python | grep 'miniconda'

It returned a list of python environments. Then I ran my_python_file.py using the following command.

~/miniconda3/envs/testenv/bin/python3.8 my_python_file.py

You can do the same thing on windows too but looking up for python and conda python environments is a bit different.

1
  • You just saved my day! This solved my probem in a context where I start my env via subprocess.Popen(..) on linux.
    – Greg7000
    Commented Feb 26, 2021 at 14:24
1

I just followed a similar solution like the one from hong-xu

So to run a shell command that calls the script with arguments and using a specific conda environment:

from a jupyter cell, goes like this :

p1 = <some-value>
run = f"conda run -n {<env-name>} python {<script-name>.py} \
    --parameter_1={p1}"

!{run}
1

The problem is that when you run the bash script, a new (linux) shell environment is created that was not initialized properly. If your intention is to activate a conda environment, and then run python through the script, you can properly initialize the created shell environment as discussed in the accepted solution.

If however you want to have the conda environment active after you finish this script, then this will not work because the conda environment has changed on the new shell and you exit that shell when you finish the script. Think of this as running bash, then conda activate... then running exit to exit that bash... More details in How to execute script in the current shell on Linux?:

TL;DR:

  1. Add the line #!/bin/bash as the first line of the script
  2. Type the command source shell_script.sh or . shell_script.sh

Note: . in bash is equivalent to source in bash.

0

This answer is similar to @Lamma answer. This worked for me ->

(1) I defined several variables; the conda activate function, environments directory and environment

conda_activate=~/anaconda3/bin/activate
conda_envs_dir=~/anaconda3/envs
conda_env=<env name>

(2) I source conda activate with the environment

source ${conda_activate} ${conda_envs_dir}/${conda_env}

(3) then you can run your python script

python <path to script.py>

This bypasses the conda init requirement. my .bashrc already was initialized and sourcing the .bashrc file didn't work for me. @Lamma's answer worked for me as well as the above code.

-5
   #open terminal or CMD as administrator

$ cd <path Anaconda3 install>\Scripts

$ activate

$ cd .. 

$ conda activate scratch
1
  • 1
    The question is 'in shell script' not in terminal Commented Jan 29, 2021 at 7:52

Not the answer you're looking for? Browse other questions tagged or ask your own question.