5

I would like to understand how variable scoping works in Jupyter notebooks.

When I create a bash notebook with two cells, environment variables that are exported are visible across cell boundaries:

In [1]:

export PATH=$PATH:~/samplepath

In [2]:

echo $PATH

/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/myuser/samplepath

But if I create a Python notebook and use cell magics to achieve the same result, variables do not seem to be visible across cell boundaries any more:

In [1]:

%%script bash
export PATH=$PATH:~/samplepath

In [2]:

%%script bash
echo $PATH

/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

This behaviour stays the same for different magics (i.e., using an exclamation mark in front of echo instead of the script magic gives the same result).

So, I'd like to understand what the scoping rules are in this case, and how I can make export truly export variables so that they are visible in the entire notebook.

1 Answer 1

7

In Jupyter notebook (as in your second example), the shell commands in the notebook are executed in a temporary subshell i.e. every command you write using ! (or under %%bash) runs in a different subprocess.

A good description from this blog post is:

The Jupyter Notebook shell commands are executed in a shell that is a subprocess of the shell running the notebook. Because of this, changes in environmental variables in the child (cell) shell will not be reflected in the parent (notebook) shell.

In[1]:

%%bash 
export var="5"
echo $var
5

In[2]:

%%bash
echo $var

No output

This is the very reason why the commands like !cd /path/ can't be used to navigate the file system. Check another blog post on this.

While the case is different with bash notebook, where everything is executed in a single shell.

Solution:

If you want to export a variable to environment shell variables, then you can use %env magic function as follows:

In[3]:

%env var "5"

In[4]:

%env var

In[5]:

%%bash
echo $var

Output of In[4] and In[5]:

'"5"'

Note: But beware, dont't do %env PATH=$PATH:~/samplepath as it'll simply replace the PATH variable thus causing problems. The above solution is recommended for non-system variables only. Nevertheless, you can edit .bashrc to universally change the PATH variable.

1
  • 1
    how do I use them together with %%bash? Nov 12, 2020 at 15:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.