93

Is there a way to access variables in the current python kernel from within a %%bash or other %%script cell?

Perhaps as command line arguments or environment variable(s)?

8 Answers 8

112

Python variables can be accessed in the first line of a %%bash or %%script cell, and so can be passed as command line parameters to the script. For example, with bash you can do this:

%%bash -s "$myPythonVar" "$myOtherVar"
echo "This bash script knows about $1 and $2"

The -s command line option allows you to pass positional parameters to bash, accessed through $n for the n-th positional parameter. Note that what's actually assigned to the bash positional variable is the result of str(myPythonVariable). If you're passing strings containing quote characters (or other bash-sensitive characters), you'll need to escape them with a backslash (eg: \").

The quotes are important - without them the python variables (string representations) are split on spaces, so if myPythonVar was a datetime.datetime with str(myPythonVar) as "2013-10-30 05:04:09.797507", the above bash script would receive 3 positional variables, the first two with values 2013-10-30 and 05:04:09.797507. It's output would be:

This bash script knows about 2013-10-30 and 05:04:09.797507

If you want to name the variables and you're running linux, here's an approach:

%%script env my_bash_variable="$myPythonVariable" bash
echo myPythonVariable\'s value is $my_bash_variable

You can specify multiple variable assignments. Again beware of quotes and other such things (here bash will complain bitterly!). To grok why this works, see the env man page.

0
76

To include python variables within bash commands run using the syntax !<some command> you can use {<variable>} as follows:

In [1]: for i in range(3):
   ...:     !echo {i+1}
   ...:     
1
2
3

While this is slightly different from what the OP asked, it is closely related and useful in performing a scripting task. This post has more great tips and examples of using shell command within IPython and Jupyter notebooks.

5
8

One problem is, if the variable that you want to give to your bash is a list, it does not work as expected.

For example, in one python cell:

l = ['A', 'B', 'C']

then if you give it directly to the magic option the next cell:

%%bash -s "$l"
for i in $1
do
echo $i
done

It will be oddly split like this:

['A',
'B',
'C']

The simplest answer is to put code inside braces {} to transform your python list in bash list, like the following:

%%bash -s "{" ".join(l)}"
for i in $1
do
echo $i
done

Which give the expected output:

A
B
C
1
  • odd that the use of double quotes inside double quotes works (: Seems the %%bash magic is looking for a } and isn't concerned about it.
    – drevicko
    Jul 5, 2018 at 13:09
8

If someone like me ends up here looking for how to use Python variables when running commands with !, just add prefix the variable with $ and that should do it:

!echo $foobar
5

You can use Python string templates if you are willing to define a new magic with:

from IPython import get_ipython
from IPython.core.magic import register_cell_magic

ipython = get_ipython()


@register_cell_magic
def pybash(line, cell):
    ipython.run_cell_magic('bash', '', cell.format(**globals()))

And then if you define a variable in Python like:

test = 'Python variables'

you can use it:

%%pybash
echo '{test} will be expanded'
echo '{{double braces will be replaced with single braces}}'

Resulting in:

Python variables will be expanded
{double braces will be replaced with single braces}
3
4

Just to note a variation, if you need to pass something other than a simple variable to the bash script:

%%bash -s $dict['key1'] $dict['key2'] $dict['key3']

goes gruesomely wrong, but

%%bash -s {dict['key1']} {dict['key2']} {dict['key3']}

works nicely.

1

Inspired by the answer from @krassowski, the following works to interpret arbitrary expressions in curly braces, with the semantics of f-strings. That is, you can use expressions such as {1+val} and not just {val}, given that the variable val is defined in the global python context (such as in a Jupyter notebook).

@register_cell_magic
def pybash(line, cell):
    cell_replaced = eval("f" + repr(cell))
    # print("Evaluating:\n{}\n-----------".format(cell_replaced))
    ipython.run_cell_magic('bash', '', cell_replaced)

As an example, define a global variable in a python cell:

val = 2

Then, in a new cell:

%%pybash
echo \\
echo {1+val}

This outputs

\
3

This is safe as long as the cell content is not controlled by an attacker.

0

No, %%script magic are autogenerated and don't do any magic inter-process data communication. (which is not the case for %%R but which is a separate magic in its own class with extra care from R peoples)

But writing your own magic that does it is not too hard to do.

7
  • 1
    Did you mean something like %%bash -c 'myvar=$MyPythonVar bash', then perhaps echo $myvar in the next line of the cell? That works...
    – drevicko
    Oct 28, 2013 at 0:28
  • so.. what did you have in mind? It complains if the %%bash .. cell has no other contents, and if I put, say, ls in there, the os comes back with /bin/echo: /bin/echo: cannot execute binary file (at least on my mac - but ubuntu does something similar i think)
    – drevicko
    Oct 29, 2013 at 0:22
  • Ah sorry, I read too fast I though what you were writing was working. Looking at the code the bash magic does not support {..} syntax except in the first line. So yo will need to write it by yourself.
    – Matt
    Oct 29, 2013 at 7:46
  • ok. I found a few ways to use the first line (answer below). I guess writing a %% magic to do it better could be nicer - perhaps special comments like #%%exppose myPythonVar then substitute it's value into the script when you see $myPythonVar or something... Perhaps one day I'll give it a whirl (:
    – drevicko
    Oct 30, 2013 at 5:43
  • In the current jupyter, the method I proposed above doesn't seem to be working - python variables are not accessible from the first line of a %%bash cell. @Matt Can you confirm?
    – drevicko
    Jul 11, 2016 at 17:39

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.