Is there a way to access variables in the current python kernel from within a %%bash
or other %%script
cell?
Perhaps as command line arguments or environment variable(s)?
Python variables can be accessed in the first line of a %%bash
or %%script
cell, and so can be passed as command line parameters to the script. For example, with bash you can do this:
%%bash -s "$myPythonVar" "$myOtherVar"
echo "This bash script knows about $1 and $2"
The -s
command line option allows you to pass positional parameters to bash, accessed through $n
for the n-th positional parameter. Note that what's actually assigned to the bash positional variable is the result of str(myPythonVariable)
. If you're passing strings containing quote characters (or other bash-sensitive characters), you'll need to escape them with a backslash (eg: \"
).
The quotes are important - without them the python variables (string representations) are split on spaces, so if myPythonVar
was a datetime.datetime
with str(myPythonVar)
as "2013-10-30 05:04:09.797507"
, the above bash script would receive 3 positional variables, the first two with values 2013-10-30
and 05:04:09.797507
. It's output would be:
This bash script knows about 2013-10-30 and 05:04:09.797507
If you want to name the variables and you're running linux, here's an approach:
%%script env my_bash_variable="$myPythonVariable" bash
echo myPythonVariable\'s value is $my_bash_variable
You can specify multiple variable assignments. Again beware of quotes and other such things (here bash will complain bitterly!). To grok why this works, see the env
man page.
To include python variables within bash commands run using the syntax !<some command>
you can use {<variable>}
as follows:
In [1]: for i in range(3):
...: !echo {i+1}
...:
1
2
3
While this is slightly different from what the OP asked, it is closely related and useful in performing a scripting task. This post has more great tips and examples of using shell command within IPython and Jupyter notebooks.
!
so willy-nilly. You can even do n = !echo {i + 1}
.
Jul 3, 2019 at 8:03
One problem is, if the variable that you want to give to your bash is a list, it does not work as expected.
For example, in one python cell:
l = ['A', 'B', 'C']
then if you give it directly to the magic option the next cell:
%%bash -s "$l"
for i in $1
do
echo $i
done
It will be oddly split like this:
['A',
'B',
'C']
The simplest answer is to put code inside braces {}
to transform your python list in bash list, like the following:
%%bash -s "{" ".join(l)}"
for i in $1
do
echo $i
done
Which give the expected output:
A
B
C
%%bash
magic is looking for a }
and isn't concerned about it.
If someone like me ends up here looking for how to use Python variables when running commands with !
, just add prefix the variable with $
and that should do it:
!echo $foobar
You can use Python string templates if you are willing to define a new magic with:
from IPython import get_ipython
from IPython.core.magic import register_cell_magic
ipython = get_ipython()
@register_cell_magic
def pybash(line, cell):
ipython.run_cell_magic('bash', '', cell.format(**globals()))
And then if you define a variable in Python like:
test = 'Python variables'
you can use it:
%%pybash
echo '{test} will be expanded'
echo '{{double braces will be replaced with single braces}}'
Resulting in:
Python variables will be expanded
{double braces will be replaced with single braces}
Just to note a variation, if you need to pass something other than a simple variable to the bash script:
%%bash -s $dict['key1'] $dict['key2'] $dict['key3']
goes gruesomely wrong, but
%%bash -s {dict['key1']} {dict['key2']} {dict['key3']}
works nicely.
Inspired by the answer from @krassowski, the following works to interpret arbitrary expressions in curly braces, with the semantics of f-strings. That is, you can use expressions such as {1+val}
and not just {val}
, given that the variable val
is defined in the global python context (such as in a Jupyter notebook).
@register_cell_magic
def pybash(line, cell):
cell_replaced = eval("f" + repr(cell))
# print("Evaluating:\n{}\n-----------".format(cell_replaced))
ipython.run_cell_magic('bash', '', cell_replaced)
As an example, define a global variable in a python cell:
val = 2
Then, in a new cell:
%%pybash
echo \\
echo {1+val}
This outputs
\
3
This is safe as long as the cell content is not controlled by an attacker.
No, %%script
magic are autogenerated and don't do any magic inter-process data communication. (which is not the case for %%R
but which is a separate magic in its own class with extra care from R
peoples)
But writing your own magic that does it is not too hard to do.
%%bash -c 'myvar=$MyPythonVar bash'
, then perhaps echo $myvar
in the next line of the cell? That works...
%%bash ..
cell has no other contents, and if I put, say, ls
in there, the os comes back with /bin/echo: /bin/echo: cannot execute binary file
(at least on my mac - but ubuntu does something similar i think)
{..}
syntax except in the first line. So yo will need to write it by yourself.
#%%exppose myPythonVar
then substitute it's value into the script when you see $myPythonVar
or something... Perhaps one day I'll give it a whirl (: