397

I'm looking at using the *.ipynb files as the source of truth and programmatically 'compiling' them into .py files for scheduled jobs/tasks.

The only way I understand to do this is via the GUI. Is there a way to do it via command line?

2
  • 1
    What do you mean by "source of truth"? IPython notebooks are just json files. You can load them and manipulate as Python dictionaries. For source code you should be iterating input keys where cell_type equals 'code'. Have a look at this scheme
    – theta
    Jun 13, 2013 at 1:03
  • 1
    Well I want to store the .ipynb in a repository and not the .py files. So then as a 'build step' I would convert the .ipynb to .py files for actual use by the automated system. You're right, I could just load the json and output only the code cells, but I was wondering whether there was something out there already that did that for me :) Jun 13, 2013 at 1:17

18 Answers 18

593

If you don't want to output a Python script every time you save, or you don't want to restart the IPython kernel:

On the command line, you can use nbconvert:

$ jupyter nbconvert --to script [YOUR_NOTEBOOK].ipynb

As a bit of a hack, you can even call the above command in an IPython notebook by pre-pending ! (used for any command line argument). Inside a notebook:

!jupyter nbconvert --to script config_template.ipynb

Before --to script was added, the option was --to python or --to=python, but it was renamed in the move toward a language-agnostic notebook system.

16
  • 12
    If you DO want one every save, in jupyter you can trigger nbconvert via pre- or post-save hooks: ContentsManager.pre_save_hook abd FileContentsManager.post_save_hook. You would add a post-save hook jupyter nbconvert --to script [notebook]
    – jaimedash
    May 3, 2016 at 16:35
  • 3
    Is there a way to do the reverse i.e convert from a python script to a notebook. For ex - having some specialized docstrings that are parsed into cells ?
    – Sujen Shah
    Jan 17, 2017 at 17:40
  • 3
    convert all notebooks in a folder jupyter nbconvert --to script /path/to/notebooks/*.ipynb
    – openwonk
    Jul 22, 2017 at 0:00
  • 12
    Thanks, it works!, but what if I don't want the # In[ ]: type stuff in the script, I want it to be clean. Is there any way to do that? Jan 31, 2018 at 10:48
  • 5
    @RishabhAgrahari Following seems to work for me. !jupyter nbconvert --to script --no-prompt notebook.ipynb
    – bytestorm
    May 27, 2019 at 7:09
134

If you want to convert all *.ipynb files from current directory to python script, you can run the command like this:

jupyter nbconvert --to script *.ipynb
2
  • Or you can run: ipython nbconvert --to script *.ipynb
    – Andrey
    Dec 23, 2020 at 11:28
  • to convert programmatically as in a function use this syntax : subprocess.run(['jupyter', 'nbconvert',ipynb_file,"--to", "script","--output",r'temp_converted_py'],shell=True) note that ipynb_file is a variable Dec 28, 2021 at 18:49
26

Here is a quick and dirty way to extract the code from V3 or V4 ipynb without using ipython. It does not check cell types, etc.

import sys,json

f = open(sys.argv[1], 'r') #input.ipynb
j = json.load(f)
of = open(sys.argv[2], 'w') #output.py
if j["nbformat"] >=4:
        for i,cell in enumerate(j["cells"]):
                of.write("#cell "+str(i)+"\n")
                for line in cell["source"]:
                        of.write(line)
                of.write('\n\n')
else:
        for i,cell in enumerate(j["worksheets"][0]["cells"]):
                of.write("#cell "+str(i)+"\n")
                for line in cell["input"]:
                        of.write(line)
                of.write('\n\n')

of.close()
3
  • 4
    Best answer if you do not want to install any of the Jupyter tools.
    – dacracot
    Oct 3, 2016 at 14:38
  • 2
    I like this. But I found out when I download .py format from Jupyter notebooks, it uses UNIX line endings even though I'm on windows. To generate the same, add the newlines='\n' as a third argument in the open output file call. (Python 3.x)
    – RufusVS
    May 30, 2019 at 19:19
  • This answer opens the possibility to read tags and extract only those cells that have it. Harder to do via the command line alone. Aug 9, 2020 at 23:57
20

Following the previous example but with the new nbformat lib version :

import nbformat
from nbconvert import PythonExporter

def convertNotebook(notebookPath, modulePath):

  with open(notebookPath) as fh:
    nb = nbformat.reads(fh.read(), nbformat.NO_CONVERT)

  exporter = PythonExporter()
  source, meta = exporter.from_notebook_node(nb)

  with open(modulePath, 'w+') as fh:
    fh.writelines(source.encode('utf-8'))
3
  • 1
    's last line of code, fh.writelines(source.encode('utf-8')) gives 'TypeError: write() argument must be str, not int' fh.writelines(source) works though.
    – BarryC
    Oct 18, 2016 at 5:43
  • Had the same issue, fixed by dropping the .encode('utf-8') piece on the last line.
    – LFoos24
    Jun 28, 2021 at 17:18
  • I solved the last line problem writing fh.write(source) instead of fh.writelines(...).
    – mosc9575
    Nov 24, 2021 at 14:33
19

Jupytext is nice to have in your toolchain for such conversions. It allows not only conversion from a notebook to a script, but you can go back again from the script to notebook as well. And even have that notebook produced in executed form.

jupytext --to py notebook.ipynb                 # convert notebook.ipynb to a .py file
jupytext --to notebook notebook.py              # convert notebook.py to an .ipynb file with no outputs
jupytext --to notebook --execute notebook.py    # convert notebook.py to an .ipynb file and run it 
3
  • Apparently there is also ipynb-py-convert, see here.
    – Wayne
    Jan 10, 2020 at 18:34
  • 1
    'jupytext' is not recognized as an internal or external command, operable program or batch file.??? Mar 31, 2020 at 15:53
  • Have you installed it @AmineChadi . See here for how to do that. If you are using it via a notebook as your command line interface, you can just run %pip install jupytext in your notebook.
    – Wayne
    Mar 31, 2020 at 20:16
8

You can do this from the IPython API.

from IPython.nbformat import current as nbformat
from IPython.nbconvert import PythonExporter

filepath = 'path/to/my_notebook.ipynb'
export_path = 'path/to/my_notebook.py'

with open(filepath) as fh:
    nb = nbformat.reads_json(fh.read())

exporter = PythonExporter()

# source is a tuple of python source code
# meta contains metadata
source, meta = exporter.from_notebook_node(nb)

with open(export_path, 'w+') as fh:
    fh.writelines(source)
0
7

I understand this is an old thread. I have faced the same issue and wanted to convert the .pynb file to .py file via command line.

My search took me to ipynb-py-convert

By following below steps I was able to get .py file

  1. Install "pip install ipynb-py-convert"
  2. Go to the directory where the ipynb file is saved via command prompt
  3. Enter the command

> ipynb-py-convert YourFileName.ipynb YourFilename.py

Eg:. ipynb-py-convert getting-started-with-kaggle-titanic-problem.ipynb getting-started-with-kaggle-titanic-problem.py

Above command will create a python script with the name "YourFileName.py" and as per our example it will create getting-started-with-kaggle-titanic-problem.py file

0
5

For converting all *.ipynb format files in current directory to python scripts recursively:

for i in *.ipynb **/*.ipynb; do 
    echo "$i"
    jupyter nbconvert  "$i" "$i"
done
1
  • 4
    I had to add the --to script argument to avoid the default HTML output in Jupiter 4.4.0.
    – trojjer
    Nov 21, 2017 at 13:51
5

Using nbconvert 6.07 and jupyter client 6.1.12:

Convert jupyter notebook to python script

$ jupyter nbconvert mynotebook.ipynb --to python

Convert jupyter notebook to python script specifying output filename

$ jupyter nbconvert mynotebook.ipnb --to python --output myscript.py
3

The following example turns an Iron Python Notebook called a_notebook.ipynb into a python script called a_python_script.py leaving out the cells tagged with the keyword remove, which I add manually to the cells that I don't want to end up in the script, leaving out visualizations and other steps that once I am done with the notebook I don't need to be executed by the script.

import nbformat as nbf
from nbconvert.exporters import PythonExporter
from nbconvert.preprocessors import TagRemovePreprocessor

with open("a_notebook.ipynb", 'r', encoding='utf-8') as f:
    the_notebook_nodes = nbf.read(f, as_version = 4)

trp = TagRemovePreprocessor()

trp.remove_cell_tags = ("remove",)

pexp = PythonExporter()

pexp.register_preprocessor(trp, enabled= True)

the_python_script, meta = pexp.from_notebook_node(the_notebook_nodes)

with open("a_python_script.py", 'w', encoding='utf-8') as f:
    f.writelines(the_python_script)
2

There's a very nice package called nb_dev which is designed for authoring Python packages in Jupyter Notebooks. Like nbconvert, it can turn a notebook into a .py file, but it is more flexible and powerful because it has a lot of nice additional authoring features to help you develop tests, documentation, and register packages on PyPI. It was developed by the fast.ai folks.

It has a bit of a learning curve, but the documentation is good and it is not difficult overall.

2

I found that there are 2 ways to convert Jupyter Notebook to plain Python script with command line. Below is the example Jupyter notebook and output from both tools.

enter image description here

1. Use nbconvert

The nbconvert is tool that is used in Jupyter Notebook User Interface in Download as functionality. It can be used as command line tool:

jupyter nbconvert --to python notebook.ipynb

Example Python script: enter image description here

2. Use jupytext

The jupytext is a package to keep .ipynb file synchronized with .py file. It can be also used to convert .ipynb files in command line. It supports several types of conversion:

Convert to Python script with light format

jupytext --to py notebook.ipynb             

Example python script:

enter image description here

Convert to Python script with percent format

jupytext --to py:percent notebook.ipynb 

Example Python script:

enter image description here

1

no file/directory error

On my mint [ubuntu] system at work, even though jupyter was already installed and notebooks worked, jupyter nbconvert --to script gave the error no file/directory until I did a separate

sudo apt-get install jupyter-nbconvert

Then all was fine with the conversion. I just wanted to add this in case anyone hits the same error (for me it was confusing as I thought the no file error referred to the notebook, which was definitely there in the local directory, took me a while to realize the subcommand was not installed).

0

Here is a jq solution that could be situationally useful. Remember, notebooks are just json.

jq -r '.cells[] | select(.cell_type  == "code") | .source[] | rtrimstr("\n")' $filename
0

The given solution works only for converting a single .py file. Here is a solution that converts all .py files in a directory and sub-directories.

First, you need to install the tool that convert only one file at a time like ipynb-py-convert

pip install ipynb-py-convert

Then cd into the place of your folder having the .py files and directories. Then we run the tool recursively on all files in the directory and subdirectories:

In powershell:

foreach ($f in Get-ChildItem "." -Filter *.ipynb -Recurse){ ipynb-py-convert $f.FullName "$($f.FullName.Substring(0,$f.FullName.Length-6)).py"}

Now if you would like to convert the otherway around from .ipynb to .py also with batch convert, you can run:

foreach ($f in Get-ChildItem "." -Filter *.py -Recurse){ ipynb-py-convert $f.FullName "$($f.FullName.Substring(0,$f.FullName.Length-3)).ipynb"}

This helped me a lot while exploring .py files. I make a copy of the project, run this code and quickly in Jupiter test different parts of the code as cells and so on. I hope it helps more people.

0
0

I have built a function to achieve this. The user does not have to install anything to use it.

#!/usr/bin/python


# A short routine to convert a Jupyter Notebook to a Python file

import json

def ipynb_to_py(input_ipynb_file,output_py_file=None):
    """
    Generate a Python script (.py) that includes all source code from the input Jupyter notebook (.ipynb).
    
    The user can input a Jupyter Notebook file from the current working directory or from a path.
    
    If the name for output Python file is not specified, 
    the output file name will copy the file name of the input Jupyter Notebook, 
    but the file exention will be changed from ".ipynb" chanegd to ".py".
    And the output Python file will be saved at the same directory of the input Jupyter Notebook.
    For example:
    ipynb_to_py("test-jupyternotebook.ipynb")
    ipynb_to_py("./test-input-dir/test-jupyternotebook.ipynb")
    
    The user can also specify an output file name that ends with ".py".
    If the output file name is provided, but no path to output file is added, 
    the file will be saved at the current working directory.
    For example:
    ipynb_to_py("test-jupyternotebook.ipynb","test1.py")
    ipynb_to_py("./test-input-dir/test-jupyternotebook.ipynb","test2.py")
        
    The user can save out the file at a target directory by adding a path to the output file.
    For example: 
    ipynb_to_py("test-jupyternotebook.ipynb","./test-outputdir/test3.py")
    ipynb_to_py("./test-input-dir/test-jupyternotebook.ipynb","./test-output-dir/test4.py")
    
    This function does not edit or delete the original input Jupyter Notebook file.
    
    Args:
    -----
        input_ipynb_file: The file name string for the Jupyter Notebook (ends with ".ipynb")
        output_py_file (optional): The file name for Python file to be created (ends with ".py"). 
    
    Returns:
    --------
        A Python file containing all source code in the Jupyter Notebook.
        
    Example usages:
    ---------------
        ipynb_to_py("test-jupyternotebook.ipynb")
        ipynb_to_py("./test-input-dir/test-jupyternotebook.ipynb")
        ipynb_to_py("test-jupyternotebook.ipynb","test1.py")
        ipynb_to_py("test-jupyternotebook.ipynb","./test-outputdir/test2.py")
        ipynb_to_py("test-jupyternotebook.ipynb","./test-outputdir/test3.py")
        ipynb_to_py("./test-input-dir/test-jupyternotebook.ipynb","./test-output-dir/test4.py")
             
    """
    # Check if the input file is a Jupyter Notebook
    if input_ipynb_file.endswith(".ipynb"):
        
        # Open the input Jupyter Notebook file
        notebook = open(input_ipynb_file)
        
        # Read its content in the json format
        notebook_content = json.load(notebook)

        # Only extract the source code snippet from each cell in the input Jupyter Notebook
        source_code_snippets = [cell['source'] for cell in notebook_content['cells']]
        
        # If the name for output Python file is not specified,
        # The name of input Jupyter Notebook will be used after changing ".ipynb" to ".py".
        if output_py_file == None:
            output_py_file = input_ipynb_file.split('.ipynb')[0]+".py"
        else:
            pass

        # Create a Python script to save out all the extracted source code snippets
        output_file = open(output_py_file,'w')

        # Print out each line in each source code snippet to the output file
        for snippet in source_code_snippets:
            for line in snippet:
                # Use end='' to avoid creating unwanted gaps between lines
                print(line,end = '',file = output_file)
            # At end of each snippet, move to the next line before printing the next one
            print('',sep = '\n',file=output_file)

        # Close the output file
        output_file.close()
        print("The path to output file:",output_py_file)
        
    else:
        print("The input file must be a Jupyter Notebook (in .ipynb format)!")
        
def main():
    pass

if __name__ == "__main__":
    main()
-1

I had this problem and tried to find the solution online. Though I found some solutions, they still have some problems, e.g., the annoying Untitled.txt auto-creation when you start a new notebook from the dashboard.

So eventually I wrote my own solution:

import io
import os
import re
from nbconvert.exporters.script import ScriptExporter
from notebook.utils import to_api_path


def script_post_save(model, os_path, contents_manager, **kwargs):
    """Save a copy of notebook to the corresponding language source script.

    For example, when you save a `foo.ipynb` file, a corresponding `foo.py`
    python script will also be saved in the same directory.

    However, existing config files I found online (including the one written in
    the official documentation), will also create an `Untitile.txt` file when
    you create a new notebook, even if you have not pressed the "save" button.
    This is annoying because we usually will rename the notebook with a more
    meaningful name later, and now we have to rename the generated script file,
    too!

    Therefore we make a change here to filter out the newly created notebooks
    by checking their names. For a notebook which has not been given a name,
    i.e., its name is `Untitled.*`, the corresponding source script will not be
    saved. Note that the behavior also applies even if you manually save an
    "Untitled" notebook. The rationale is that we usually do not want to save
    scripts with the useless "Untitled" names.
    """
    # only process for notebooks
    if model["type"] != "notebook":
        return

    script_exporter = ScriptExporter(parent=contents_manager)
    base, __ = os.path.splitext(os_path)

    # do nothing if the notebook name ends with `Untitled[0-9]*`
    regex = re.compile(r"Untitled[0-9]*$")
    if regex.search(base):
        return

    script, resources = script_exporter.from_filename(os_path)
    script_fname = base + resources.get('output_extension', '.txt')

    log = contents_manager.log
    log.info("Saving script at /%s",
             to_api_path(script_fname, contents_manager.root_dir))

    with io.open(script_fname, "w", encoding="utf-8") as f:
        f.write(script)

c.FileContentsManager.post_save_hook = script_post_save

To use this script, you can add it to ~/.jupyter/jupyter_notebook_config.py :)

Note that you may need to restart the jupyter notebook / lab for it to work.

1
-2

The %notebook foo.ipynb magic command will export the current IPython to "foo.ipynb".

More info by typing %notebook?

Not the answer you're looking for? Browse other questions tagged or ask your own question.