473

I find myself frequently using Python's interpreter to work with databases, files, etc -- basically a lot of manual formatting of semi-structured data. I don't properly save and clean up the useful bits as often as I would like. Is there a way to save my input into the shell (db connections, variable assignments, little for loops and bits of logic) -- some history of the interactive session? If I use something like script I get too much stdout noise. I don't really need to pickle all the objects -- though if there is a solution that does that, it would be OK. Ideally I would just be left with a script that ran as the one I created interactively, and I could just delete the bits I didn't need. Is there a package that does this, or a DIY approach?

20 Answers 20

460

IPython is extremely useful if you like using interactive sessions. For example for your use-case there is the %save magic command, you just input %save my_useful_session 10-20 23 to save input lines 10 to 20 and 23 to my_useful_session.py (to help with this, every line is prefixed by its number).

Furthermore, the documentation states:

This function uses the same syntax as %history for input ranges, then saves the lines to the filename you specify.

This allows for example, to reference older sessions, such as

%save current_session ~0/
%save previous_session ~1/

Look at the videos on the presentation page to get a quick overview of the features.

17
  • 15
    How to save all the lines? Without specifying the range, it creates an empty file. :(
    – balki
    Commented Sep 1, 2011 at 19:37
  • 47
    @balki, IPython's prompt tells you how many lines are in your history (i.e. In[48]). So save filename 1-48 would save your whole session.
    – Ben Page
    Commented Sep 22, 2011 at 10:51
  • 2
    Also, is it possible to load this file back into ipython and keep your input history intact?
    – Ben Page
    Commented Sep 22, 2011 at 10:58
  • 18
    @BenPage Use "ipython -i [filename]" on the saved .py file, from the bash promt in order to load the file back before returning to an interactive console! (without the -i flag you don't get the interactive console after running the file). Commented Nov 29, 2012 at 14:30
  • 12
    @user4779, as one would expect: %load my_useful_session Commented Jan 5, 2017 at 2:23
218

From Andrew Jones's website (archived):

import readline
readline.write_history_file('/home/ahj/history')
15
  • 12
    Creates an empty file for me on windows 7 Commented May 14, 2013 at 7:19
  • 14
    Worked like a charm for me in Ubuntu.
    – Johndt
    Commented Sep 5, 2014 at 17:06
  • 5
    @ubershmekel - Looks like it only works on Unix
    – Joel B
    Commented Feb 5, 2015 at 15:04
  • 44
    Fails on my TI-83
    – Dan
    Commented Jan 13, 2016 at 18:13
  • 4
    Remember to do it at the end of your session. Mine was looking a bit empty after calling the function only at the start.
    – dolphus333
    Commented Apr 27, 2016 at 22:50
101

There is a way to do it. Store the file in ~/.pystartup...

# Add auto-completion and a stored history file of commands to your Python
# interactive interpreter. Requires Python 2.0+, readline. Autocomplete is
# bound to the Esc key by default (you can change it - see readline docs).
#
# Store the file in ~/.pystartup, and set an environment variable to point
# to it:  "export PYTHONSTARTUP=/home/user/.pystartup" in bash.
#
# Note that PYTHONSTARTUP does *not* expand "~", so you have to put in the
# full path to your home directory.

import atexit
import os
import readline
import rlcompleter

historyPath = os.path.expanduser("~/.pyhistory")

def save_history(historyPath=historyPath):
    import readline
    readline.write_history_file(historyPath)

if os.path.exists(historyPath):
    readline.read_history_file(historyPath)

atexit.register(save_history)
del os, atexit, readline, rlcompleter, save_history, historyPath

and then set the environment variable PYTHONSTARTUP in your shell (e.g. in ~/.bashrc):

export PYTHONSTARTUP=$HOME/.pystartup

You can also add this to get autocomplete for free:

readline.parse_and_bind('tab: complete')

Please note that this will only work on *nix systems. As readline is only available in Unix platform.

6
  • Mac OS X uses editline, so there is tab-complete functionality available, but the exact command is different: readline.parse_and_bind("bind ^I rl_complete")
    – Miles
    Commented Jun 3, 2009 at 23:45
  • That was crazy fast, Nadia, many thanks. I will try both of the answers -- target platform is Ubuntu, BTW
    – unmounted
    Commented Jun 4, 2009 at 0:01
  • readline.parse_and_bind('tab: complete') works if you use MacPorts Python. Commented Jun 5, 2012 at 19:04
  • 3
    This should be framed. Worked on RHEL 6.6. Pyenv, 2.7.10.
    – BilliAm
    Commented Sep 16, 2015 at 22:27
  • Why I didn't search for this all of these years! This solution is great as uses original tool that is fast and light.
    – iman
    Commented Jun 17, 2017 at 18:39
79

If you are using IPython you can save to a file all your previous commands using the magic function %history with the -f parameter, p.e:

%history -f /tmp/history.py
2
  • Wasnt sure about the location of the file using save magic. This was very helpful
    – alpha_989
    Commented Feb 2, 2018 at 18:30
  • 6
    Use %history -g -f full_history.py to get history from all sessions, not just the current one. See also: documentation for %history. Commented Mar 28, 2018 at 0:38
23

After installing Ipython, and opening an Ipython session by running the command:

ipython

from your command line, just run the following Ipython 'magic' command to automatically log your entire Ipython session:

%logstart

This will create a uniquely named .py file and store your session for later use as an interactive Ipython session or for use in the script(s) of your choosing.

14

Also, reinteract gives you a notebook-like interface to a Python session.

1
  • 4
    reinteract is now superseded by the IPython Notebook and QtConsole. Commented Jan 26, 2014 at 17:17
11

In addition to IPython, a similar utility bpython has a "save the code you've entered to a file" feature

3
  • 3
    can anybody explain how this is done with bpython? I tried ctrl+s but it didn't work (version 0.14.2 on top of Python 3.4.3 running Gnome terminal)
    – Yibo Yang
    Commented Jan 25, 2016 at 7:38
  • It's bound to F7 by default. F1 will show you the help and current keybindings.
    – Jim K.
    Commented Jun 6, 2017 at 20:30
  • F7 launches the external editor. Ctrl+s should prompt for a simple save to file feature, but it's bound to "stop" in bash and zsh, so you'll need to choose an alternative in the bpython config.
    – RCross
    Commented Feb 21, 2020 at 11:22
11

I had to struggle to find an answer, I was very new to iPython environment.

This will work

If your iPython session looks like this

In [1] : import numpy as np
....
In [135]: counter=collections.Counter(mapusercluster[3])
In [136]: counter
Out[136]: Counter({2: 700, 0: 351, 1: 233})

You want to save lines from 1 till 135 then on the same ipython session use this command

In [137]: %save test.py 1-135

This will save all your python statements in test.py file in your current directory ( where you initiated the ipython).

1
  • how to do this in pycharm with anaconda
    – Saran Zeb
    Commented May 19, 2022 at 17:48
4

There is %history magic for printing and saving the input history (and optionally the output).

To store your current session to a file named my_history.py:

>>> %hist -f my_history.py

History IPython stores both the commands you enter, and the results it produces. You can easily go through previous commands with the up- and down-arrow keys, or access your history in more sophisticated ways.

You can use the %history magic function to examine past input and output. Input history from previous sessions is saved in a database, and IPython can be configured to save output history.

Several other magic functions can use your input history, including %edit, %rerun, %recall, %macro, %save and %pastebin. You can use a standard format to refer to lines:

%pastebin 3 18-20 ~1/1-5

This will take line 3 and lines 18 to 20 from the current session, and lines 1-5 from the previous session.

See %history? for the Docstring and more examples.

Also, be sure to explore the capabilities of %store magic for lightweight persistence of variables in IPython.

Stores variables, aliases and macros in IPython’s database.

d = {'a': 1, 'b': 2}
%store d  # stores the variable
del d

%store -r d  # Refresh the variable from IPython's database.
>>> d
{'a': 1, 'b': 2}

To autorestore stored variables on startup, specifyc.StoreMagic.autorestore = True in ipython_config.py.

1
  • Enabling saving output history can be done by enabling c.HistoryManager.db_log_output in the config file.
    – Ikke
    Commented Jun 3, 2016 at 7:34
4

The %history command is awesome, but unfortunately it won't let you save things that were %paste 'd into the sesh. To do that I think you have to do %logstart at the beginning (although I haven't confirmed this works).

What I like to do is

%history -o -n -p -f filename.txt

which will save the output, line numbers, and '>>>' before each input (o, n, and p options). See the docs for %history here.

3

Just putting another suggesting in the bowl: Spyder

enter image description here

It has History log and Variable explorer. If you have worked with MatLab, then you'll see the similarities.

0
2

As far as Linux goes, one can use script command to record the whole session. It is part of util-linux package so should be on most Linux systems . You can create and alias or function that will call script -c python and that will be saved to a typescript file. For instance, here's a reprint of one such file.

$ cat typescript                                                                                                      
Script started on Sat 14 May 2016 08:30:08 AM MDT
Python 2.7.6 (default, Jun 22 2015, 17:58:13) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> print 'Hello Pythonic World'
Hello Pythonic World
>>> 

Script done on Sat 14 May 2016 08:30:42 AM MDT

Small disadvantage here is that the script records everything , even line-feeds, whenever you hit backspaces , etc. So you may want to use col to clean up the output (see this post on Unix&Linux Stackexchange) .

1

there is another option --- pyslice. in the "wxpython 2.8 docs demos and tools", there is a open source program named "pyslices".

you can use it like a editor, and it also support using like a console ---- executing each line like a interactive interpreter with immediate echo.

of course, all the blocks of codes and results of each block will be recorded into a txt file automatically.

the results are logged just behind the corresponding block of code. very convenient.

the overview of pyslices

1
  • PySlices author here. Glad you like PySlices (and that you even found it). It's not quite abandon-ware yet (I actually still use it), but the version included in wxPython itself often doesn't work well. If you want to follow updates you can get them from the wx_py package: pypi.python.org/pypi/wx_py, github.com/davidmashburn/wx_py
    – David
    Commented Sep 23, 2016 at 16:40
1

Some comments were asking how to save all of the IPython inputs at once. For %save magic in IPython, you can save all of the commands programmatically as shown below, to avoid the prompt message and also to avoid specifying the input numbers. currentLine = len(In)-1 %save -f my_session 1-$currentLine

The -f option is used for forcing file replacement and the len(IN)-1 shows the current input prompt in IPython, allowing you to save the whole session programmatically.

1

In IPython, I first use

In [2]: %hist

to view my past code. I select the chunk I want to save and then paste it into file my_file.py using the %%file magic (short for %%writefile)

In [3]: %%file my_file.py
   ...: # paste code here 
   ...:  
   ...:  

hitting return two times in the end.

To append to file use the option -a: %%file -a my_file.py.

If needed, I can list, edit, etc. the file in the underlying command line using the exclamation mark

In [5]: !ls -l my_file.py
In [6]: !vi my_file.py
0

For those using spacemacs, and ipython that comes with python-layer, save magic creates a lot of unwanted output, because of the constant auto-completion command working in the backround such as:

len(all_suffixes)
';'.join(__PYTHON_EL_get_completions('''len'''))
';'.join(__PYTHON_EL_get_completions('''all_substa'''))
len(all_substantives_w_suffixes)
';'.join(__PYTHON_EL_get_completions('''len'''))
';'.join(__PYTHON_EL_get_completions('''all'''))
';'.join(__PYTHON_EL_get_completions('''all_'''))
';'.join(__PYTHON_EL_get_completions('''all_w'''))
';'.join(__PYTHON_EL_get_completions('''all_wo'''))
';'.join(__PYTHON_EL_get_completions('''all_wor'''))
';'.join(__PYTHON_EL_get_completions('''all_word'''))
';'.join(__PYTHON_EL_get_completions('''all_words'''))
len(all_words_w_logograms)
len(all_verbs)

To avoid this just save the ipython buffer like you normally save any other: spc f s

0

If you use bpython, all your command history is by default saved to ~/.pythonhist.

To save the commands for later reusage you can copy them to a python script file:

$ cp ~/.pythonhist mycommands.py

Then edit that file to clean it up and put it under Python path (global or virtual environment's site-packages, current directory, mentioning in *.pth, or some other way).

To include the commands into your shell, just import them from the saved file:

>>> from mycommands import *
1
  • life savior. a hard to find answer Commented Dec 11, 2019 at 14:51
0

I'd like to suggest another way to maintain python session through tmux on linux. you run tmux, attach your self to the session you opened (if not attached after opening it directly). execute python and do whatever you are doing on it. then detach from session. detaching from a tmux session does not close the session. the session remains open.

pros of this method: you can attach to this session from any other device (in case you can ssh your pc)

cons of this method: this method does not relinquish the resources used by the opened python session until you actually exist the python interpreter.

0

To save input and output on XUbuntu:

  1. In XWindows, run iPython from the Xfce terminal app
  2. click Terminal in the top menu bar and look for save contents in the dropdown

I find this saves the input and output, going all the way back to when I opened the terminal. This is not ipython specific, and would work with ssh sessions or other tasks run from the terminal window.

-1

You can use built-in function open: I use it in all of my programs in which I need to store some history (Including Calculator, etc.) for example:

#gk-test.py or anything else would do
try: # use the try loop only if you haven't created the history file outside program
    username = open("history.txt").readline().strip("\n")
    user_age = open("history.txt").readlines()[1].strip("\n")
except FileNotFoundError:
    username = input("Enter Username: ")
    user_age = input("Enter User's Age: ")
    open("history.txt", "w").write(f"{username}\n{user_age}")
#Rest of the code is secret! try it your own!

I would thank to all of them who liked my comments! Thank you for reading this!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.