233

I am using tqdm to print progress in a script I'm running in a Jupyter notebook. I am printing all messages to the console via tqdm.write(). However, this still gives me a skewed output like so:

enter image description here

That is, each time a new line has to be printed, a new progress bar is printed on the next line. This does not happen when I run the script via terminal. How can I solve this?

3

12 Answers 12

362

Try using tqdm.notebook.tqdm instead of tqdm, as outlined here.

This could be as simple as changing your import to:

from tqdm.notebook import tqdm

EDIT: After testing, it seems that tqdm actually works fine in 'text mode' in Jupyter notebook. It's hard to tell because you haven't provided a minimal example, but it looks like your problem is caused by a print statement in each iteration. The print statement is outputting a number (~0.89) in between each status bar update, which is messing up the output. Try removing the print statement.

11
  • 5
    I haven't used a print() statement, I used tqdm.write(). However, tqdm_notebook gives good results. Thanks : ) Feb 14, 2017 at 9:00
  • Do you know if it supports Python 3.6? I've not had luck with this
    – Jon
    Oct 8, 2017 at 17:59
  • 1
    What error are you getting? It works fine for me. Impossible to help with so little info... Have you enabled ipywidgets in jupyer? Have you just plain tqdm, rather than tqdm_notebook? This works well with Python 3.6 and Jupyter 1.0.0. Oct 10, 2017 at 3:11
  • tqdm_notebook from tqdm 4.19.4 is working for me on Python 3.6, Jupyter notebook 5.0.0, and ipywidgets 7.0.3. Oct 31, 2017 at 21:50
  • 2
    This seems to be an easiest way: from tqdm.autonotebook import tqdm
    – Iopheam
    Nov 3, 2020 at 15:43
55

This is an alternative answer for the case where tqdm_notebook doesn't work for you.

Given the following example:

from time import sleep
from tqdm import tqdm

values = range(3)
with tqdm(total=len(values)) as pbar:
    for i in values:
        pbar.write('processed: %d' %i)
        pbar.update(1)
        sleep(1)

The output would look something like this (progress would show up red):

  0%|          | 0/3 [00:00<?, ?it/s]
processed: 1
 67%|██████▋   | 2/3 [00:01<00:00,  1.99it/s]
processed: 2
100%|██████████| 3/3 [00:02<00:00,  1.53it/s]
processed: 3

The problem is that the output to stdout and stderr are processed asynchronously and separately in terms of new lines.

If say Jupyter receives on stderr the first line and then the "processed" output on stdout. Then once it receives an output on stderr to update the progress, it wouldn't go back and update the first line as it would only update the last line. Instead it will have to write a new line.

Workaround 1, writing to stdout

One workaround would be to output both to stdout instead:

import sys
from time import sleep
from tqdm import tqdm

values = range(3)
with tqdm(total=len(values), file=sys.stdout) as pbar:
    for i in values:
        pbar.write('processed: %d' % (1 + i))
        pbar.update(1)
        sleep(1)

The output will change to (no more red):

processed: 1   | 0/3 [00:00<?, ?it/s]
processed: 2   | 0/3 [00:00<?, ?it/s]
processed: 3   | 2/3 [00:01<00:00,  1.99it/s]
100%|██████████| 3/3 [00:02<00:00,  1.53it/s]

Here we can see that Jupyter doesn't seem to clear until the end of the line. We could add another workaround for that by adding spaces. Such as:

import sys
from time import sleep
from tqdm import tqdm

values = range(3)
with tqdm(total=len(values), file=sys.stdout) as pbar:
    for i in values:
        pbar.write('processed: %d%s' % (1 + i, ' ' * 50))
        pbar.update(1)
        sleep(1)

Which gives us:

processed: 1                                                  
processed: 2                                                  
processed: 3                                                  
100%|██████████| 3/3 [00:02<00:00,  1.53it/s]

Workaround 2, set description instead

It might in general be more straight forward not to have two outputs but update the description instead, e.g.:

import sys
from time import sleep
from tqdm import tqdm

values = range(3)
with tqdm(total=len(values), file=sys.stdout) as pbar:
    for i in values:
        pbar.set_description('processed: %d' % (1 + i))
        pbar.update(1)
        sleep(1)

With the output (description updated while it's processing):

processed: 3: 100%|██████████| 3/3 [00:02<00:00,  1.53it/s]

Conclusion

You can mostly get it to work fine with plain tqdm. But if tqdm_notebook works for you, just use that (but then you'd probably not read that far).

1
37

Most of the answers are outdated now. Better if you import tqdm correctly.

from tqdm import tqdm_notebook as tqdm

enter image description here

1
  • 25
    It changed again: TqdmDeprecationWarning: This function will be removed in tqdm==5.0.0 Please use tqdm.notebook.tqdm instead of tqdm.tqdm_notebook
    – stason
    Mar 5, 2020 at 4:00
27

To complete oscarbranson's answer: it's possible to automatically pick console or notebook versions of progress bar depending on where it's being run from:

from tqdm.autonotebook import tqdm

More info can be found here

1
  • That's great, thanks for the addition. Mar 16, 2022 at 11:48
21

None of the above works for me. I find that running the following sorts this issue after error (It just clears all the instances of progress bars in the background):

from tqdm import tqdm

# blah blah your code errored

tqdm._instances.clear()
2
  • 2
    Thank you! However it throws and error if no instances exists. Still want to use it with scripts and Hydrogen IDE. Here is my code. try: # Avoids problem on notebooks / Hydrogen IDE tqdm.tqdm._instances.clear() except Exception: pass Mar 22, 2020 at 21:19
  • Yup it will throw an exception if no instance exists. Is there an issue with your try except approach? Mar 29, 2020 at 13:26
19

If the other tips here don't work and - just like me - you're using the pandas integration through progress_apply, you can let tqdm handle it:

from tqdm.autonotebook import tqdm
tqdm.pandas()

df.progress_apply(row_function, axis=1)

The main point here lies in the tqdm.autonotebook module. As stated in their instructions for use in IPython Notebooks, this makes tqdm choose between progress bar formats used in Jupyter notebooks and Jupyter consoles - for a reason still lacking further investigations on my side, the specific format chosen by tqdm.autonotebook works smoothly in pandas, while all others didn't, for progress_apply specifically.

1
6

For everyone who is on windows and couldn't solve the duplicating bars issue with any of the solutions mentioned here. I had to install the colorama package as stated in tqdm's known issues which fixed it.

pip install colorama

Try it with this example:

from tqdm import tqdm
from time import sleep

for _ in tqdm(range(5), "All", ncols = 80, position = 0):
    for _ in tqdm(range(100), "Sub", ncols = 80, position = 1, leave = False):
        sleep(0.01)

Which will produce something like:

All:  60%|████████████████████████                | 3/5 [00:03<00:02,  1.02s/it]
Sub:  50%|██████████████████▌                  | 50/100 [00:00<00:00, 97.88it/s]
5

Using Python 3.9.2 and tqdm==4.62.3:

from tqdm.notebook import tqdm

for item in tqdm(list_of_items):
    do_something(item)
3

tqdm automatically loads the fitting progress bar, either for normal code or interactive environments via

from tqdm.auto import tqdm
0

Use tqdm_notebook

from tqdm import tqdm_notebook as tqdm

x=[1,2,3,4,5]

for i in tqdm(range(0,len(x))):

    print(x[i])
0

In my case I just needed to update ipywidgets to get rid of the extra printing when using regular from tqdm import tqdm.

0

Another Example for Progress Bar using tqdm

from tqdm import tqdm

my_list = list(range(100))
with tqdm(total=len(my_list)) as pbar:
    for x in my_list:
       pbar.update(1)
1
  • This doesn't solve the issue. May 17 at 11:52

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.