6

I am having a very weird behavior for matplotlib's savefig function. I am doing as follows:

import os

# creating the output folder 
if not os.path.exists(output_folder):
   os.makedirs(output_folder)

# adding the name of the figure to the created output folder
plt.savefig(output_folder + 'forecasts_scatter_%s' % model_name)
# plt.savefig(os.path.join(output_folder, 'forecasts_scatter_%s.png' % model_name))
plt.close()

The errors:

Traceback (most recent call last):
  File "C:/Users/96171/Desktop/ministry_of_public_health/CodeUbrCorrected/bmwmlmcw.py", line 56, in <module>
    lm.cross_validation(model, hyperparameters[model_name], model_name)
  File "C:\Users\96171\Desktop\ministry_of_public_health\CodeUbrCorrected\cross_validation_smogn.py", line 554, in cross_validation
    self.cross_validation_grid(model_used, hyperparams, model_name)
  File "C:\Users\96171\AppData\Local\Programs\Python\Python36\lib\site-packages\sklearn\utils\testing.py", line 348, in wrapper
    return fn(*args, **kwargs)
  File "C:\Users\96171\Desktop\ministry_of_public_health\CodeUbrCorrected\cross_validation_smogn.py", line 1444, in cross_validation_grid
    'predicted')
  File "C:\Users\96171\Desktop\ministry_of_public_health\CodeUbrCorrected\cross_validation_smogn.py", line 1772, in plot_actual_vs_predicted_scatter_bisector
    plt.savefig(output_folder + 'forecasts_scatter_%s' % model_name)
  File "C:\Users\96171\AppData\Local\Programs\Python\Python36\lib\site-packages\matplotlib\pyplot.py", line 689, in savefig
    res = fig.savefig(*args, **kwargs)
  File "C:\Users\96171\AppData\Local\Programs\Python\Python36\lib\site-packages\matplotlib\figure.py", line 2094, in savefig
    self.canvas.print_figure(fname, **kwargs)
  File "C:\Users\96171\AppData\Local\Programs\Python\Python36\lib\site-packages\matplotlib\backend_bases.py", line 2075, in print_figure
    **kwargs)
  File "C:\Users\96171\AppData\Local\Programs\Python\Python36\lib\site-packages\matplotlib\backends\backend_agg.py", line 521, in print_png
    cbook.open_file_cm(filename_or_obj, "wb") as fh:
  File "C:\Users\96171\AppData\Local\Programs\Python\Python36\lib\contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "C:\Users\96171\AppData\Local\Programs\Python\Python36\lib\site-packages\matplotlib\cbook\__init__.py", line 407, in open_file_cm
    fh, opened = to_filehandle(path_or_file, mode, True, encoding)
  File "C:\Users\96171\AppData\Local\Programs\Python\Python36\lib\site-packages\matplotlib\cbook\__init__.py", line 392, in to_filehandle
    fh = open(fname, flag, encoding=encoding)
FileNotFoundError: [Errno 2] No such file or directory: '../output_ubr_shallow/all_columns_minus_weather_minus_lags_minus_civilians_balance/smogn_0.95_Manhattan_rel_mat1/grid_search/train_test_forecasts_scatterplot_bisector/forecasts_scatter_lasso.png'

The output I am specify is a relative path

'../output_ubr_shallow/all_columns_minus_weather_minus_lags_minus_civilians_balance/smogn_0.95_Manhattan_rel_mat1/grid_search/train_test_forecasts_scatterplot_bisector/'

and when I do os.path.exists(output_folder) it gives me True.

I know that we can specify relative paths in savefig. Otherwise what am I doing wrong ?

I also tried this:

# changing the .png thing and using os.path.join
plt.savefig(os.path.join(output_folder, 'forecasts_scatter_' + model_name + '.png'))

and this:

# changing the .png thing and using os.path.join
plt.savefig(os.path.join(output_folder, 'forecasts_scatter_%s.png' % model_name))

But still getting the same results.

Whats very weird I ran the same code from different script. I modified a bit the code to make it as follows:

 p = output_folder + 'forecasts_scatter_' + model_name + '.png'
 filename = os.path.split(p)[1]
 directory = os.path.split(p)[0]

 if not os.path.exists(directory):
    os.makedirs(directory)

 savepath = os.path.join(directory, filename)
 print('save path isss: {}'.format(savepath))
 plt.savefig(savepath)
 plt.close()

The first script has this as the output path:

save path isss: ../output_ubr_shallow/all_columns_balance/smogn_0.95_Manhattan_rel_mat1/grid_search/train_test_forecasts_scatterplot_bisector\forecasts_scatter_lasso.png

and the second script has this as the output path

save path isss: ../output_ubr_shallow/all_columns_minus_weather_minus_lags_minus_civilians_balance/smogn_0.95_Manhattan_rel_mat1/grid_search/train_test_forecasts_scatterplot_bisector\forecasts_scatter_lasso.png

The first is working and the second is not. Both scripts are found in the same directory!!!

3

2 Answers 2

13

Wow after spending many hours on this I just replaced this:

this:

plt.savefig(os.path.join(output_folder, 'forecasts_scatter_%s.png' % model_name))

by this

plt.savefig(os.path.join(output_folder, 'scatter_%s.png' % model_name))

and it worked. I'm not sure if this is a possible bug in matplotlib. I think the length of the path in savefig has some limits.

I opened an issue on github https://github.com/matplotlib/matplotlib/issues/17313

0

This issue is addressed in Pathname too long to open?

It is not related to matplotlib or Python, it's MS-DOS that has a maximum length to path strings of 260 characters. A workaround suggested in the above link is to use an extended path string, by prefixing your path with \\??\. I have not tested it, but in your case you should be able to do:

output_folder = r'\\??\..\output_ubr_shallow\all_columns_minus_weather_minus_lags_minus_civilians_balance\smogn_0.95_Manhattan_rel_mat1\grid_search\train_test_forecasts_scatterplot_bisector\'
plt.savefig(os.path.join(output_folder, 'forecasts_scatter_%s.png' % model_name))

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.