72

I encounter a RunTimeError while I am trying to run the code in my machine's CPU instead of GPU. The code is originally from this GitHub project - IBD: Interpretable Basis Decomposition for Visual Explanation. This is for a research project. I tried putting the CUDA as false and looked at other solutions on this website.

GPU = False               # running on GPU is highly suggested
CLEAN = False             # set to "True" if you want to clean the temporary large files after generating result
APP = "classification"    # Do not change! mode choide: "classification", "imagecap", "vqa". Currently "imagecap" and "vqa" are not supported.
CATAGORIES = ["object", "part"]   # Do not change! concept categories that are chosen to detect: "object", "part", "scene", "material", "texture", "color"

CAM_THRESHOLD = 0.5                 # the threshold used for CAM visualization
FONT_PATH = "components/font.ttc"   # font file path
FONT_SIZE = 26                      # font size
SEG_RESOLUTION = 7                  # the resolution of cam map
BASIS_NUM = 7                       # In decomposition, this is to decide how many concepts are used to interpret the weight vector of a class.

Here is the error:

Traceback (most recent call last):
  File "test.py", line 22, in <module>
    model = loadmodel()
  File "/home/joshuayun/Desktop/IBD/loader/model_loader.py", line 48, in loadmodel
    checkpoint = torch.load(settings.MODEL_FILE)
  File "/home/joshuayun/.local/lib/python3.6/site-packages/torch/serialization.py", line 387, in load
    return _load(f, map_location, pickle_module, **pickle_load_args)
  File "/home/joshuayun/.local/lib/python3.6/site-packages/torch/serialization.py", line 574, in _load
    result = unpickler.load()
  File "/home/joshuayun/.local/lib/python3.6/site-packages/torch/serialization.py", line 537, in persistent_load
    deserialized_objects[root_key] = restore_location(obj, location)
  File "/home/joshuayun/.local/lib/python3.6/site-packages/torch/serialization.py", line 119, in default_restore_location
    result = fn(storage, location)
  File "/home/joshuayun/.local/lib/python3.6/site-packages/torch/serialization.py", line 95, in _cuda_deserialize
    device = validate_cuda_device(location)
  File "/home/joshuayun/.local/lib/python3.6/site-packages/torch/serialization.py", line 79, in validate_cuda_device
    raise RuntimeError('Attempting to deserialize object on a CUDA '
RuntimeError: Attempting to deserialize object on a CUDA device but 
  torch.cuda.is_available() is False. If you are running on a CPU-only machine, 
  please use torch.load with map_location='cpu' to map your storages to the CPU.
1
  • Edited the question for clarity. Commented May 30, 2019 at 7:45

12 Answers 12

68

If you don't have gpu then use map_location=torch.device('cpu') with load model.load()

my_model = net.load_state_dict(torch.load('classifier.pt', map_location=torch.device('cpu')))
1
  • You still get the same warning with this code, it's likely because you dumped it with Pickle instead of Pytorch. Do torch.save instead of pickle.dump.
    – Sirupsen
    Commented Oct 7, 2022 at 15:00
42

Just giving a smaller answer. To solve this, you could change the parameters of the function named load() in the serialization.py file. This is stored in: ./site-package/torch/serialization.py

Write:

def load(f, map_location='cpu', pickle_module=pickle, **pickle_load_args):

instead of:

def load(f, map_location=None, pickle_module=pickle, **pickle_load_args):

Hope it helps.

0
25

"If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU."

model = torch.load('model/pytorch_resnet50.pth',map_location ='cpu')
1
  • This worked for me. Was trying to serialize and deserialize a PyTorch model as a pickle, which got the error in the OP's question, and led me to this answer Commented Feb 26, 2021 at 16:59
19

I have tried add "map_location='cpu'" in load function, but it doesn't work for me.

If you use a model trained by GPU on a CPU only computer, then you may meet this bug. And you can try this solution.

solution

class CPU_Unpickler(pickle.Unpickler):
    def find_class(self, module, name):
        if module == 'torch.storage' and name == '_load_from_bytes':
            return lambda b: torch.load(io.BytesIO(b), map_location='cpu')
        else: return super().find_class(module, name)

contents = CPU_Unpickler(f).load()
4
  • 1
    Thank you! So useful, I opened another (logged-in) browser profile just to come back and upvote. :-)
    – G__
    Commented Sep 10, 2021 at 18:19
  • 1
    for some reason, torch was ignoring map_location=torch.device("cpu") whenever I used it, but this worked!
    – Vivek
    Commented Dec 4, 2021 at 15:47
  • 1
    This works, the accepted answer didn't
    – Arco Bast
    Commented Mar 3, 2023 at 12:18
  • 1
    This works even for loading objects with GPU tensors saved via pickle. Thank you. Commented Aug 4, 2023 at 11:15
9

You can remap the Tensor location at load time using the map_location argument to torch.load.

On the following repository,in file "test.py", model = loadmodel() calls the model_loader.py file to load the model with torch.load().

While this will only map storages from GPU0, add the map_location:

torch.load(settings.MODEL_FILE, map_location={'cuda:0': 'cpu'})

In the model_loader.py file, add, map_location={'cuda:0': 'cpu'} whereever, torch.load() function is called.

6

You may have to re-install torch for your desired CUDA Toolkit version.

Lookup the command from Getting Started page and re-install.

# uninstall first
pip uninstall torch torchvision torchaudio

# e.g. for CUDA Toolkit 1.8
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
4

As you state the problem hints you are trying to use a cuda-model on non-cuda machine. Pay attention to the details of the error message - please use torch.load with map_location='cpu' to map your storages to the CPU. I've had similar problem when I tried to load (from a checkpoint) pre-trained model on my cpu-only machine. The model was trained on a cuda machine so it couldn't be properly loaded. Once I added the map_location='cpu' argument to the load method everything worked.

2
  • 2
    This may not be the case. I get this problem although I run the model on the same GPU machine that was trained on.
    – cerebrou
    Commented Nov 4, 2019 at 14:59
  • 1
    I'm also seeing this error on a GPU machine, torch.cuda.is_available() --> True
    – crypdick
    Commented Apr 17, 2020 at 22:07
3

There is much easier way. Just add map_location to torch.load(path, map_location='cpu') as map_location='cpu':

def load_checkpoint(path) -> 'LanguageModel':
    checkpoint = torch.load(path, map_location='cpu')
    model = LanguageModel(
        number_of_tokens=checkpoint['number_of_tokens'],
        max_sequence_length=checkpoint['max_sequence_length'],
        embedding_dimension=checkpoint['embedding_dimension'],
        number_of_layers=checkpoint['number_of_layers'],
        number_of_heads=checkpoint['number_of_heads'],
        feed_forward_dimension=checkpoint['feed_forward_dimension'],
        dropout_rate=checkpoint['dropout_rate']
    ).to(get_device())
    model.load_state_dict(checkpoint['model_state_dict'])
    return model.to(get_device())
2

I faced the same problem, Instead of modifying the existing code, which was running good yesterday, First I checked whether my GPU is free or not running

nvidia-smi

I could see that, its under utilized, therefore as traditional solution, I shutdown the laptop and restarted it and it got working.

(One thing I kept in mind that, earlier it was working and I haven't changed anything in code therefore it should work once I restart it and it got working and I was able to use the GPU)

2

nothing worked for me- my pickle was a custom object- in a script file with the line

device = torch.device("cuda")

finally, I managed to take Spikes solution, and adapt it to my needs with simple open(path,"rb"), so for any other unfortunate developers:

class CPU_Unpickler(pickle.Unpickler):
    def find_class(self, module, name):
        if module == 'torch.storage' and name == '_load_from_bytes':
            return lambda b: torch.load(io.BytesIO(b), map_location='cpu')
        else: return super().find_class(module, name)

contents = CPU_Unpickler(open(path,"rb")).load()
1
  • It still doesn't work for me even if I use this instead. do you have any other solutions? Commented Nov 15, 2022 at 3:37
1

For some reason, this also happens with portainer, even though your machines have GPUs. A crude solution would be to just restart it. It usually happens if you fiddle with the state of the container after it has been deployed (e.g. you change the restart policies while the container is running), which makes me think it's some portainer issue.

0

A solution that works for me: I was with cuda 12.1.1, pytorch==2.0.1 and torchvision-0.13.1a0. Output of torch.cuda.is_available() was FALSE

First, I removed these packages

conda remove pytorch torchvision cudatoolkit pytorch

Then

conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia

In the end, package versions were pytorch-2.0.1, torchvision-0.15.2, and of course cuda 11.7. And output of torch.cuda.is_available() is now TRUE

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.