171

This:

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)

for data in dataloader:
    inputs, labels = data
    outputs = model(inputs)

Gives the error:

RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same

0

8 Answers 8

283

You get this error because your model is on the GPU, but your data is on the CPU. So, you need to send your input tensors to the GPU.

inputs, labels = data                         # this is what you had
inputs, labels = inputs.cuda(), labels.cuda() # add this line

Or like this, to stay consistent with the rest of your code:

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

inputs, labels = inputs.to(device), labels.to(device)

The same error will be raised if your input tensors are on the GPU but your model weights aren't. In this case, you need to send your model weights to the GPU.

model = MyModel()

if torch.cuda.is_available():
    model.cuda()

See the documentation for cuda(), and its opposite, cpu().

0
33

The new API is to use .to() method.

The advantage is obvious and important. Your device may tomorrow be something other than "cuda":

  • cpu
  • cuda
  • mkldnn
  • opengl
  • opencl
  • ideep
  • hip
  • msnpu
  • xla

So try to avoid model.cuda() It is not wrong to check for the device

dev = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")

or to hardcode it:

dev=torch.device("cuda") 

same as:

dev="cuda"

In general you can use this code:

model.to(dev)
data = data.to(dev)
1
  • 2
    I get an error RuntimeError: Input, output and indices must be on the current device even after moving the model and data to the gpu using to(device).
    – Evan Zamir
    May 22, 2021 at 14:37
4

As already mentioned in the previous answers, the issue can be that your model is trained on the GPU, but it's tested on the CPU. If that's the case then you need to port your model's weights and the data from the GPU to the CPU like this:

device = args.device # "cuda" / "cpu"
if "cuda" in device and not torch.cuda.is_available():
    device = "cpu"
data = data.to(device)
model.to(device)

NOTE: Here we still check if the configuration arguments are set to GPU or CPU, so that this piece of code can be used for both training (on the GPU) and testing (on the CPU).

4

Notice that (from pytorch documentation):

If the self Tensor already has the correct torch.dtype and torch.device, then self is returned. Otherwise, the returned tensor is a copy of self with the desired torch.dtype and torch.device.

That is, you might need to do:

model = model.to("cuda")
data = data.to("cuda")

Instead of just:

model.to("cuda")
data.to("cuda")

With the first approach you'll be in the safe side.

1
  • This is spot on and was exactly the issue I'm having. For reference, following link is shows the documentation stating the quoted paragraph from the answer author - pytorch.org/docs/stable/generated/…
    – qwerty
    Sep 25 at 23:15
2

When loading a model, both weights and inputs have to be in the same device, we can do this by using the .to(device) as pointed by others.

However it might be the case that also the datatype of the saved weights and the input tensors are different. If this is the case then we must also change the datatype of both model weights and inputs:

model = torch.load(PATH).type(torch.FloatTensor).to(device)
input = input.type(torch.FloatTensor).to(device)
1
   * when you get this error::RuntimeError: Input type 
   (torch.FloatTensor) and weight type (torch.cuda.FloatTensor should 
   be the same
   # Move tensors to GPU is CUDA is available
   # Check if CUDA is available

  train_on_gpu = torch.cuda.is_available()

  If train_on_gpu:
      print("CUDA is available! Training on GPU...")
  else:
      print("CUDA is not available. Training on CPU...")

 -------------------
 # Move tensors to GPU is CUDA is available
if train_on_gpu:

model.cuda()
0
x = x.to(device, dtype=torch.float32)

y = y.to(device, dtype=torch.float32)

Works, perfectly fine...

2
  • As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.
    – Community Bot
    Oct 3, 2022 at 17:29
  • In my case this works fine too :)
    – Andrey
    Oct 6 at 10:45
-1

First check cuda is available or not:

  if torch.cuda.is_available():
      device = 'cuda'
  else:
      device = 'cpu'

In case you want to load some model do this:

  checkpoint = torch.load('./generator_release.pth', map_location=device)
  G = Generator().to(device)

Now you probably get this error:

RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same

It is needed to convert the type of input data from torch.tensor to torch.cuda.tensor by :

if torch.cuda.is_available():
  data = data.cuda()
result = G(data)

and then convert the result from torch.cuda.tensor to torch.tensor:

if torch.cuda.is_available():
    result = result.cpu()

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.