When you call model.to(device)
(assuming device
is a GPU) your model parameters will be moved to your GPU. Regarding to your comment: they are moved from CPU memory to GPU memory then.
By default newly created tensors are created on CPU, if not specified otherwise. So this applies also for your inputs
and labels
.
The problem here is that all operands of an operation need to be on the same device! If you leave out the to
and use CPU tensors as input you will get an error message.
Here is an short example for illustration:
import torch
# device will be 'cuda' if a GPU is available
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
# creating a CPU tensor
cpu_tensor = torch.rand(10)
# moving same tensor to GPU
gpu_tensor = cpu_tensor.to(device)
print(cpu_tensor, cpu_tensor.dtype, type(cpu_tensor), cpu_tensor.type())
print(gpu_tensor, gpu_tensor.dtype, type(gpu_tensor), gpu_tensor.type())
print(cpu_tensor*gpu_tensor)
Output:
tensor([0.8571, 0.9171, 0.6626, 0.8086, 0.6440, 0.3682, 0.9920, 0.4298, 0.0172,
0.1619]) torch.float32 <class 'torch.Tensor'> torch.FloatTensor
tensor([0.8571, 0.9171, 0.6626, 0.8086, 0.6440, 0.3682, 0.9920, 0.4298, 0.0172,
0.1619], device='cuda:0') torch.float32 <class 'torch.Tensor'> torch.cuda.FloatTensor
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-15-ac794171c178> in <module>()
12 print(gpu_tensor, gpu_tensor.dtype, type(gpu_tensor), gpu_tensor.type())
13
---> 14 print(cpu_tensor*gpu_tensor)
RuntimeError: Expected object of type torch.FloatTensor but found type torch.cuda.FloatTensor for argument #2 'other'