RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same

Question

I am trying to train the following CNN as follows, but I keep getting the same error regarding .cuda() and I am not sure how to fix it. Here is a chunk of my code so far.

import matplotlib.pyplot as plt
import numpy as np
import torch
from torch import nn
from torch import optim
import torch.nn.functional as F
import torchvision
from torchvision import datasets, transforms, models
from torch.utils.data.sampler import SubsetRandomSampler


data_dir = "/home/ubuntu/ML2/ExamII/train2/"
valid_size = .2

# Normalize the test and train sets with torchvision
train_transforms = transforms.Compose([transforms.Resize(224),
                                           transforms.ToTensor(),
                                           ])

test_transforms = transforms.Compose([transforms.Resize(224),
                                          transforms.ToTensor(),
                                          ])

# ImageFolder class to load the train and test images
train_data = datasets.ImageFolder(data_dir, transform=train_transforms)
test_data = datasets.ImageFolder(data_dir, transform=test_transforms)


# Number of train images
num_train = len(train_data)
indices = list(range(num_train))
# Split = 20% of train images
split = int(np.floor(valid_size * num_train))
# Shuffle indices of train images
np.random.shuffle(indices)
# Subset indices for test and train
train_idx, test_idx = indices[split:], indices[:split]
# Samples elements randomly from a given list of indices
train_sampler = SubsetRandomSampler(train_idx)
test_sampler = SubsetRandomSampler(test_idx)
# Batch and load the images
trainloader = torch.utils.data.DataLoader(train_data, sampler=train_sampler, batch_size=1)
testloader = torch.utils.data.DataLoader(test_data, sampler=test_sampler, batch_size=1)


#print(trainloader.dataset.classes)

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = models.resnet50(pretrained=True)

model.fc = nn.Sequential(nn.Linear(2048, 512),
                                 nn.ReLU(),
                                 nn.Dropout(0.2),
                                 nn.Linear(512, 10),
                                 nn.LogSigmoid())
                                 # nn.LogSoftmax(dim=1))
# criterion = nn.NLLLoss()
criterion = nn.BCELoss()
optimizer = optim.Adam(model.fc.parameters(), lr=0.003)
model.to(device)

#Train the network
for epoch in range(2):  # loop over the dataset multiple times

    running_loss = 0.0
    for i, data in enumerate(trainloader, 0):
        # get the inputs; data is a list of [inputs, labels]
        inputs, labels = data

        # zero the parameter gradients
        optimizer.zero_grad()

        # forward + backward + optimize
        outputs = model(inputs)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()

        # print statistics
        running_loss += loss.item()
        if i % 2000 == 1999:    # print every 2000 mini-batches
            print('[%d, %5d] loss: %.3f' %
                  (epoch + 1, i + 1, running_loss / 2000))
            running_loss = 0.0

print('Finished Training')

However, I keep getting this error in the console:

RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same`

Any thoughts on how to fix it? I read that maybe the model hasn't been pushed into my GPU, but not sure how to fix it. Thanks!

Answer 1

You get this error because your model is on the GPU, but your data is on the CPU. So, you need to send your input tensors to the GPU.

inputs, labels = data                         # this is what you had
inputs, labels = inputs.cuda(), labels.cuda() # add this line

Or like this, to stay consistent with the rest of your code:

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

inputs, labels = inputs.to(device), labels.to(device)

The same error will be raised if your input tensors are on the GPU but your model weights aren't. In this case, you need to send your model weights to the GPU.

model = MyModel()

if torch.cuda.is_available():
    model.cuda()

Here is the documentation for cuda() and cpu(), its opposite.

Answer 2

The new API is to use .to() method.

The advantage is obvious and important. Your device may tomorrow be something other than "cuda":

• cpu
• cuda
• mkldnn
• opengl
• opencl
• ideep
• hip
• msnpu
• xla

So try to avoid model.cuda() It is not wrong to check for the device

dev = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")

or to hardcode it:

dev=torch.device("cuda") 

same as:

dev="cuda"

In general you can use this code:

model.to(dev)
data = data.to(dev)

Answer 3

As already mentioned in the previous answers, the issue can be that your model is trained on the GPU, but it's tested on the CPU. If that's the case then you need to port your model's weights and the data from the GPU to the CPU like this:

device = args.device # "cuda" / "cpu"
if "cuda" in device and not torch.cuda.is_available():
    device = "cpu"
data = data.to(device)
model.to(device)

NOTE: Here we still check if the configuration arguments are set to GPU or CPU, so that this piece of code can be used for both training (on the GPU) and testing (on the CPU).

Answer 4

   * when you get this error::RuntimeError: Input type 
   (torch.FloatTensor) and weight type (torch.cuda.FloatTensor should 
   be the same
   # Move tensors to GPU is CUDA is available
   # Check if CUDA is available

  train_on_gpu = torch.cuda.is_available()

  If train_on_gpu:
      print("CUDA is available! Training on GPU...")
  else:
      print("CUDA is not available. Training on CPU...")

 -------------------
 # Move tensors to GPU is CUDA is available
if train_on_gpu:

model.cuda()

Answer 5

I have same problem,My CNN model:

class CNN(nn.Module):
   def __init__(self):
      super(CNN,self).__init__()
      self.device = torch.device(device)
      self.dummy_param = nn.Parameter(torch.empty(0))
      l1 = nn.Conv2d(3, 64,    kernel_size=(3, 3), stride=(1, 1), padding= (1,1)).to(device)
      l2 = nn.Conv2d(64, 128,  kernel_size=(3, 3), stride=(1, 1), padding=(1,1)).to(device)
      l3 = nn.Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)).to(device)
      l4 = nn.Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)).to(device)
      l5 = nn.Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)).to(device)
      self.layers = [l1,l2,l3,l4,l5]
      self.layers = [l1,l2]

  def forward(self,x):
    features = []
    for l in self.layers:
  
      x = l(x)
      features.append(x)
  return features

I put for Conv2d.to(device) its work for me.

Answer 6

First check cuda is available or not:

  if torch.cuda.is_available():
      device = 'cuda'
  else:
      device = 'cpu'

In case you want to load some model do this:

  checkpoint = torch.load('./generator_release.pth', map_location=device)
  G = Generator().to(device)

Now you probably get this error:

RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same

It is needed to convert the type of input data from torch.tensor to torch.cuda.tensor by :

if torch.cuda.is_available():
  data = data.cuda()
result = G(data)

and then convert the result from torch.cuda.tensor to torch.tensor:

if torch.cuda.is_available():
    result = result.cpu()