22

reference fast.ai

github repository of fast.ai (as the code elevates the library which is built on top of PyTorch)

Please scroll the discussion a bit

I am running the following code, and get an error while trying to pass the data to the predict_array function

The code is failing when i am trying to use it to predict directly on a single image but it run's perfectly when that same image is in a test folder

from fastai.conv_learner import *
from planet import f2

PATH = 'data/shopstyle/'

metrics=[f2]
f_model = resnet34

def get_data(sz):
    tfms = tfms_from_model(f_model, sz, aug_tfms=transforms_side_on, max_zoom=1.05)
    return ImageClassifierData.from_csv(PATH, 'train', label_csv, tfms=tfms, suffix='.jpg', val_idxs=val_idxs, test_name='test')

def print_list(list_or_iterator):
        return "[" + ", ".join( str(x) for x in list_or_iterator) + "]"

label_csv = f'{PATH}prod_train.csv'
n = len(list(open(label_csv)))-1
val_idxs = get_cv_idxs(n)

sz = 64
data = get_data(sz)

print("Loading model...")
learn = ConvLearner.pretrained(f_model, data, metrics=metrics)
learn.load(f'{sz}')
#learn.load("tmp")

print("Predicting...")
learn.precompute=False
trn_tfms, val_tfrms = tfms_from_model(f_model, sz)
#im = val_tfrms(open_image(f'{PATH}valid/4500132.jpg'))
im = val_tfrms(np.array(PIL.Image.open(f'{PATH}valid/4500132.jpg')))
preds = learn.predict_array(im[None])
p=list(zip(data.classes, preds))
print("predictions = " + print_list(p))

Here's the Traceback I am Getting

  Traceback (most recent call last):
  File "predict.py", line 34, in <module>
    preds = learn.predict_array(im[None])
  File "/home/ubuntu/fastai/courses/dl1/fastai/learner.py", line 266, in predict_array
    def predict_array(self, arr): return to_np(self.model(V(T(arr).cuda())))
  File "/home/ubuntu/src/anaconda3/envs/fastai/lib/python3.6/site-packages/torch/nn/modules/module.py", line 325, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/ubuntu/src/anaconda3/envs/fastai/lib/python3.6/site-packages/torch/nn/modules/container.py", line 67, in forward
    input = module(input)
  File "/home/ubuntu/src/anaconda3/envs/fastai/lib/python3.6/site-packages/torch/nn/modules/module.py", line 325, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/ubuntu/src/anaconda3/envs/fastai/lib/python3.6/site-packages/torch/nn/modules/batchnorm.py", line 37, in forward
    self.training, self.momentum, self.eps)
  File "/home/ubuntu/src/anaconda3/envs/fastai/lib/python3.6/site-packages/torch/nn/functional.py", line 1011, in batch_norm
    raise ValueError('Expected more than 1 value per channel when training, got input size {}'.format(size))
ValueError: Expected more than 1 value per channel when training, got input size [1, 1024]

Things I have Tried

  • np.expand_dims(IMG,axis=0) or image = image[..., np.newaxis]

  • Tried a different way of reading the image

    img = cv2.imread(img_path)
    img = cv2.resize(img, dsize = (200,200))
    img = np.einsum('ijk->kij', img)
    img = np.expand_dims(img, axis =0) 
    img = torch.from_numpy(img) 
    learn.model(Variable(img.float()).cuda())
    

BTW the error still remains

ValueError: Expected more than 1 value per channel when training, got input size [1, 1024]

Can't find any reference in The Google search also..

0

3 Answers 3

51

It will fail on batches of size 1 if we use feature-wise batch normalization.

As Batch normalization computes:

y = (x - mean(x)) / (std(x) + eps)

If we have one sample per batch then mean(x) = x, and the output will be entirely zero (ignoring the bias). We can't use that for learning...

2
  • 1
    I found this helpful. Thanks!
    – Hello Lili
    May 15, 2018 at 15:02
  • Glad it was useful!
    – Aditya
    Mar 23, 2020 at 16:05
29

To use your trained model, call model.eval() to disable further training. This stops BatchNorm layers from updating their mean and variance, and allows input of just one sample. Use model.train() to resume training mode, if needed.

3
  • 2
    Your answer is really helpful to me. Thank you!
    – ngovanmao
    Jul 7, 2021 at 12:11
  • This helps with GPU training with torch==2.0 but gives the same error with CPU training.
    – Hans K
    May 9 at 5:35
  • This has hence been fixed, apologies.
    – Hans K
    May 14 at 7:01
0

I faced the same issue today. My batch size was five. It seems like my data set was the last batch was only 1 so it cause an error in the batch normalization layer. Changing the batch size to a value that make sure the last batch is not equal to 1 solved my problem. in my case I changed from 5 to 6

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.