0

I am using the LSTM structure:

layers = [ ...
    sequenceInputLayer(1)
    bilstmLayer(100,'OutputMode','last')
    fullyConnectedLayer(2)
    softmaxLayer
    classificationLayer
    ];

options = trainingOptions('adam', ...
    'MaxEpochs',30, ...
    'MiniBatchSize', 150, ...
    'InitialLearnRate', 0.01, ...
    'GradientThreshold', 1, ...
    'plots','training-progress', ...
    'Verbose',false);

and net = trainNetwork(XTrain,Ytrain,layers,options);

where

  • Xtrain is 1x100 cell array (Xtrain{1,1} gives a data array of size 1000x1, Xtrain{1,2} is another set of data array of size 1000x1 etc). This means that I have 100 examples of feature vectors each example is of dimension 1000.

  • Ytrain is the response variable 0/1 and is an array of size 100x1. The response is of type double.

I simply assumed the MiniBatchSize parameter as 150. I tried with other values as well say 50,60,70...nothing seems to influence the performance. So I don't exactly follow what this parameter denotes and how to find a value for it. Can somebody please help in explaining what this means and ideally what it should be? thank you

1 Answer 1

1

The parameter is actually explained on the Mathworks documentation page:

Size of the mini-batch to use for each training iteration, specified as the comma-separated pair consisting of MiniBatchSize and a positive integer. A mini-batch is a subset of the training set that is used to evaluate the gradient of the loss function and update the weights. See Stochastic Gradient Descent.

As for what it is, there exist several answers on the web, both on Stackexchange websites here and here and elsewhere on the web. Generally, it should influence the convergence of your optimization algorithm, and how much memory is used during the computation.
Note that it is generally preferred by now to use smaller batch sizes, as long as you don't have too much training data.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.