A question asked today gave a surprising result with respect to implicit array creation:
array1 = 5*rand(496736,1);
array2 = 25*rand(9286,1);
output = zeros(numel(array1), numel(array2)); % Requires 34GB RAM
output = zeros(numel(array1), numel(array2),'logical'); % Requires 4.3GB RAM
output = abs(bsxfun(@minus, array1.', array2)) <= 2; % Requires 34GB RAM
output = pdist2(array1(:), array2(:)) <= 2; % Requires 34GB RAM
So far so good. An array containing 496736*9286 double values should be 34GB, and a logical array containing the same amount of elements requires just 4.3GB (8 times smaller). This happens for the latter two because they use an intermediate matrix containing all the distance pairs in double precision, requiring the full 34GB, whereas a logical matrix is directly pre-allocated as just logicals, and needs 4.3GB.
The surprising part comes here:
output = abs(array1.' - array2); % Requires 34GB RAM
output = abs(array1.' - array2) <= 2; % Requires 4.3GB RAM ?!?
What?!? Why doesn't implicit expansion require the same 34GB RAM due to creation of the intermediate double matrix output = abs(array1.' - array2)
?
This is especially strange since implicit expansion is, as I understood it, a short way of writing the old bsxfun
solutions. Thus why does bsxfun
create the full, 34GB, matrix, whereas implicit expansion does not?
Does MATLAB somehow recognise the output of the operation should be a logical matrix?
All tests performed on MATLAB R2018b, Ubuntu 18.04, 16GB RAM (i.e. 34GB arrays error out)
bsxfun
doc: Compared to usingbsxfun
, implicit expansion offers faster speed of execution, better memory usage, and improved readability of code. Guess, MathWorks put the magic right there, and of course, there's no documentation on that. Maybe predicting the proper output type from parsing arguments from outer to inner is a thing now!? Hopefully, some non-MathWorks people can provide some more information.d=a+b+c
not being computed astmp=a+b; d=tmp+c
, but rather asd(i)=a(i)+b(i)+c(i)
? Anyone with the ability to test this (max memory usage for example)?