2

I have an array with some values that belongs to a set. I would like to transform this array in a binary matrix, each column of this matrix will represent each possible value of the set, the row value is 1 for the column that matches the input array or 0 for all the others. I think a name for that is something like a binary pivot.

The input array is a column of a table type

Example of input array (The previous example were only capital letters, which led to misinterpretation):

'Apple'
'Banana'
'Cherry'
'Dragonfruit'
'Apple'
'Cherry'

So, in this example input could assume 4 different values: 'Apple', 'Banana', 'Cherry' or 'Dragonfruit', in my real scenario it can be more than 4.

Example Output matrix:

1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
1 0 0 0
0 0 1 0

I have achieved this desired behavior, but I would like to know if there is a better way to perform this operation. In a vectorized way (without the for-loop for each category) or using a built-in function.

 function [ binMatrix, categs ] = pivotToBinaryMatrix( input )
      categorizedInput = categorical(input);

      categs = categories(categorizedInput);

      binMatrix = zeros(size(atributo, 1), size(categorias, 1));

      for i = 1: size(caters,1)
           binMatrix(:,i) = ismember(categorizedInput, categs(i));
      end
 end

For about 50.000 entries with 9 categories it performed in 0.075137 seconds.

EDIT: I've improved the examples, because the previous examples led to misinterpretation.

2
  • So did we help? Consider accepting one of our answers if we did.
    – rayryeng
    Apr 29, 2015 at 18:21
  • I've edited my post to reply to your edits. It only required a very small edit, but our methods are still faster than your's. To use any of our methods, all you have to do is make one call to unique and use the third output into any of our algorithms.
    – rayryeng
    May 6, 2015 at 11:53

3 Answers 3

5

Here's my take on the problem:

input = ['ABCDAB']';
binMatrix = bsxfun(@eq,input,unique(input)');

For the benchmarking, I ran it on a Windows 7 machine, 4Gb RAM, Intel i7-2600 CPU 3.4 GHz, borrowing @rayryeng initialization code:

% Generate dictionary from A up to I
ch = char(65 + (0:8));

rng(123);

% Generate 50000 random characters
v = randi(9, 50000, 1);
inputArray = ch(v);

time=0;
for ii=1:100
    tic;
    binMatrix = bsxfun(@eq,inputArray,unique(inputArray)');
    t = toc;
    time=time+t;
end
disp(time/100);

Which gave me 0.001203 seconds. For an extensive comparison of methods, please refer to @ryaryeng's answer.

5
  • 1
    Reading your answers way too much I guess. Had to learn it sometime ;)
    – brodoll
    Apr 28, 2015 at 4:00
  • Did some runtime tests and seems like this is quite efficient and I could think of this reason, well done on using it!!
    – Divakar
    Apr 28, 2015 at 5:17
  • Cool! Did you try using it on a 50000 length vector like the OP did? This only is done on a six element vector.
    – rayryeng
    Apr 28, 2015 at 6:06
  • @Divakar - Did a timing test. bsxfun is slower. I'll update my post. Raw sub2ind wins!
    – rayryeng
    Apr 28, 2015 at 7:30
  • 1
    Good catch @rayryeng the timing is done for the six element array only. I'll edit asap so it is not misleading!
    – brodoll
    Apr 28, 2015 at 9:31
3

I'm going to assume that your input array is a cell array of characters like so:

inputArray = {'Apple', 'Banana', 'Cherry', 'Dragonfruit', 'Apple', 'Cherry'};

You can convert the above into a numeric array by using the unique function's third output. What's great about this is that unique assigns a unique ID in sorted order, and so if you have a cell array of characters, it respects a lexicographical ordering of the characters.

Next, declare a matrix of zeros (like you did above) then use sub2ind to index into the matrix and set the values to 1.

Something like this. Bear in mind that I initialized the output slightly differently. It's a trick I learned to allocate a matrix of zeroes that is quite fast. See here: Faster way to initialize arrays via empty matrix multiplication? (Matlab)

inputArray = {'Apple', 'Banana', 'Cherry', 'Dragonfruit', 'Apple', 'Cherry'};
[~,~,inputNum] = unique(inputArray);
inputNum = inputNum.'; %// To make compatible in dimensions
binMatrix(numel(inputArray), max(inputNum)) = 0;
binMatrix(sub2ind(size(binMatrix), 1:numel(inputArray), inputNum)) = 1;

Another method would be to create a sparse logical array where we set the right row and column positions to be 1, then use this to index into our zeroes array and set the values accordingly.

Something like:

inputArray = {'Apple', 'Banana', 'Cherry', 'Dragonfruit', 'Apple', 'Cherry'};
[~,~,inputNum] = unique(inputArray);
inputNum = inputNum.'; %// To make compatible in dimensions
binMatrix = sparse(1:numel(inputArray), inputNum, 1, numel(inputArray), max(inputNum));
binMatrix = full(binMatrix);

Let's put this all together in a timing script. I've incorporated the two methods above, plus your old method, plus Divakar's (only the first method) and brodroll's (very ingenious btw) method. For Divakar's and brodroll's method, I have also used unique with the third output as your original inquiry had capital letters which confused as all. Using the third output can easily convert their previous methods to your new specifications.

BTW, your example and your code are mismatched. Your example has it set so the each column is an index but it's each row. For the timing tests, I'm going to transpose your result.I'm running MATLAB R2013a on Mac OS X 10.10.3 with 16 GB of RAM and an Intel i7 2.3 GHz processor. So:

clear all;
close all;

%// Generate dictionary
chars = {'Apple', 'Banana', 'Cherry', 'Dragonfruit'};

rng(123);

%// Generate 50000 random words
v = randi(numel(chars), 50000, 1);
inputArray = chars(v);
[~,~,inputNum] = unique(inputArray);
inputNum = inputNum.'; %// To make compatible in dimensions

%// Timing #1 - sub2ind
tic;
binMatrix(numel(inputArray), max(inputNum)) = 0;
binMatrix(sub2ind(size(binMatrix), 1:numel(inputArray), inputNum)) = 1;
t = toc;

clear binMatrix;

%// Timing #2 - sparse
tic;
binMatrix = sparse(1:numel(inputArray), inputNum, 1, numel(inputArray), max(inputNum));
binMatrix = full(binMatrix);
t2 = toc;

clear binMatrix;

%// Timing #3 - ismember and for
tic;
binMatrix = zeros(numel(inputArray), numel(chars));
for i = 1: size(binMatrix,1)
binMatrix(i,:) = ismember(chars, inputArray(i));
end
t3 = toc;

%// Timing #4 - bsxfun
clear binMatrix;
tic;
binMatrix = bsxfun(@eq,inputNum',unique(inputNum)); %// Changed to make dimensions match
t4 = toc;

clear binMatrix;

%// Timing #5 - raw sub2ind
tic;
binMatrix(numel(inputArray), max(inputNum)) = 0;
binMatrix( (inputNum-1)*size(binMatrix,1) + [1:numel(inputArray)] ) = 1;
t5 = toc;

fprintf('Timing using sub2ind: %f seconds\n', t);
fprintf('Timing using sparse: %f seconds\n', t2);
fprintf('Timing using ismember and loop: %f seconds\n', t3);
fprintf('Timing using bsxfun: %f seconds\n', t4);
fprintf('Timing using raw sub2ind: %f seconds\n', t5);

We get:

Timing using sub2ind: 0.004223 seconds
Timing using sparse: 0.004252 seconds
Timing using ismember and loop: 2.771389 seconds
Timing using bsxfun: 0.020739 seconds
Timing using raw sub2ind: 0.000773 seconds

In terms of rank:

  1. Raw sub2ind
  2. sub2ind
  3. sparse
  4. bsxfun
  5. OP's method
8
  • @Divakar - OH I KNOW. I use that pre-allocation hack all the time. I don't even use zeros anymore. If you're curious, I added in timing tests and I also added in using the sparse method. Looks like sparse is slightly faster.
    – rayryeng
    Apr 28, 2015 at 3:58
  • Huh! Interesting, could be useful knowledge for the future!
    – Divakar
    Apr 28, 2015 at 3:59
  • 1
    @rayryeng Thanks for the tests and specially for that hack. Gotta put that into action!
    – brodoll
    Apr 28, 2015 at 3:59
  • 1
    Used some more hacky action to get rid of sub2ind and stayed with that pre-allocation hack! Hope, that's okay! :)
    – Divakar
    Apr 28, 2015 at 4:12
  • 1
    @rayryeng With a bigger datasize of 20000000 for v, I got these values - pastebin.com/fxx16JaY. I skipped the ismember, as it was taking too long to compute.
    – Divakar
    Apr 28, 2015 at 8:31
3

If you don't mind all zeros columns in cases where you have non-successive characters in the input array, something like 'ABEACF', where 'D' is missing, you can use this -

col_idx = inputArray - 'A' + 1;
binMatrix(numel(inputArray), max(col_idx) ) = 0;
binMatrix( (col_idx-1)*size(binMatrix,1) + [1:numel(inputArray)] ) = 1;

If you do care about that issue and would like no all-zeros columns, you can use a modified version of it -

[~,unq_pos,col_idx] = unique(inputArray,'stable');
binMatrix(numel(inputArray), numel(unq_pos)) = 0;
binMatrix( (col_idx-1)*size(binMatrix,1) + [1:numel(inputArray)].' ) = 1;

Basically both these approaches use the same hacky technique to pre-allocate as listed in Undocumented MATLAB and also listed in the other answer by @rayryeng. On top of it, it uses a raw version of sub2ind.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.