2

Hello I have a cell array of char (separated by underscore) that I would like to convert to double. I do it in a for loop, but since the dimensions are very big, it takes a lot of time. I would like to use cellfun, but I don't know how to pass the delimiter.

Can you help me?

listofwords = {'02_04_04_52';'02_24_34_02'};
for i = 1 : size(listofwords,1)
    listofwords_double(i,:) = str2double(strsplit(listofwords{i},'_'))./1000;
end

listofwords_double2= cellfun(@strsplit , listofwords);

Benchmark

As requested by Divakar

>> benchmark1
Speedup with EVAL over NO-LOOP-SSCANF = -46.3398%
>> benchmark1
Speedup with EVAL over NO-LOOP-SSCANF = -46.4068%
>> benchmark1
Speedup with EVAL over NO-LOOP-SSCANF = -47.1129%
>> benchmark1
Speedup with EVAL over NO-LOOP-SSCANF = -46.2882%
>> benchmark1
Speedup with EVAL over NO-LOOP-SSCANF = -46.2325%
>> benchmark1
Speedup with EVAL over NO-LOOP-SSCANF = -46.0161%
>> benchmark1
Speedup with EVAL over NO-LOOP-SSCANF = -46.9728%
>> benchmark1
Speedup with EVAL over NO-LOOP-SSCANF = -46.4267%
>> benchmark1
Speedup with EVAL over NO-LOOP-SSCANF = -46.2867%
>> benchmark1
Speedup with EVAL over NO-LOOP-SSCANF = -46.3031%
3
  • Wow, that goes so bad for my evil script! :-) However, thanks for helping us settle the matter, I - and I'm sure @Divakar too - really appreciate you taking the time.
    – user2271770
    Oct 22, 2014 at 17:21
  • @CST-Link Time to edit your conclusions in your solution I believe :)
    – Divakar
    Oct 22, 2014 at 17:22
  • @Divakar Haha, it might be, but not yet. :-) Soon, though.
    – user2271770
    Oct 22, 2014 at 17:30

2 Answers 2

5

You can use anonymous function like this -

listofwords_double2= cellfun(@(x) strsplit(x,'_') , listofwords,'uni',0)

Another approach with regexp and a one-liner -

cell2mat(cellfun(@(x) str2double(regexp(x,'_','Split'))./1000 , listofwords,'uni',0))

Performance oriented solutions

Approach #1

N = 4; %// Edit this to 10 in your actual case
cat_cell = strcat(listofwords,'_');
one_str = [cat_cell{:}];
one_str(end)=[];
sep_cells = regexp(one_str,'_','Split');
out = reshape(str2double(sep_cells),N,[]).'./1000; %//'# desired output

Approach #2

Benchmarking the above solution suggests strcat could prove to be the bottleneck. To get rid of that you can use a cumsum based approach for that part. This is listed next -

N = 4; %// Edit this to 10 in your actual case

lens = cellfun(@numel,listofwords);
tlens = sum(lens);
idx = zeros(1,tlens); %// Edit this to "idx(1,tlens)=0;" for more performance
idx(cumsum(lens(1:end-1))+1)=1;
idx2 = (1:tlens) + cumsum(idx);

one_str(1:max(idx2))='_';
one_str(idx2) = [listofwords{:}];

sep_cells = regexp(one_str,'_','Split');
out = reshape(str2double(sep_cells),N,[]).'./1000; %//'# desired output

Approach #3

Now, this one uses sscanf and appears to be really fast. Here's the code -

N = 4; %// Edit this to 10 in your actual case
lens = cellfun(@numel,listofwords);
tlens = sum(lens);
idx(1,tlens)=0;
idx(cumsum(lens(1:end-1))+1)=1;
idx2 = (1:tlens) + cumsum(idx);

one_str(1:max(idx2)+1)='_';
one_str(idx2) = [listofwords{:}];
delim = repmat('%d_',1,N*numel(lens));
out = reshape(sscanf(one_str, delim),N,[])'./1000; %//'# desired output

Benchmarking

As requested by @CST-Link, here's the benchmark comparing his "Kraken" eval against approach #3. The benchmarking code would look something like this -

clear all

listofwords = repmat({'02_04_04_52_23_14_54_672_0'},100000,1);
for k = 1:50000
    tic(); elapsed = toc(); %// Warm up tic/toc
end

tic
N = 9; %// Edit this to 10 in your actual case
lens = cellfun(@numel,listofwords);
tlens = sum(lens);
idx(1,tlens)=0;
idx(cumsum(lens(1:end-1))+1)=1;
idx2 = (1:tlens) + cumsum(idx);

one_str(1:max(idx2)+1)='_';
one_str(idx2) = [listofwords{:}];
delim = repmat('%d_',1,N*numel(lens));
out = reshape(sscanf(one_str, delim),N,[])'./1000; %//'# desired output
time1 = toc;
clear out delim one_str idx2 idx tlens lens N

tic
n_numbers = 1+sum(listofwords{1}=='_');
n_words   = numel(listofwords);
listofwords_double = zeros(n_numbers, n_words);
for i = 1:numel(listofwords)
        temp = ['[', listofwords{i}, ']'];
        temp(temp=='_') = ';';
        listofwords_double(:,i) = eval(temp);
end;
listofwords_double = (listofwords_double / 1000).';
time2 = toc;
speedup = ((time1-time2)/time2)*100;
disp(['Speedup with EVAL over NO-LOOP-SSCANF = ' num2str(speedup) '%'])

And here are the benchmark results when the code is run for a few number of times -

>> benchmark1
Speedup with EVAL over NO-LOOP-SSCANF = 0.30609%
>> benchmark1
Speedup with EVAL over NO-LOOP-SSCANF = 0.012241%
>> benchmark1
Speedup with EVAL over NO-LOOP-SSCANF = -2.3146%
>> benchmark1
Speedup with EVAL over NO-LOOP-SSCANF = 0.33678%
>> benchmark1
Speedup with EVAL over NO-LOOP-SSCANF = -1.8189%
>> benchmark1
Speedup with EVAL over NO-LOOP-SSCANF = -0.12254%

Looking at the results and observing some negative speedups (indicating sscanf to be better in those cases) among some positive speedups, my opinion would be to stick with sscanf.

24
  • @Divakar First of all, thanks for admitting (implicitly) that cellfun is expressive but not efficient - which was my initial point. So I'll run the test tomorrow; I'm also curious about the outcome. Until soon. :-)
    – user2271770
    Oct 22, 2014 at 0:08
  • @Divakar Hi. 1) I run your benchmark test on MATLAB R2012a (Win, 2Gb RAM, Intel i5 vPro) and my script presented an approx. 12% speed-up. Moreover, if I switch the order of execution - my script first, then yours - I get 17%. 2) The statement about sscanf being faster than eval is not accurate. You're comparing a single call of sscanf with 100000 calls of eval so you don't account for the 99999 overheads in function call. 3) My challenge was for a million strings, not one hundred thousands: your approach gives gives me "out of memory" errors, because cumsum.
    – user2271770
    Oct 22, 2014 at 8:25
  • @Divakar So, I'd like your permission to add these conclusions to the answer. May I? :-)
    – user2271770
    Oct 22, 2014 at 8:26
  • @CST-Link On 1) I skipped clearing out memory at the end of your code, as that was the end of the benchmark script. So, if you are switching the order of the codes, please clear out all the variables used, just like I did after toc of no-loop sscanf method. Also, could you increase the number of cells (which was one million in your test case) to the max until it throws out-of-memory error? On 2) Well I have gone with no-loop sscanf, otherwise my approach would be identical to your sscanf based approach and there won't be any approach #3 or benchmark.
    – Divakar
    Oct 22, 2014 at 9:02
  • @CST-Link On 3) I had out-of-memory error with one million strings, that's why I settled for 100 thousand. Now, I have 4GB RAM. You said you have 2GB RAM (which is sort of rare these days I would think), so maybe you can go with 10 thousand cells? With how many cells/strings did you get the "out of memory" error at cumsum? Yeah sure, please go ahead and add whatever conclusions or results you have in your solution, would be interesting to have a look!
    – Divakar
    Oct 22, 2014 at 9:02
2

A solution could be:

listofwords_double2 = cellfun(@(x) str2double(strsplit(x, '_'))./ 1000, listofwords);

Just a side note: my version of Matlab does not have strsplit, so I cannot test it.

8
  • 1
    You need to use UniformOutput with false I think.
    – Divakar
    Oct 21, 2014 at 13:05
  • how can I get a matrix of doubles?
    – gabboshow
    Oct 21, 2014 at 13:07
  • @gabboshow use cell2mat wrapping listofwords_double2.
    – Divakar
    Oct 21, 2014 at 13:08
  • can I use cell2mat inside cellfun...all in 1 line? :)
    – gabboshow
    Oct 21, 2014 at 13:09
  • @gabboshow Let me ask you - Do you always have 3 underscores and 4 numbers in each cell?
    – Divakar
    Oct 21, 2014 at 13:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.