4

I'm writing a version of Python's doctest test-runner, for MATLAB (it partly works...). For this to work, I need to run the code in people's examples in their m-file help. I want variables to carry over from one line to the next, e.g.

% >> I = 5 + 33; % expect no output
% >> I
% 
% I =
% 
%     38
%

To run the tests, I have a loop over matches to the REGEX that searches for the tests. For each match, I evalc the example and make sure the result matches:

for I = 1:length(examples)
    try
        got = evalc(examples(I).source);
    catch exc
        got = ['??? ' exc.message];
    end

    % process the result...
end

The problem is that the example's definition of I has now clobbered the loop variable in my loop, since the assignments carry over from the eval into the outer scope. I looked around for something capable of creating a new scope/workspace, but evalin can only re-use the caller's workspace, which is even worse. I've also considered options with calling a sub-function or save/load, but not gotten anywhere, but maybe I just haven't thought hard enough.

So I guess I need to just name all my variables doctest__system__* and live with the namespace problems... unless you have another idea for a strategy to avoid variable name conflicts?

1 Answer 1

4

A very interesting project for sure.. I think the best option you have is to write a separate function to execute the tests, and use a unique prefix for all variable inside of this function to avoid name conflict. Here my attempt at this:

function [PREFIX_b varargout] = testContext(PREFIX_src, PREFIX_srcOutput)
    %# TESTCONTEXT   Executes the source code and tests for
    %#               equality against the expected output
    %#
    %#   Input:
    %#       PREFIX_src       - source to execute, cellarry of statements
    %#       PREFIX_srcOutput - output to expect, cellarray of output of each statement
    %#
    %#   Output:
    %#       PREFIX_b         - true/false for success/failure of test
    %#                          note that the output is strtrim()'ed then strcmp()'ed
    %#       varargout{1}     - variable names assigned in this confined context
    %#       varargout{2}     - variable values assigned
    %#
    %#   Example 1:
    %#       source = { 'I = 5+33;' 'I' };
    %#       output = { [], ['I =' char(10) '    38'] };
    %#       b = testContext(source, output);
    %#
    %#   Example 2:
    %#       source = { 'I = 5+33; J = 2;' 'K = 1;' 'disp(I+J+K)' };
    %#       output = { [], [], '41' };
    %#       [b varNames varValues] = testContext(source, output);
    %#
    %#   See also: eval evalc
    %#

    PREFIX_b = true;

    try
        %# for each statement
        for PREFIX_i=1:numel(PREFIX_src)
            %# evaluate
            PREFIX_output = evalc( PREFIX_src{PREFIX_i} );
            PREFIX_output = strtrim(PREFIX_output);            %# trim whitespaces
            %# compare output
            if ~isempty( PREFIX_srcOutput{PREFIX_i} )
                if ~strcmp(PREFIX_output,PREFIX_srcOutput{PREFIX_i})
                    PREFIX_b = false;
                    return
                end
            end
        end

        if nargout > 1
            %# list created variables in this context
            %#clear ans
            PREFIX_vars = whos('-regexp', '^(?!PREFIX_).*');   %# java regex negative lookahead
            varargout{1} = { PREFIX_vars.name };

            if nargout > 2
                %# return those variables
                varargout{2} = cell(1,numel(PREFIX_vars));
                for PREFIX_i=1:numel(PREFIX_vars)
                    [~,varargout{2}{PREFIX_i}] = evalc( PREFIX_vars(PREFIX_i).name );
                end
            end
        end

    catch ME
        warning(ME.identifier, ME.message)
        PREFIX_b = false;
        varargout{1} = {};
        varargout{2} = {};
    end
end

I assume you are able to parse the m-file to recover the examples to test, where you have each statement along with its expected output.

As an example, consider this simple test embedded in the header of a function:

I = 5 + 33;
J = 2*I;
disp(I+J)

Since only the last statement has an output, we test it as:

source = {'I = 5 + 33;' 'J = 2*I;' 'disp(I+J)'};
output = {[], [], '114'};
[b varNames varValues] = testContext(source, output)

the results:

b =
     1
varNames = 
    'I'    'J'
varValues = 
    [38]    [76]

It shows whether the test passed of failed. Optionally, the function returns a list of variables created in that context along with their values.

1
  • Mmm, I just finished a slightly different version, but I like yours better. Mine is based on doing a save/load between every line of code. Yours doesn't mess with all that, although it's more fiddly because there are more variables that have to have the prefix. Would you mind if I integrate (a version of) your function into the project? It's a BSD-licensed project.
    – rescdsk
    Jul 20, 2010 at 19:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.