54

When writing more than a trivial script in bash, I often wonder how to make the code testable.

It is typically hard to write tests for bash code, due to the fact that it is low on functions that take a value and return a value, and high on functions that check and set some aspect in the environment, modify the file-system, invoke a program, etc. - functions that depend on the environment or have side effects. Thus the setup and test code become much more complicated than the code they test.


For example, consider a simple function to test:

function add_to_file() {
  local f=$1
  cat >> $f
  sort -u $f -o $f
}

Test code for this function might consist of:

add_to_file.before:

foo
bar
baz

add_to_file.after:

bar
baz
foo
qux

And test code:

function test_add_to_file() {
   cp add_to_file.{before,tmp}
   add_to_file add_to_file.tmp
   cmp add_to_file.{tmp,after} && echo pass || echo fail
   rm add_to_file.tmp
}

Here 5 lines of code are tested by 6 lines of test code and 7 lines of data.


Now consider a slightly more complicated case:

function distribute() {
   local file=$1 ; shift
   local hosts=( "$@" )
   for host in "${hosts[@]}" ; do
     rsync -ae ssh $file $host:$file
   done
}

I can't even say how to start write a test for that...


So, is there a good way to do TDD in bash scripts, or should I give up and put my efforts elsewhere?

4

8 Answers 8

42

So here is what I learned:

  1. There are some testing frameworks written in bash and for bash, however...

  2. It is not so much that Bash is not suitable for TDD (although some other languages come to mind that are a better fit), but the typical tasks that Bash is used for (Installation, System configuration), that are hard to write tests for, and in particularly hard to setup the test.

  3. The poor data structure support in Bash makes it hard to separate logic from side-effect, and indeed there is typically little logic in Bash scripts. That makes it hard to break scripts into testable chunks. There are some functions that can be tested, but that is the exception, not the rule.

  4. Function are a good thing (tm), but they can only go so far.

  5. Nested functions can be even better, but they are also limited.

  6. At the end of the day, with major effort some coverage can be obtained, but it will test the less interesting part of the code, and will keep the bulk of the testing as a good (or bad) old manual testing.

Meta: I decided to answer (and accept) my own question, because I was unable to choose between Sinan Ünür's (voted up) and mouviciel's (voted up) answers that where equally useful and insightful. I want to note Stefano Borini's answer, that although not impressed me initially, I learned to appreciate it over time. Also his design patterns or best practices for shell scripts answer (voted up) referred above was useful.

0
12

From an implementation point of view, I suggest shUnit2 or bats.

From a practical point of view, I suggest not to give up. I use TDD on bash scripts and I confirm that it is worth the effort.

Of course, I get about twice as many lines of test than of code but with complex scripts, efforts in testing are a good investment. This is true in particular when your client changes its mind near the end of the project and modifies some requirements. Having a regression test suite is a big aid in changing complex bash code.

3
  • I have started using shunit2 (code.google.com/p/shunit2) which is dead simple and seems to work flawlessly.
    – jpsecher
    Commented Sep 27, 2012 at 10:53
  • 3
    Wouldn't it be easier to use a higher level programming language instead of bash?
    – inf3rno
    Commented Aug 5, 2015 at 2:07
  • 2
    @inf3rno — The question is about bash. Higher level programming languages might not be available (e.g., an embedded system using busy box).
    – mouviciel
    Commented Aug 5, 2015 at 7:04
11

If you are writing code at the same time with tests, try to make it high on functions that don't use anything besides their parameters and don't modify environment. That is, if your function might as well run in a subshell, then it will be easy to test. It takes some arguments and outputs something to stdout, or to a file, or maybe it does something on the system, but caller does not feel side effects.

Yes, you will end up with big chain of functions passing down some WORKING_DIR variable that might as well be global, but this is minor inconvenience comparing to the task of tracking what does each function read and modify. Enabling unit tests is just a free bonus too.

Try to minimize cases where you need output. A little subshell abuse will go long way to keeping things nicely separated (at the expense of performance).

Instead of linear structure, where functions are called, set some environment, then other ones are called, all pretty much on one level, try to go for deep call tree with minimum data going back. Returning stuff in bash is inconvenient if you adopt self-imposed abstinence from global vars...

2
  • 14
    I sense a book coming on: "Functional programming in POSIX sh". Commented Aug 22, 2009 at 15:17
  • 1
    Yes! We need first class function pointers! Or very cheap evals!
    – Eugene
    Commented Aug 22, 2009 at 19:29
4

If you code a bash program large enough to require TDD, you are using the wrong language.

I suggest you to read my previous post on best practices in bash programming, you will probably find something useful to make your bash program testable, but my statement above stays.

Design patterns or best practices for shell scripts

4
  • 10
    No. 1, Bash is a capable language, it can be the right tool for many jobs. 2. Even if this is true, I might not be in the position to choose the tool. This answer simply avoids the question. Sorry.
    – Chen Levy
    Commented Aug 22, 2009 at 9:51
  • 5
    bash is the right tool for many jobs, but not for all of them. If a task is clearly too demanding for a bash script, you are using the wrong tool. As for point 2, I can agree with you, but be very wary of a manager forcing you to use a blatantly wrong tool for a complex task. If it screws up, it will be your fault. Commented Aug 22, 2009 at 9:54
  • You wrote yourself: "I had to use it due to human factors and legacy", it is not "a manager forcing [me] to use a blatantly wrong tool". Anyway thanks for the link to stackoverflow.com/questions/78497/….
    – Chen Levy
    Commented Aug 23, 2009 at 8:10
  • Actually you are half right, half wrong. Using TDD (if possible) is always better than debug. A high level language with OOP is a more natural fit to bigger projects than a low level language like bash.
    – inf3rno
    Commented Aug 5, 2015 at 2:11
3

Writing what Meszaros calls consumer tests is hard in any language. Another approach is to verify the behavior of commands such as rsync manually, then write unit tests to prove specific functionality without hitting the network. In this slightly-modified example, $run is used to print the side-effects if the script is run with the keyword "test"

function distribute {
    local file=$1 ; shift
    for host in $@ ; do
        $run rsync -ae ssh $file $host:$file
    done
}

if [[ $1 == "test" ]]; then
    run="echo"
else
    distribute schedule.txt $*
    exit 0
fi

#    
# Built-in self-tests
#

output=$(mktemp)
expected=$(mktemp)
set -e
trap "rm $got $expected" EXIT

distribute schedule.txt login1 login2 > $output
cat << EOF > $expected
rsync -ae ssh schedule.txt login1:schedule.txt
rsync -ae ssh schedule.txt login2:schedule.txt
EOF
diff $output $expected
echo -n '.'

echo; echo "PASS"
0
3

These did quite a nice job for me:

  • Cucumber - TDD/BDD framework
  • Aruba - Cucumber plugin for testing CLI applications

Additionally, you can stub just about everything you want by doing something like this:

#
# code.sh
#
some_function_calling_some_external_binary()
{
    if ! external_binary action_1; then
        # ...
    fi  

    if ! external_binary action_2; then
        # ...
    fi
}

#
# test.sh
#

# now for the test, simply stub your external binary:
external_binary()
{
    if [ "$@" = "action_1" ]; then
        # stub action_1

    elif [ "$@" = "action_2" ]; then
        # stub action_2

    else
        external_binary $@
    fi  
}
1

The advanced bash scripting guide has an example of an assert function but here is a simpler and more flexible assert function - just use eval of $* to test any condition.

assert() {
  if ! eval $* ; then
      echo
      echo "===== Assertion failed:  \"$*\" ====="
      echo "File \"$0\", line:$LINENO line:${BASH_LINENO[*]}"
      echo line:$(caller 0)
      exit 99
  fi  
}

# e.g. USAGE:
assert [[ $r == 42 ]]
assert "((r==42))"

BASH_LINENO and caller bash builtin are bash shell specific.

0

take a look at Outthentic framework - it is designed to create scenarios which runs any Bash code and then analyze the stdout using formal DSL, it's pretty easy to build any Tdd/blackbox tests suite upon this tool.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.