140

I'm writing a bash script which has set -u, and I have a problem with empty array expansion: bash appears to treat an empty array as an unset variable during expansion:

$ set -u
$ arr=()
$ echo "foo: '${arr[@]}'"
bash: arr[@]: unbound variable

(declare -a arr doesn't help either.)

A common solution to this is to use ${arr[@]-} instead, thus substituting an empty string instead of the ("undefined") empty array. However this is not a good solution, since now you can't discern between an array with a single empty string in it and an empty array. (@-expansion is special in bash, it expands "${arr[@]}" into "${arr[0]}" "${arr[1]}" …, which makes it a perfect tool for building command lines.)

$ countArgs() { echo $#; }
$ countArgs a b c
3
$ countArgs
0
$ countArgs ""
1
$ brr=("")
$ countArgs "${brr[@]}"
1
$ countArgs "${arr[@]-}"
1
$ countArgs "${arr[@]}"
bash: arr[@]: unbound variable
$ set +u
$ countArgs "${arr[@]}"
0

So is there a way around that problem, other than checking the length of an array in an if (see code sample below), or turning off -u setting for that short piece?

if [ "${#arr[@]}" = 0 ]; then
   veryLongCommandLine
else
   veryLongCommandLine "${arr[@]}"
fi

Update: Removed bugs tag due to explanation by ikegami.

1
  • Things may be even more nasty when using a construction like cmd "${A[@]}" "${B[@]}" "${C[@]}" where any of the array could be empty
    – U. Windl
    Nov 18, 2022 at 10:44

12 Answers 12

97

According to the documentation,

An array variable is considered set if a subscript has been assigned a value. The null string is a valid value.

No subscript has been assigned a value, so the array isn't set.

But while the documentation suggests an error is appropriate here, this is no longer the case since 4.4.

$ bash --version | head -n 1
GNU bash, version 4.4.19(1)-release (x86_64-pc-linux-gnu)

$ set -u

$ arr=()

$ echo "foo: '${arr[@]}'"
foo: ''

There is a conditional you can use inline to achieve what you want in older versions: Use ${arr[@]+"${arr[@]}"} instead of "${arr[@]}".

$ function args { perl -E'say 0+@ARGV; say "$_: $ARGV[$_]" for 0..$#ARGV' -- "$@" ; }

$ set -u

$ arr=()

$ args "${arr[@]}"
-bash: arr[@]: unbound variable

$ args ${arr[@]+"${arr[@]}"}
0

$ arr=("")

$ args ${arr[@]+"${arr[@]}"}
1
0: 

$ arr=(a b c)

$ args ${arr[@]+"${arr[@]}"}
3
0: a
1: b
2: c

Tested with bash 4.2.25 and 4.3.11.

17
  • 5
    Can anyone explain how and why this works? I'm confused about what [@]+ actually does and why the second ${arr[@]} won't cause an unbound error. Jul 21, 2016 at 9:16
  • 5
    ${parameter+word} only expands word if parameter is not unset.
    – ikegami
    Nov 22, 2016 at 17:51
  • 2
    ${arr+"${arr[@]}"} is shorter and seems to work just as well. Jan 14, 2017 at 18:01
  • 3
    @Per Cerderberg, Doesn't work. unset arr, arr[1]=a, args ${arr+"${arr[@]}"} vs args ${arr[@]+"${arr[@]}"}
    – ikegami
    Jan 15, 2017 at 6:15
  • 1
    To be precise, in cases where the + expansion doesn't occur (namely, an empty array) the expansion is replaced with nothing, which is exactly what an empty array expands to. :+ is unsafe because it also treats a single-element ('') array as unset and similarly expands to nothing, losing the value.
    – dimo414
    May 1, 2020 at 23:00
76

The only safe idiom is ${arr[@]+"${arr[@]}"}

Unless you only care about Bash 4.4+, but you wouldn't be looking at this question if that were the case :)

This is already the recommendation in ikegami's answer, but there's a lot of misinformation and guesswork in this thread. Other patterns, such as ${arr[@]-} or ${arr[@]:0}, are not safe across all major versions of Bash.

As the table below shows, the only expansion that is reliable across all modern-ish Bash versions is ${arr[@]+"${arr[@]}"} (column +"). Of note, several other expansions fail in Bash 4.2, including (unfortunately) the shorter ${arr[@]:0} idiom, which doesn't just produce an incorrect result but actually fails. If you need to support versions prior to 4.4, and in particular 4.2, this is the only working idiom.

Screenshot of different idioms across versions

Unfortunately other + expansions that, at a glance, look the same do indeed emit different behavior. Using :+ instead of + (:+" in the table), for example, does not work because :-expansion treats an array with a single empty element (('')) as "null" and thus doesn't (consistently) expand to the same result.

Quoting the full expansion instead of the nested array ("${arr[@]+${arr[@]}}", "+ in the table), which I would have expected to be roughly equivalent, is similarly unsafe in 4.2.

You can see the code that generated this data along with results for several additional version of bash in this gist.

11
  • 2
    I don't see you testing "${arr[@]}". Am I missing something? From what I can see it works at least in 5.x.
    – x-yuri
    May 9, 2020 at 17:49
  • 2
    @x-yuri yes, Bash 4.4 fixed the situation; you don't need to use this pattern if you know your script will only run on 4.4+, but many systems are still on earlier versions.
    – dimo414
    May 9, 2020 at 22:28
  • Absolutely. Despite looking nice (e.g. formatting), extra-spaces are great evil of bash, causing lots of troubles
    – agg3l
    Aug 4, 2020 at 2:11
  • On bash 4.4.20(1) this doesn't work as intended. the variable expansion quote in this answer does not count the number of items in an array. Worse, it will leave the variable unquoted.
    – inetknght
    Oct 16, 2020 at 15:06
  • @inetknght can you share a MCVE of what you're observing? Despite the odd syntax this is properly quoted. The outer (unqouted) expansion expands to the internal (quoted) expansion when the array is non-empty.
    – dimo414
    Oct 16, 2020 at 22:24
25

@ikegami's accepted answer is subtly wrong! The correct incantation is ${arr[@]+"${arr[@]}"}:

$ countArgs () { echo "$#"; }
$ arr=('')
$ countArgs "${arr[@]:+${arr[@]}}"
0   # WRONG
$ countArgs ${arr[@]+"${arr[@]}"}
1   # RIGHT
$ arr=()
$ countArgs ${arr[@]+"${arr[@]}"}
0   # Let's make sure it still works for the other case...
3
  • No longer makes a difference. bash-4.4.23: arr=('') && countArgs "${arr[@]:+${arr[@]}}" produces 1. But ${arr[@]+"${arr[@]}"} form allows to differentiate between empty/non-empty value by adding/not adding colon.
    – x-yuri
    Jan 25, 2019 at 18:36
  • arr=('') && countArgs ${arr[@]:+"${arr[@]}"} -> 0, arr=('') && countArgs ${arr[@]+"${arr[@]}"} -> 1.
    – x-yuri
    Jan 25, 2019 at 18:48
  • 2
    This has been fixed in my answer long ago. (In fact, I'm sure I've previously left a comment on this answer to that effect?!)
    – ikegami
    Sep 19, 2019 at 11:07
17

Turns out array handling has been changed in recently released (2016/09/16) bash 4.4 (available in Debian stretch, for example).

$ bash --version | head -n1
bash --version | head -n1
GNU bash, version 4.4.0(1)-release (x86_64-pc-linux-gnu)

Now empty arrays expansion does not emits warning

$ set -u
$ arr=()
$ echo "${arr[@]}"

$ # everything is fine
1
  • I can confirm, with bash-4.4.12 "${arr[@]}" would suffice.
    – x-yuri
    Oct 24, 2018 at 20:32
16

this may be another option for those who prefer not to duplicate arr[@] and are okay to have an empty string

echo "foo: '${arr[@]:-}'"

to test:

set -u
arr=()
echo a "${arr[@]:-}" b # note two spaces between a and b
for f in a "${arr[@]:-}" b; do echo $f; done # note blank line between a and b
arr=(1 2)
echo a "${arr[@]:-}" b
for f in a "${arr[@]:-}" b; do echo $f; done
5
  • 11
    This will work if you are just interpolating the variable, but if you want to use the array in a for this would end up with a single empty string when the array is undefined/defined-as-empty, where as you might want the loop body to not run if the array is not defined. Sep 6, 2016 at 9:14
  • 1
    thanks @AshBerlin, I added a for loop to my answer so readers are aware
    – Jayen
    Sep 6, 2016 at 22:59
  • -1 to this approach, it's simply incorrect. This replaces an empty array with a single empty-string, which is not the same. The pattern suggested in the accepted answer, ${arr[@]+"${arr[@]}"}, correctly preserves the empty-array state.
    – dimo414
    May 1, 2020 at 19:43
  • See also my answer showing the situations where this expansion breaks down.
    – dimo414
    May 1, 2020 at 22:57
  • it's not incorrect. it explicitly says it'll give an empty string, and there's even two examples where you can see the empty string.
    – Jayen
    May 2, 2020 at 5:31
7

@ikegami's answer is correct, but I consider the syntax ${arr[@]+"${arr[@]}"} dreadful. If you use long array variable names, it starts to looks spaghetti-ish quicker than usual.

Try this instead:

$ set -u

$ count() { echo $# ; } ; count x y z
3

$ count() { echo $# ; } ; arr=() ; count "${arr[@]}"
-bash: abc[@]: unbound variable

$ count() { echo $# ; } ; arr=() ; count "${arr[@]:0}"
0

$ count() { echo $# ; } ; arr=(x y z) ; count "${arr[@]:0}"
3

It looks like the Bash array slice operator is very forgiving.

So why did Bash make handling the edge case of arrays so difficult? Sigh. I cannot guarantee you version will allow such abuse of the array slice operator, but it works dandy for me.

Caveat: I am using GNU bash, version 3.2.25(1)-release (x86_64-redhat-linux-gnu) Your mileage may vary.

6
  • 9
    ikegami originally had this, but removed it because it is unreliable, both in theory (there is no reason why this should work) and in practice (the OP's version of bash didn't accept it).
    – user743382
    Jun 2, 2014 at 13:57
  • @hvd: Thanks for the update. Readers: Please add a comment if you find versions of bash where the above code does not work.
    – kevinarpe
    Apr 3, 2015 at 13:47
  • hvp already did, and I'll tell you too: "${arr[@]:0}" gives -bash: arr[@]: unbound variable.
    – ikegami
    May 27, 2015 at 14:31
  • One thing that should work across versions is to set a default array value to arr=("_dummy_"), and use the expansion ${arr[@]:1} everywhere. This is mentioned in other answers, referring to sentinel values.
    – init_js
    Mar 7, 2018 at 2:00
  • 1
    @init_js: Your edit was sadly rejected. I suggest you add as a separate answer. (Ref: stackoverflow.com/review/suggested-edits/19027379)
    – kevinarpe
    Mar 12, 2018 at 11:54
6

"Interesting" inconsistency indeed.

Furthermore,

$ set -u
$ echo $#
0
$ echo "$1"
bash: $1: unbound variable   # makes sense (I didn't set any)
$ echo "$@" | cat -e
$                            # blank line, no error

While I agree that the current behavior may not be a bug in the sense that @ikegami explains, IMO we could say the bug is in the definition (of "set") itself, and/or the fact that it's inconsistently applied. The preceding paragraph in the man page says

... ${name[@]} expands each element of name to a separate word. When there are no array members, ${name[@]} expands to nothing.

which is entirely consistent with what it says about the expansion of positional parameters in "$@". Not that there aren't other inconsistencies in the behaviors of arrays and positional parameters... but to me there's no hint that this detail should be inconsistent between the two.

Continuing,

$ arr=()
$ echo "${arr[@]}"
bash: arr[@]: unbound variable   # as we've observed.  BUT...
$ echo "${#arr[@]}"
0                                # no error
$ echo "${!arr[@]}" | cat -e
$                                # no error

So arr[] isn't so unbound that we can't get a count of its elements (0), or a (empty) list of its keys? To me these are sensible, and useful -- the only outlier seems to be the ${arr[@]} (and ${arr[*]}) expansion.

2

I am complementing on @ikegami's (accepted) and @kevinarpe's (also good) answers.

You can do "${arr[@]:+${arr[@]}}" to workaround the problem. The right-hand-side (i.e., after :+) provides an expression that will be used in case the left-hand-side is not defined/null.

The syntax is arcane. Note that the right hand side of the expression will undergo parameter expansion, so extra attention should be paid to having consistent quoting.

: example copy arr into arr_copy
arr=( "1 2" "3" )
arr_copy=( "${arr[@]:+${arr[@]}}" ) # good. same quoting. 
                                    # preserves spaces

arr_copy=( ${arr[@]:+"${arr[@]}"} ) # bad. quoting only on RHS.
                                    # copy will have ["1","2","3"],
                                    # instead of ["1 2", "3"]

Like @kevinarpe mentions, a less arcane syntax is to use the array slice notation ${arr[@]:0} (on Bash versions >= 4.4), which expands to all the parameters, starting from index 0. It also doesn't require as much repetition. This expansion works regardless of set -u, so you can use this at all times. The man page says (under Parameter Expansion):

  • ${parameter:offset}

  • ${parameter:offset:length}

    ... If parameter is an indexed array name subscripted by @ or *, the result is the length members of the array beginning with ${parameter[offset]}. A negative offset is taken relative to one greater than the maximum index of the specified array. It is an expansion error if length evaluates to a number less than zero.

This is the example provided by @kevinarpe, with alternate formatting to place the output in evidence:

set -u
function count() { echo $# ; };
(
    count x y z
)
: prints "3"

(
    arr=()
    count "${arr[@]}"
)
: prints "-bash: arr[@]: unbound variable"

(
    arr=()
    count "${arr[@]:0}"
)
: prints "0"

(
    arr=(x y z)
    count "${arr[@]:0}"
)
: prints "3"

This behaviour varies with versions of Bash. You may also have noticed that the length operator ${#arr[@]} will always evaluate to 0 for empty arrays, regardless of set -u, without causing an 'unbound variable error'.

1
  • Unfortunately the :0 idiom fails in Bash 4.2, so this isn't a safe approach. See my answer.
    – dimo414
    May 1, 2020 at 22:52
1

Here are a couple of ways to do something like this, one using sentinels and another using conditional appends:

#!/bin/bash
set -o nounset -o errexit -o pipefail
countArgs () { echo "$#"; }

arrA=( sentinel )
arrB=( sentinel "{1..5}" "./*" "with spaces" )
arrC=( sentinel '$PWD' )
cmnd=( countArgs "${arrA[@]:1}" "${arrB[@]:1}" "${arrC[@]:1}" )
echo "${cmnd[@]}"
"${cmnd[@]}"

arrA=( )
arrB=( "{1..5}" "./*"  "with spaces" )
arrC=( '$PWD' )
cmnd=( countArgs )
# Checks expansion of indices.
[[ ! ${!arrA[@]} ]] || cmnd+=( "${arrA[@]}" )
[[ ! ${!arrB[@]} ]] || cmnd+=( "${arrB[@]}" )
[[ ! ${!arrC[@]} ]] || cmnd+=( "${arrC[@]}" )
echo "${cmnd[@]}"
"${cmnd[@]}"
1

Now, as technically right the "${arr[@]+"${arr[@]}"}" version is, you never want to use this syntax for appending to an array, ever!

This is, as this syntax actually expands the array and then appends. And that means that there is a lot going on computational- and memory-wise!

To show this, I made a simple comparison:

# cat array_perftest_expansion.sh
#! /usr/bin/bash

set -e
set -u

loops=$1

arr=()
i=0

while [ $i -lt $loops ] ; do
        arr=( ${arr[@]+"${arr[@]}"} "${i}" )
        #arr=arr[${#arr[@]}]="${i}"

        i=$(( i + 1 ))
done

exit 0

And then:

# timex ./array_perftest_expansion.sh 1000

real           1.86
user           1.84
sys            0.01

But with the second line enabled instead, just setting the last entry directly:

arr=arr[${#arr[@]}]="${i}"



# timex ./array_perftest_last.sh 1000

real           0.03
user           0.02
sys            0.00

If that is not enough, things get much worse, when you try to add more entries!

When using 4000 instead of 1000 loops:

# timex ./array_perftest_expansion.sh 4000

real          33.13
user          32.90
sys            0.22

Just setting the last entry:

# timex ./array_perftest_last.sh 4000

real           0.10
user           0.09
sys            0.00

And this gets worse and worse ... I could not wait for the expansion version to finish a loop of 10000!

With the last element instead:

# timex ./array_perftest_last.sh 10000

real           0.26
user           0.25
sys            0.01

Never use such an array expansion for any reason.

1
  • Appending to an empty array never was a problem, and appending an empty array just works (and it's not more complicated). set -u; a=(); a+=()
    – U. Windl
    Nov 18, 2022 at 11:04
0

Interesting inconsistency; this lets you define something which is "not considered set" yet shows up in the output of declare -p

arr=()
set -o nounset
echo ${arr[@]}
 =>  -bash: arr[@]: unbound variable
declare -p arr
 =>  declare -a arr='()'

UPDATE: as others mentioned, fixed in 4.4 released after this answer was posted.

3
  • That's just incorrect array syntax; you need echo ${arr[@]} (but prior to Bash 4.4 you'll still see an error).
    – dimo414
    May 1, 2020 at 22:50
  • Thanks @dimo414, next time suggest an edit instead of downvoting. BTW if you had tried echo $arr[@] yourself you would have seen that the error message is different.
    – MarcH
    May 2, 2020 at 5:51
  • And this "answer" just repeats the question? It should be deleted IMHO.
    – U. Windl
    Nov 18, 2022 at 11:08
-2

The most simple and compatible way seems to be:

$ set -u
$ arr=()
$ echo "foo: '${arr[@]-}'"
2
  • Right, so it's OK for string interpolation but not looping. Jan 18, 2019 at 6:01
  • Maybe refer to the question "What's the difference between an empty parameter and no parameter?" ;-) When expanding into a string it does not make a difference.
    – U. Windl
    Nov 18, 2022 at 11:11

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.