Shifting bash arrays

(Originally published on Reddit, archived here in lightly edited form for posterity.)

Someone just asked me how to shift a bash array. He knew how to shift positional parameters (the familiar shift N command), but when he tried:

shift arr N

he of course got the error: bash: shift: arr: numeric argument required

After reminding him to RTFbashM, I gave him the magic incantation:

arr=("${arr[@]:N}")

bash helpfully extends the ${parameter:offset:length} substring expansion syntax to array slicing as well, so the above simply says “take the contents of arr from index N onwards, create a new array out of it, then overwrite arr with this new value”.

But is it significant that I use @ instead of * as the array index, and are those double quotes really necessary?

YES. Here’s why:

a=(This are a "test with spaces")  # grammatically incorrect to avoid later confusion

echo ${a[0]:2}    => "is"  # substring of a[0]
echo ${a:2}       => "is"  # $a === $a[0], so we get the same result
echo ${a[1]:2}    => "e"   # substring of a[1]

echo ${a[@]:2}    => "a test with spaces"    # slice of a[2:end] as a single string
echo ${a[*]:2}    => "a test with spaces"    # ditto, but...
echo "${a[@]:2}"  => "a" "test with spaces"  # individual elements of a[2:end]
echo "${a[*]:2}"  => "a test with spaces"    # a[2:end] as a concatenated string

And here’s what happens when we try to assign the above array slice attempts to actual arrays:

$ b1=(${a[@]:2})            # expand as single string, bash then word-splits
$ c1=(${a[*]:2})            # ditto
$ b2=("${a[@]:2}")          # expand as individual elements, no bash word-splitting
$ c2=("${a[*]:2}")          # expand as single string, no bash word-splitting

$ declare -p b1 c1 b2 c2
declare -a b1=([0]="a" [1]="test" [2]="with" [3]="spaces")
declare -a c1=([0]="a" [1]="test" [2]="with" [3]="spaces")
declare -a b2=([0]="a" [1]="test with spaces")
declare -a c2=([0]="a test with spaces")

And, with the help of bash namerefs, we can actually create a shift_array function that mimics the shift command for indexed arrays:

# shift_array <arr_name> [<n>]
shift_array() {
  # Create nameref to real array
  local -n arr="$1"
  local n="${2:-1}"
  arr=("${arr[@]:${n}}")
}

$ a=(This is a "test with spaces")
$ shift_array a 2
$ declare -p a
declare -a a=([0]="a" [1]="test with spaces")

How to use the length parameter as a substring/slice length is left as an exercise to the reader.

NOTE: I’ve published shift_array in my bash_functions GitHub repo.