Bash script doesn't handle curly brackets correctly (I believe)

[pbiel]

Hi geekos! And a happy new year to all of you!

I'm having a bit of a trouble with rsync. A working command I use for remote back-ups is this one:

rsync -razP --exclude={'*.git*','*.argh'} ./Data/ bar@foo.online:/home/42/precious_data

However, my plan was to have this as a (re-useable) shell script. And so I went and created this script:

options="-razP --exclude={'*.git*','*.argh'} --delete"
local_dir="./Data/"
remote_dir="bar@foo.online:/home/42/precious_data"

rsync ${options} ${local_dir} ${remote_dir}

It kind-of-works. However, all files (including those I nick-named 'argh' here), will also be copied. That's huge binary files I don't want to have!

Small aberrations, such as using \{…\} or *.argh* did not work. Even the seemingly straight-forward "--exclude *.git* --exclude *.argh" failed. I'm a bit puzzled by now.

Does anyone of you see the (probably stupid) mistake I made here?

[hcvv]

To begin with, this is not a script at all, it is only some statmenets. A sxript should start with the so called "shebang" as first line

#!/usr/bin/bash

To skip some (a lot of) steps that are less important here:
Your rsync statement is first split by the shell into "words". That is done on, amongst more, white space. Then variable expansipn is done, which will replace e.g. the word

${options}

with the string

-razP --exclude={'*.git*','*.argh'} --delete

. That new "word" is NOT split again. It will become ONE argument to the rsync command when it is executed on request of the shell. And the argument

-razP --exclude={'*.git*','*.argh'} --delete

will not be understood by rsync in the way you intended.

I have not tested this, but it may be that using eval will help you:

eval rsync ${options} ${local_dir} ${remote_dir}

[nrickert]

options="-razP --exclude={'*.git*','*.argh'} --delete"

I suggest you try changing the above line to:

options='-razP --exclude={"*.git*","*.argh"} --delete'

Please note that I have not tested this.

[hcvv]

BTW, I will move this to Programming/Scripting.

[pbiel]

Thanks for all the input.

Here are some remarks …

• Using #!/usr/bin/env bash was done (my minimal example was just too minimal, sorry)
• eval alone did not solve it
• eval + using --exclude 'a' -- exclude 'b' … worked, but is not very elegant to use
• in combination with the exchange of " " ↔ ' ', it seems to work

Here's my (for now final) minimal script:

#!/usr/bin/env bash

options='-razP --exclude={"*.git*","*.svn*","*.argh"} --delete'
local_dir='./Data/'
remote_dir='bar@foo.online:/home/42/precious_data'

# copy changes to the server
eval rsync ${options} ${local_dir} ${remote_dir}
exit

[hcvv]

Hm, I tried to reproduce in a test what I said in post #2, but I can not.

[malcolmlewis]

Hi
Just an FYI from a packaging experience, 'env' is not used and stripped from any scripts, the shell is an explicit declaration... `#!/usr/bin/bash` if it's not working then maybe your $PATH has an issue.

[hcvv]

Yes,, the OP's shebang is better, but as there was none (shown), I went for the one that is often understood easiest.

[arvidjaar]

eval alone did not solve it

You do not show your script so we have no way to comment on it.

eval + using --exclude 'a' -- exclude 'b' … worked, but is not very elegant to use

In what way exactly? Your "elegant" solution is

• less flexible as it is impossible to programmatically build exclude list incrementally
• more error prone as now you have to take extra care to quote every possible unwanted shell expansion
• more slow as it adds extra unneeded processing

Having some construct in a language does not mean it must be used by all means. Brace expansion was designed for interactive use to reduce amount of typing. Scripts must be robust and easy to understand unless you attempt to win obfuscated programming contest or prepare your home work.

in combination with the exchange of " " ↔ ' ', it seems to work


Here's my (for now final) minimal script:

Which works just fine if you swap ' and ".

`#!/usr/bin/bash` if it's not working then maybe your $PATH has an issue.

How can $PATH affect absolute pathname?

[chestnet]

rsync -razP --exclude={'*.git*','*.argh'} ./Data/ bar@foo.online:/home/42/precious_data

I would tend to avoid BASH brace expansion for the above example with this instead:

rsync -razP --exclude='*.git*' --exclude='*.argh' ./Data/ bar@foo.online:/home/42/precious_data

When you transition to a much longer list of exclusions, you might be better off with the more flexible --filter parameter that keeps your command line a bit more readable:

#!/bin/bash

cat <<EOF >tmpfilterfile
exclude *.git*
exclude *.argh
exclude *.bak
exclude *~
exclude .deleteme1
exclude .deleteme3
exclude .deleteme5
include .addme2
include .addme4
include .addme6
exclude .gitignore
exclude .git/
EOF

rsync -razP --filter ". tmpfilterfile" ./Data/ bar@foo.online:/home/42/precious_data