Page 1 of 2 12 LastLast
Results 1 to 10 of 11

Thread: Bash script doesn't handle curly brackets correctly (I believe)

  1. #1
    Join Date
    Jun 2014
    Location
    Bonn, Germany
    Posts
    78

    Lightbulb Bash script doesn't handle curly brackets correctly (I believe)

    Hi geekos! And a happy new year to all of you!

    I'm having a bit of a trouble with rsync. A working command I use for remote back-ups is this one:
    Code:
    rsync -razP --exclude={'*.git*','*.argh'} ./Data/ bar@foo.online:/home/42/precious_data
    However, my plan was to have this as a (re-useable) shell script. And so I went and created this script:
    Code:
    options="-razP --exclude={'*.git*','*.argh'} --delete"
    local_dir="./Data/"
    remote_dir="bar@foo.online:/home/42/precious_data"
    
    rsync ${options} ${local_dir} ${remote_dir}
    It kind-of-works. However, all files (including those I nick-named 'argh' here), will also be copied. That's huge binary files I don't want to have!

    Small aberrations, such as using \{…\} or *.argh* did not work. Even the seemingly straight-forward "--exclude *.git* --exclude *.argh" failed. I'm a bit puzzled by now.

    Does anyone of you see the (probably stupid) mistake I made here?

  2. #2
    Join Date
    Jun 2008
    Location
    Netherlands
    Posts
    30,990

    Default Re: Bash script doesn't handle curly brackets correctly (I believe)

    To begin with, this is not a script at all, it is only some statmenets. A sxript should start with the so called "shebang" as first line
    Code:
    #!/usr/bin/bash

    To skip some (a lot of) steps that are less important here:
    Your rsync statement is first split by the shell into "words". That is done on, amongst more, white space. Then variable expansipn is done, which will replace e.g. the word
    Code:
    ${options}
    with the string
    Code:
    -razP --exclude={'*.git*','*.argh'} --delete
    . That new "word" is NOT split again. It will become ONE argument to the rsync command when it is executed on request of the shell. And the argument
    Code:
    -razP --exclude={'*.git*','*.argh'} --delete
    will not be understood by rsync in the way you intended.

    I have not tested this, but it may be that using eval will help you:
    Code:
    eval rsync ${options} ${local_dir} ${remote_dir}
    Henk van Velden

  3. #3
    Join Date
    Aug 2010
    Location
    Chicago suburbs
    Posts
    16,018
    Blog Entries
    3

    Default Re: Bash script doesn't handle curly brackets correctly (I believe)

    Quote Originally Posted by pbiel View Post
    Code:
    options="-razP --exclude={'*.git*','*.argh'} --delete"
    I suggest you try changing the above line to:
    Code:
    options='-razP --exclude={"*.git*","*.argh"} --delete'
    Please note that I have not tested this.
    openSUSE Leap 15.4; KDE Plasma 5.24.4;
    testing Tumbleweed.

  4. #4
    Join Date
    Jun 2008
    Location
    Netherlands
    Posts
    30,990

    Default Re: Bash script doesn't handle curly brackets correctly (I believe)

    BTW, I will move this to Programming/Scripting.
    Henk van Velden

  5. #5
    Join Date
    Jun 2014
    Location
    Bonn, Germany
    Posts
    78

    Default Re: Bash script doesn't handle curly brackets correctly (I believe)

    Thanks for all the input.

    Here are some remarks …

    • Using #!/usr/bin/env bash was done (my minimal example was just too minimal, sorry)
    • eval alone did not solve it
    • eval + using --exclude 'a' -- exclude 'b' … worked, but is not very elegant to use
    • in combination with the exchange of " " ↔ ' ', it seems to work


    Here's my (for now final) minimal script:
    Code:
    #!/usr/bin/env bash
    
    options='-razP --exclude={"*.git*","*.svn*","*.argh"} --delete'
    local_dir='./Data/'
    remote_dir='bar@foo.online:/home/42/precious_data'
    
    # copy changes to the server
    eval rsync ${options} ${local_dir} ${remote_dir}
    exit

  6. #6
    Join Date
    Jun 2008
    Location
    Netherlands
    Posts
    30,990

    Default Re: Bash script doesn't handle curly brackets correctly (I believe)

    Hm, I tried to reproduce in a test what I said in post #2, but I can not.
    Henk van Velden

  7. #7
    Join Date
    Jun 2008
    Location
    East of Podunk
    Posts
    33,263
    Blog Entries
    15

    Default Re: Bash script doesn't handle curly brackets correctly (I believe)

    Hi
    Just an FYI from a packaging experience, 'env' is not used and stripped from any scripts, the shell is an explicit declaration... `#!/usr/bin/bash` if it's not working then maybe your $PATH has an issue.
    Cheers Malcolm °¿° SUSE Knowledge Partner (Linux Counter #276890)
    SUSE SLE, openSUSE Leap/Tumbleweed (x86_64) | GNOME DE
    If you find this post helpful and are logged into the web interface,
    please show your appreciation and click on the star below... Thanks!

  8. #8
    Join Date
    Jun 2008
    Location
    Netherlands
    Posts
    30,990

    Default Re: Bash script doesn't handle curly brackets correctly (I believe)

    Yes,, the OP's shebang is better, but as there was none (shown), I went for the one that is often understood easiest.
    Henk van Velden

  9. #9
    Join Date
    Sep 2012
    Posts
    7,880

    Default Re: Bash script doesn't handle curly brackets correctly (I believe)

    Quote Originally Posted by pbiel View Post
    eval alone did not solve it
    You do not show your script so we have no way to comment on it.
    eval + using --exclude 'a' -- exclude 'b' … worked, but is not very elegant to use
    In what way exactly? Your "elegant" solution is


    • less flexible as it is impossible to programmatically build exclude list incrementally
    • more error prone as now you have to take extra care to quote every possible unwanted shell expansion
    • more slow as it adds extra unneeded processing


    Having some construct in a language does not mean it must be used by all means. Brace expansion was designed for interactive use to reduce amount of typing. Scripts must be robust and easy to understand unless you attempt to win obfuscated programming contest or prepare your home work.
    in combination with the exchange of " " ↔ ' ', it seems to work


    Here's my (for now final) minimal script:
    Which works just fine if you swap ' and ".

    Quote Originally Posted by malcolmlewis View Post
    `#!/usr/bin/bash` if it's not working then maybe your $PATH has an issue.
    How can $PATH affect absolute pathname?

  10. #10
    Join Date
    Jan 2009
    Location
    San Jose, CA, USA
    Posts
    30

    Default Re: Bash script doesn't handle curly brackets correctly (I believe)

    Quote Originally Posted by pbiel View Post

    Code:
    rsync -razP --exclude={'*.git*','*.argh'} ./Data/ bar@foo.online:/home/42/precious_data
    I would tend to avoid BASH brace expansion for the above example with this instead:

    Code:
    rsync -razP --exclude='*.git*' --exclude='*.argh' ./Data/ bar@foo.online:/home/42/precious_data
    When you transition to a much longer list of exclusions, you might be better off with the more flexible --filter parameter that keeps your command line a bit more readable:

    Code:
    #!/bin/bash
    
    cat <<EOF >tmpfilterfile
    exclude *.git*
    exclude *.argh
    exclude *.bak
    exclude *~
    exclude .deleteme1
    exclude .deleteme3
    exclude .deleteme5
    include .addme2
    include .addme4
    include .addme6
    exclude .gitignore
    exclude .git/
    EOF
    
    rsync -razP --filter ". tmpfilterfile" ./Data/ bar@foo.online:/home/42/precious_data

Page 1 of 2 12 LastLast

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •