Bash Double-quotes, Wildcards, Variable Expansion

Just thought I'd pass this along. I was doing some reading and found these entries on opensuse.org email list pertaining to Bash and the usage of double-quotes in the "for" keyword structure. The links are shown below and one snippet. Also for those starting out, I found http://www.linuxcommand.org/learning_the_shell.php, to be a really nice site, that uses real world examples, and a very logical approach. It was particularly interesting to view their script, "new_script", used to build a new Bash template. It is very complete with documentation, trap, help/usage, getopts, and shows overall good (belt and suspenders) Bash programming techniques (i.e., best practices). The author of the site is working on a book. If it is a reflection of their site, it should be good.

http://lists.opensuse.org/opensuse/2.../msg02240.html
http://lists.opensuse.org/opensuse/2.../msg01738.html

You need to use double quotes: In double quotes, shell variables are expanded, i.e., their value is inserted. In single quotes, variables are not expanded. The correct version of your script is:

for i in *.mp3
do echo "$i"
mp32ogg "$i"
done

Some might argue that the double quotes at echo are superfluous; but they are not: They are proper shell programming. If there is a file with two spaces in it, it is now output the proper way; otherwise the two spaces would have been turned into one space.

The lesson: If you have a shall variable and pass it as an argument to a command, ALWAYS include it in double quotes UNLESS if you want the value broken along its white space. Everything else is sloppy programming and might bite you some time in the future. You can do that in interactive usage, but refrain from it in shell scripts.

Oh yes, and within quotes, shell wildcards are not expanded. Therefore you must not put any quotes around *.mp3, as some posters did it. [/b]

Similar caution should be exercised with when using find with xargs. E.g.

find old/ -type f -name '*.mp3' | xargs rm -f

A filename with a space will cause problems. The way to do it is:

find old/ -type f -name '*.mp3' -print0 | xargs -0 rm -f

Similar caution should be exercised with when using find with xargs. E.g.
...
[/b]

Good point.

Boy find and xargs are very powerful.

Your example reminded me also of, #7, "Use xargs outside of find.", in
"Learn 10 good UNIX usage habits"

I was experimenting with find/xargs to copy newer files in a directory. Any thoughts on the command below:

# Look for files newer than the tag file lastUsed. Do only the current directory and
# print only the file name followed by a Null.
find . -maxdepth 1 -newer ${BackupDir}/lastUsed -type f -printf '%f\0' |
xargs -0tI {} cp -a {} ${BackupDir}/{}.bkup


One thing you don't see in those "for" examples on the net (or Bash books), is what happens if there is no match. In the example in the opensuse.org email list, if there are no mp3 files in the current directory, $i will be "*.mp3". If you start processing $i, you could end up with some pretty surprising results, especially if you don't quote it and wildcard expansion occurs. I suppose one should at least test to see if $i is a file ( [ -f "$i" ]) before file processing, but also maybe check to see if $i is equal to the pattern ([ "$i" -eq "*.mp3" ]).

One thing you don't see in those "for" examples on the net (or Bash books), is what happens if there is no match. In the example in the opensuse.org email list, if there are no mp3 files in the current directory, $i will be "*.mp3". If you start processing $i, you could end up with some pretty surprising results, especially if you don't quote it and wildcard expansion occurs. I suppose one should at least test to see if $i is a file ( [ -f "$i" ]) before file processing, but also maybe check to see if $i is equal to the pattern ([ "$i" -eq "*.mp3" ]).
[/b]

Or you could do one of the following in your script before the expansion:

shopt -s nullglob
shopt -s failglob

See man bash.

I'm running Bash 3.2.25/OpenSuSE 10.3, as is. The default behavior in /bin/Bash, interactive or batch, for the "for command", when there are no matches, is to set the variable to the pattern. But as I re-read the Bash reference, the opengroup standard, and bash manpage, that shouldn't be the case.

"If there are no items in the expansion of words, no commands are executed, and the return status is zero."
http://www.gnu.org/software/bash/manual/ba...ping-Constructs

If no items result from the expansion, the compound-list shall not be executed.
http://www.opengroup.org/onlinepubs/009695...tag_02_09_04_03

The return status is the exit status of the last command that executes. If the expansion of the items following in results in an empty list, no commands are executed, and the return status is 0.
http://www.freebsd.org/cgi/man.cgi?query=b...amp;format=html
Note: Search for "for name" to position by the for command.
[/b]

A simple example: for f in *.nomatch; do echo "$f"; done

Thu 03-13-2008 09:41
===>**for f in *.nomatch; do echo "$f"; done
*.nomatch

According to the doc, the "echo $f" should not occur, but it is.

This appears to be the default behavior in Bash in OpenSuSE.

A shopt -p, shows nullglob to be off.

Something seems wrong. What am I missing here?

Interesting.

Thanks.