Script eating a char.

Hi,

I have a bash script that reads a file and stores it on an array.
The file is text, but msdos formated text: it is actually a a list of
mp3 files in .m3u format. It goes like this:


> #EXTM3U
> #EXTINF:54,01/ - Pietro Locatelli - SkyFM: Mostly Classical - Concerto Grosso in D Major, Op. 1, No. 9 , mvmt. II. Largo (Capella Istropolitana, Jaroslav Krecek)
> Classical\0000 -- Pietro Locatelli - Concerto Grosso in D Major, Op. 1, No. 9 , mvmt. II. Largo (Capella Istropolitana, Jaroslav Krecek).mp3
> #EXTINF:486,02/ - Mauro Giuliani - SkyFM: Mostly Classical - 12 Landler for Flute and Guitar (Mikail Helasvuo, Jukka Savijoki)
> Classical\0001 -- Mauro Giuliani - 12 Landler for Flute and Guitar (Mikail Helasvuo, Jukka Savijoki).mp3
> ...

The problem is that the script “removes” the backslash on the odds line
(excluding the first or header), which is not an escape, it is a DOS
path separator character (Classical\0000). I do not want that,
obviously: it breaks the workings of the file.

The problem happens here:


declare -a EXTINF
declare -a FULLNAME
COUNTTOT=0

NUMEROLINEAS=0
COUNTTOT=0

while read LINEA  ; do

if  $NUMEROLINEAS -eq 0 ]; then
HEADER=$LINEA
NUMEROLINEAS=1
continue
fi

ODD=`expr $NUMEROLINEAS % 2`
if  $ODD -eq 1 ]; then
echo "ODD $NUMEROLINEAS  -- $LINEA"
EXTINF$COUNTTOT]="$LINEA"
else
echo "EVEN $NUMEROLINEAS  -- $LINEA"
#echo
FULLNAME$COUNTTOT]="$LINEA"
COUNTTOT=`expr $COUNTTOT + 1`
fi

NUMEROLINEAS=`expr $NUMEROLINEAS + 1`
done < $FICHERO

Simplifying the code for clarity, we get to this minimal script:


#!/bin/bash
COUNT=0
while read LINEA  ; do
echo $LINEA
COUNT=`expr $COUNT + 1`
if  $COUNT -ge 5 ]; then
break
fi
done < Classical.m3u

That’s all that is needed to see it. The sample file produces this:


> cer@AmonLanc:~/streamtuner2/L2> ./testbad
> #EXTM3U
> #EXTINF:54,01/ - Pietro Locatelli - SkyFM: Mostly Classical - Concerto Grosso in D Major, Op. 1, No. 9 , mvmt. II. Largo (Capella Istropolitana, Jaroslav Krecek)
> Classical0000 -- Pietro Locatelli - Concerto Grosso in D Major, Op. 1, No. 9 , mvmt. II. Largo (Capella Istropolitana, Jaroslav Krecek).mp3
> #EXTINF:486,02/ - Mauro Giuliani - SkyFM: Mostly Classical - 12 Landler for Flute and Guitar (Mikail Helasvuo, Jukka Savijoki)
> Classical0001 -- Mauro Giuliani - 12 Landler for Flute and Guitar (Mikail Helasvuo, Jukka Savijoki).mp3
> cer@AmonLanc:~/streamtuner2/L2>


Compare with the first code section of the post.

It is interesting to notice that the script eats the “”, but not the
control-M chars at the end of each line.

How can I avoid this?


Cheers / Saludos,

Carlos E. R.
(from 12.3 x86_64 “Dartmouth” at Telcontar)

Use “read -r”. Or another program (shell is not really that great at working with random data).

Carlos E. R. wrote:
> Hi,
>
> I have a bash script that reads a file and stores it on an array.
> The file is text, but msdos formated text: it is actually a a list of
> mp3 files in .m3u format. It goes like this:
>
>


>> #EXTM3U
>> #EXTINF:54,01/ - Pietro Locatelli - SkyFM: Mostly Classical - Concerto Grosso in D Major, Op. 1, No. 9 , mvmt. II. Largo (Capella Istropolitana, Jaroslav Krecek)
>> Classical\0000 -- Pietro Locatelli - Concerto Grosso in D Major, Op. 1, No. 9 , mvmt. II. Largo (Capella Istropolitana, Jaroslav Krecek).mp3
>> #EXTINF:486,02/ - Mauro Giuliani - SkyFM: Mostly Classical - 12 Landler for Flute and Guitar (Mikail Helasvuo, Jukka Savijoki)
>> Classical\0001 -- Mauro Giuliani - 12 Landler for Flute and Guitar (Mikail Helasvuo, Jukka Savijoki).mp3
>> ...
> 

The problem is that the script “removes” the backslash on the odds line
(excluding the first or header), which is not an escape, it is a DOS
path separator character (Classical\0000). I do not want that,
obviously: it breaks the workings of the file.

The problem happens here:


> declare -a EXTINF
> declare -a FULLNAME
> COUNTTOT=0
>
>      NUMEROLINEAS=0
>      COUNTTOT=0
>
>      while read LINEA  ; do
>
>         if  $NUMEROLINEAS -eq 0 ]; then
>              HEADER=$LINEA
>              NUMEROLINEAS=1
>              continue
>          fi
>
>          ODD=`expr $NUMEROLINEAS % 2`
>          if  $ODD -eq 1 ]; then
>              echo "ODD $NUMEROLINEAS  -- $LINEA"
>              EXTINF$COUNTTOT]="$LINEA"
>          else
>              echo "EVEN $NUMEROLINEAS  -- $LINEA"
>              #echo
>              FULLNAME$COUNTTOT]="$LINEA"
>              COUNTTOT=`expr $COUNTTOT + 1`
>          fi
>
>        NUMEROLINEAS=`expr $NUMEROLINEAS + 1`
>      done < $FICHERO
> 

Simplifying the code for clarity, we get to this minimal script:


> #!/bin/bash
>      COUNT=0
>      while read LINEA  ; do
>          echo $LINEA
>          COUNT=`expr $COUNT + 1`
>          if  $COUNT -ge 5 ]; then
>             break
>          fi
>      done < Classical.m3u
> 

That’s all that is needed to see it. The sample file produces this:


>> cer@AmonLanc:~/streamtuner2/L2> ./testbad
>> #EXTM3U
>> #EXTINF:54,01/ - Pietro Locatelli - SkyFM: Mostly Classical - Concerto Grosso in D Major, Op. 1, No. 9 , mvmt. II. Largo (Capella Istropolitana, Jaroslav Krecek)
>> Classical0000 -- Pietro Locatelli - Concerto Grosso in D Major, Op. 1, No. 9 , mvmt. II. Largo (Capella Istropolitana, Jaroslav Krecek).mp3
>> #EXTINF:486,02/ - Mauro Giuliani - SkyFM: Mostly Classical - 12 Landler for Flute and Guitar (Mikail Helasvuo, Jukka Savijoki)
>> Classical0001 -- Mauro Giuliani - 12 Landler for Flute and Guitar (Mikail Helasvuo, Jukka Savijoki).mp3
>> cer@AmonLanc:~/streamtuner2/L2>
>
> 

>
> Compare with the first code section of the post.
>
> It is interesting to notice that the script eats the "", but not the
> control-M chars at the end of each line.
>
>
> How can I avoid this?
>

internet search reveals that i should use "read -r " instead of simple
“read” and it works.

source:-
http://www.bashguru.com/2010/05/how-to-read-file-line-by-line-in-shell.html


GNOME 3.6.2
openSUSE Release 12.3 (Dartmouth) 64-bit
Kernel Linux 3.7.10-1.16-desktop

On Wed, 02 Oct 2013 02:33:37 +0000, Carlos E. R. wrote:

> which is not an escape, it is a DOS path separator character

Bash scripting doesn’t know this. All it knows is the \ character is an
escape character.

Quick fix:

s/\/\\/g

Should replace one slash with two in vi.

Jim


Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

On 2013-10-02 05:06, arvidjaar wrote:
> Use “read -r”. Or another program (shell is not really that great at
> working with random data).

“read -r” does it, thanks: just tried.

I had already finished the program this afternoon when I noticed that
/problem/ in the output. The shell is limited, but considering that my
choices are bash or pascal (or even C), I thought that bash would be
faster coding for this simple things :wink:

I was considering using another language, because I really needed two
dimensional arrays, and bash arrays are one dimension only. So I made do
with array_a, array_b… ugly, but it works. If I get more spare time or
the push for it, I’ll recode on another language.


Cheers / Saludos,

Carlos E. R.
(from 12.3 x86_64 “Dartmouth” at Telcontar)

On 2013-10-02 05:24, vazhavandan wrote:
> Carlos E. R. wrote:

> internet search reveals that i should use "read -r " instead of simple
> “read” and it works.

I did not think that the culprit was “read”. The bash man page is huge,
difficult to find things.

> source:-
> http://www.bashguru.com/2010/05/how-to-read-file-line-by-line-in-shell.html

Interesting link, thanks.


Cheers / Saludos,

Carlos E. R.
(from 12.3 x86_64 “Dartmouth” at Telcontar)

On 2013-10-02 05:26, Jim Henderson wrote:
> On Wed, 02 Oct 2013 02:33:37 +0000, Carlos E. R. wrote:

> Quick fix:
>
> s/\/\\/g
>
> Should replace one slash with two in vi.

Yes, but that means altering the source.m3u files.
I actually edited out the output file to solve the issue for the time
being, this afternoon…


Cheers / Saludos,

Carlos E. R.
(from 12.3 x86_64 “Dartmouth” at Telcontar)

On Wed, 02 Oct 2013 03:58:07 +0000, Carlos E. R. wrote:

> On 2013-10-02 05:26, Jim Henderson wrote:
>> On Wed, 02 Oct 2013 02:33:37 +0000, Carlos E. R. wrote:
>
>
>> Quick fix:
>>
>> s/\/\\/g
>>
>> Should replace one slash with two in vi.
>
> Yes, but that means altering the source.m3u files.
> I actually edited out the output file to solve the issue for the time
> being, this afternoon…

That’d do it as well. Another option would be to just run the source
files through a quick script (you could probably do it with sed, even) to
add the second slash inline when you’re doing the other parsing of the
file.

Jim


Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

On 2013-10-02 06:41, Jim Henderson wrote:

> That’d do it as well. Another option would be to just run the source
> files through a quick script (you could probably do it with sed, even) to
> add the second slash inline when you’re doing the other parsing of the
> file.

Ah, yes, I see… It did not occur to me.

But adding the “-r” to the read in the script works very well and easy :slight_smile:


Cheers / Saludos,

Carlos E. R.
(from 12.3 x86_64 “Dartmouth” at Telcontar)