Trying a simple sed thing.

English Programming/Scripting
#1

Hi,

I’m trying a thing with sed (trying to change a single line in a text file, automatically).
From the “info sed” page I got some links:

http://sed.sourceforge.net/grabbag/
http://sed.sourceforge.net/grabbag/tutorials/sed1line.txt
http://sed.sourceforge.net/grabbag/tutorials/do_it_with_sed.txt

on the last one, I look at this section:


There are functions, that handle user text directly (insert, append, change).
The format of that text is

command\
first line\
second line\
....\
last line

no ending \ for the last line

example in a sed script file:

/#include <termios\.h>/{
i\
#ifdef SYSV
a\
#else\
#include <sgtty.h>\
#endif
}

that would search for lines `#include &lt;termios.h&gt;' and then
would write

#ifdef SYSV
#include &lt;termios.h&gt;
#else
#include &lt;sgtty.h&gt;
#endif

Now, for writing the same script on one line, the -e mechanism
is needed... what follows each -e can be considered as an input
line from a sed script file, so nothing kept us from doing

sed -e '/#include &lt;termios\.h&gt;/{' \
-e 'i\' \
-e '#ifdef SYSV' \
-e 'a\' \
-e '#else\' \
-e '#include &lt;sgtty.h&gt;\' \
-e '#endif' \
-e '}'

on the command line, of course the trailing `\'s could be omitted if
we wrote all of this on one line and thus, getting a fast edit-and-test
working

So I have this sample text file to change (to feed to sed):


cer@minas-tirith:~/tmp/sedtest> cat hosts
# IP-Address  Full-Qualified-Hostname  Short-Hostname
#

127.0.0.1       localhost

#nm-mine-placemarker
192.168.42.167  minas-tirith.valinor Minas-Tirith
#marker

#Wifi-casa
cer@minas-tirith:~/tmp/sedtest>

And I try to run this:


cer@minas-tirith:~/tmp/sedtest> sed -n -e '/#nm-mine-placemarker/{' -e 'c\' -e 'Some text test\' -e '}' hosts
sed: -e expression #1, char 0: unmatched `{'
cer@minas-tirith:~/tmp/sedtest&gt;

which has me baffled. The ‘{’ is matched with another ‘}’ at the end of the sequence…

(using “i” as in the sample from the documentation also fails the same).

This other one that I tried before works:


cer@minas-tirith:~/tmp/sedtest> sed -n '/#nm-mine-placemarker/{n;p;}' hosts
192.168.42.167  minas-tirith.valinor Minas-Tirith
cer@minas-tirith:~/tmp/sedtest>

Ideas? Is the documentation wrong?

–
Cheers / Saludos,

Carlos E. R.

(from 13.1 x86_64 “Bottle” (Minas Tirith))

#2

Documentation says that argument to -e is “commands”; and in your example that does not work it is only part of command.

#3

Hi,

This might not be what you are looking for but If you are trying to edit a file i can help with ed. You are trying to replace a line with some text, that is how i parse your question.

Check out this output. The ,p means just print the output to stdout like sed does.

printf '%s
' '/^#nm-mine-placemarker/c' 'Some text test'  . ,p | ed -s hosts

If that is ok then replace the ,p with** w** to make the changes.

printf '%s
' '/^#nm-mine-placemarker/c' 'Some text test'  . w | ed -s hosts

Taken from ed(1p)


Sypnosis

(.,.)c
<text>
.

The c command shall delete the addressed lines, then accept input text that replaces these lines; the current line shall be set to the address of the last line input; or, if there were none, at the line after the last line deleted; if the lines deleted were originally at the end of the buffer, the current line number shall be set to the address of the new last line; if no lines remain in the buffer, the current line number shall be set to zero. Address 0 shall be valid for this command; it shall be interpreted as if address 1 were specified.

Now we have to wait for the sed expert or if i have misunderstood the question please tell me.

#4

On 2014-08-02 16:56, arvidjaar wrote:
>
> Documentation says that argument to -e is “commands”; and in your
> example that does not work it is only part of command.

it is a modified example taken from the documentation directly
referenced on the “info sed” page:


File: sed.info,  Node: Other Resources,  Next: Reporting Bugs,  Prev:
Limitatio\
ns,  Up: Top

6 Other Resources for Learning About `sed'
******************************************

....

Also of interest are
`http://www.student.northpark.edu/pemente/sed/index.htm' and
`http://sed.sf.net/grabbag', which include `sed' tutorials and other
`sed'-related goodies.

And the example is:


sed -e '/#include <termios\.h>/{' \
-e 'i\' \
-e '#ifdef SYSV' \
-e 'a\' \
-e '#else\' \
-e '#include <sgtty.h>\' \
-e '#endif' \
-e '}'

–
Cheers / Saludos,

Carlos E. R.

(from 13.1 x86_64 “Bottle” (Minas Tirith))

#5

The result is

sed -n -e '/#nm-mine-placemarker/{c\Some text test\}' hosts

Final ‘}’ is quoted.

#6

The result is

sed -n -e '/#nm-mine-placemarker/{c\Some text test\}' hosts

Final ‘}’ is quoted.

#7

On 2014-08-02 17:06, jetchisel wrote:
>
> Hi,
>
> This might not be what you are looking for but If you are trying to edit
> a file i can help with ed. You are trying to replace a line with some
> text, that is how i parse your question.

Almost :slight_smile:

What I intend is to replace the line following this flag line:



#nm-mine-placemarker
192.168.42.167  minas-tirith.valinor Minas-Tirith
#marker

I want to replace the entire line that has the IP to my machine,
detected by searching for the previous line (nm-mine-placemarker),
because searching for “minas-tirith.valinor” may find several hits and
fails.

I’ll have a look at what you posted later, thanks :slight_smile:

–
Cheers / Saludos,

Carlos E. R.

(from 13.1 x86_64 “Bottle” (Minas Tirith))

#8

On 2014-08-02 17:36, arvidjaar wrote:

> The result is
> Code:
> --------------------
> sed -n -e ‘/#nm-mine-placemarker/{c\Some text test}’ hosts
> --------------------
>
> Final ‘}’ is quoted.
>
>


cer@minas-tirith:~/tmp/sedtest> sed -n -e '/#nm-mine-placemarker/{c\Some text test\}' hosts
sed: -e expression #1, char 0: unmatched `{'
cer@minas-tirith:~/tmp/sedtest&gt;

Same error… Does it work for you?

–
Cheers / Saludos,

Carlos E. R.

(from 13.1 x86_64 “Bottle” (Minas Tirith))

#9

Of course not. It cannot work because final ‘}’ is quoted.

#10

Hi,

You can add +1 in the code above. Which means + one line below the pattern #nm-mine-placemarker.

printf '%s
' '/^#nm-mine-placemarker/**+1**c' 'Some Test Text' . ,p | ed -s hosts

Again replace the ,p with w to make the changes.

The bash syntax would be.

ed -s hosts <<< $'/^#nm-mine-placemarker/**+1**c
Some Test Text
.
w'

The ^ insinde the** //** means start, it basically means line that starts with a hash followed by nm-mine…
BRE or Basic Regular Expression is the pattern inside that //, it is the same with sed.

#11

On 2014-08-02 18:16, arvidjaar wrote:
>
> robin_listas;2657106 Wrote:
>>
>>
>> Same error… Does it work for you?
>>
>
> Of course not. It cannot work because final ‘}’ is quoted.

Sorry, but I do not see what do you mean.


cer@minas-tirith:~/tmp/sedtest> sed -n -e '/#nm-mine-placemarker/{c\Some
text test\} hosts
>

It is impossible to remove the final quote.

–
Cheers / Saludos,

Carlos E. R.

(from 13.1 x86_64 “Bottle” (Minas Tirith))

#12

On 2014-08-02 18:26, jetchisel wrote:

> The bash syntax would be.
>
>
> Code:
> --------------------
> ed -s hosts <<< $’/^#nm-mine-placemarker/+1c
Some Test Text
.
w’
> --------------------

I get:


cer@minas-tirith:~/tmp/sedtest> ed -s hosts <<< $'/^#nm-mine-placemarker/*+1*c
Some Test Text
.
w'
?
cer@minas-tirith:~/tmp/sedtest>

> The ^ insinde the* //* means start, it basically means line that
> starts with a hash followed by nm-mine…
> BRE or Basic Regular Expression is the pattern inside that //, it is the
> same with sed.

But if I remove those asterisks, it works. So I have one incantation that works, thanks :slight_smile:

What are the triple pipe? The “<<<”? A single or a double I understand, but a tri
ple I don’t. :-

–
Cheers / Saludos,

Carlos E. R.

(from 13.1 x86_64 “Bottle” (Minas Tirith))

#13

On 2014-08-02 19:48, Carlos E. R. wrote:
> On 2014-08-02 18:16, arvidjaar wrote:

> It is impossible to remove the final quote.

Not “quoted”, but escaped!


cer@minas-tirith:~/tmp/sedtest> cp hosts.copy hosts ; sed -e '/#nm-mine-placemarker/{;n;c\' -e 'Some text test' -e '}' hosts
# IP-Address  Full-Qualified-Hostname  Short-Hostname
#

127.0.0.1       localhost

#nm-mine-placemarker
Some text test
#marker

#Wifi-casa
cer@minas-tirith:~/tmp/sedtest>

–
Cheers / Saludos,

Carlos E. R.

(from 13.1 x86_64 “Bottle” (Minas Tirith))

#14

The asterisk was probably created by making it color blue :-(, sorry about that i have learn some thing here to. Not to color code :).
The triple <<< is called a “here-string” the cousin of “here-docs”, which is bash only.

echo foo bar baz  more | grep -o foo

Is the same as

grep -o foo <<< 'foo bar baz more'

One note on sed, it is not a file editor, it is a Stream EDitor, although 99% of the time sed **-i **will do what you want and it is basically

sed blah..blah... original.text > temp.text && mv temp.text original.text

without the -i flag and it is a GNU extension only. It does not write the changes to the original file but to a temp file, unlike what a real text editor does.

#15

On 2014-08-02 16:18, Carlos E. R. wrote:

Just for the curious, the working version :slight_smile:

What I’m doing is hooking a script into network manager; I want to have a line in the “/etc/hosts” file that lists my current network address and machine name, because it is needed by services such as postfix.

If I don’t, postfix complains every minute:


<2.2> 2014-08-01 18:58:13 minas-tirith postfix 30064 - -  fatal: parameter inet_interfaces: no local interface found for 192.168.42.167
<2.4> 2014-08-01 18:58:14 minas-tirith postfix 2019 - -  warning: process /usr/lib/postfix/pickup pid 30064 exit status 1
<2.4> 2014-08-01 18:58:14 minas-tirith postfix 2019 - -  warning: /usr/lib/postfix/pickup: bad command startup -- throttling
<2.2> 2014-08-01 18:59:14 minas-tirith postfix 30079 - -  fatal: parameter inet_interfaces: no local interface found for 192.168.42.167
<2.4> 2014-08-01 18:59:15 minas-tirith postfix 2019 - -  warning: process /usr/lib/postfix/pickup pid 30079 exit status 1
<2.4> 2014-08-01 18:59:15 minas-tirith postfix 2019 - -  warning: /usr/lib/postfix/pickup: bad command startup -- throttling
<2.2> 2014-08-01 19:00:15 minas-tirith postfix 30140 - -  fatal: parameter inet_interfaces: no local interface found for 192.168.42.167
<2.4> 2014-08-01 19:00:16 minas-tirith postfix 2019 - -  warning: process /usr/lib/postfix/pickup pid 30140 exit status 1
<2.4> 2014-08-01 19:00:16 minas-tirith postfix 2019 - -  warning: /usr/lib/postfix/pickup: bad command startup -- throttling
<2.2> 2014-08-01 19:01:16 minas-tirith postfix 30153 - -  fatal: parameter inet_interfaces: no local interface found for 192.168.42.167
<2.4> 2014-08-01 19:01:17 minas-tirith postfix 2019 - -  warning: process /usr/lib/postfix/pickup pid 30153 exit status 1
<2.4> 2014-08-01 19:01:17 minas-tirith postfix 2019 - -  warning: /usr/lib/postfix/pickup: bad command startup -- throttling

So, in directory “/etc/NetworkManager/dispatcher.d” I created a script named “mine”:


#! /bin/sh

CONEXION="$1"
ACCION="$2"
LOGGER=/usr/bin/logger
FACILIDAD=user.notice
#daemon.warn
TAG=nm-mine

$LOGGER -t $TAG -p $FACILIDAD "Going into nm-mine, got '$1', '$2', '$3', '$4', '$5'"


case "$ACCION" in
up)
$LOGGER -t $TAG -p $FACILIDAD "Connection $CONEXION is going up, uuid: '$CONNECTION_UUID', IPv4.0: '$IP4_ADDRESS_0', IPv4.1: '$IP4_ADDRESS_1', IPv4.2: '$IP4_ADDRESS_2', IPv4.3: '$IP4_ADDRESS_3', IPv4.4: '$IP4_ADDRESS_4', Domain.4: '$IP4_DOMAINS', DNS: '$IP4_NAMESERVERS', num routes: '$IP4_NUM_ROUTES', route.0: '$IP4_ROUTE_0', route.1: '$IP4_ROUTE_1'"

$LOGGER -t $TAG -p $FACILIDAD "Connection $CONEXION, uuid: '$CONNECTION_UUID', DHCP4_HOST_NAME: '$DHCP4_HOST_NAME'"

# see "man NetworkManager" for more info.

case "$CONEXION" in
enp0s29f7u1)
sed --in-place=nmhook -e '/#nm-mine-placemarker/{;n;c\' -e '192.168.42.167  minas-tirith.valinor minas-tirith' -e '}' /etc/hosts
;;
wlan0)
sed --in-place=nmhook -e '/#nm-mine-placemarker/{;n;c\' -e '192.168.1.129  minas-tirith.valinor minas-tirith' -e '}' /etc/hosts
;;
*) echo "Interfaz desconocida, la IP en hosts debe estar mal." ;;
esac


;;

down)
$LOGGER -t $TAG -p $FACILIDAD "Connection $CONEXION is going down"
sed --in-place -e '/#nm-mine-placemarker/{;n;c\' -e '127.0.0.1  minas-tirith.valinor minas-tirith' -e '}' /etc/hosts
;;
*)
$LOGGER -t $TAG -p $FACILIDAD "Connection $CONEXION reports action $ACCION for which I do nothing."
;;
esac

This is work in progress, but so far it is usable. I get the correct line in “/etc/hosts”, like:


#nm-mine-placemarker
192.168.1.129  minas-tirith.valinor minas-tirith

or

#nm-mine-placemarker
127.0.0.1  minas-tirith.valinor minas-tirith

which changes automatically as I connect the laptop to my mobile phone or not.

Next step would be to get the actual IP from the “$IP4_ADDRESS_0” parameter automatically, instead of hardcoded in the script. That can wait.
It comes as a string like this: “192.168.42.167/24 192.168.42.129”. The first word is the host address, the second is the gateway.
I know how to do that, I think :slight_smile:

One curiosity is that the syslog entries always go to “/var/log/NetworkManager”, no matter what facility I use. It is ignored.

–
Cheers / Saludos,

Carlos E. R.

(from 13.1 x86_64 “Bottle” (Minas Tirith))