we have a problem with dhcpcd at boot time on any openSUSE version from 11.0 to 11.3. It seems that a number of workstations never send out DHCPDISCOVER or DHCPREQUEST at boot time, we have verified this with packet dumps. The dhcp client progress bar is displayed on the console but eventually times out, goes into background and the system continues booting. This is a problem because the timeout takes a long time and users have to wait. Sometimes the display manager is even started but users cannot login yet since we’re using LDAP authentication. Eventually these systems just continue to use their old lease and networking works. Curiously, when we do a network restart after boot, the clients send DHCPDISCOVER/-REQUEST normally, we only have this problem at boot time.
On the server side we’re using ISC dhcpcd-1.3.22pl4-223.13 on SLES 10 SP2. I have read about others who had the same problem, they switched from dhcpcd to dhclient. I have also tried this, but for us dhclient is not an option for a number of other reasons. Another thing I have tried is setting DHCLIENT_SLEEP (“Some interfaces need time to initialize. Add the latency time in seconds”) to up to two minutes to give the interface time to initialize. Unfortunately this didn’t change anything.
Does anyone have any other ideas that we could try or research?
How are you doing the LAN trace, meaning with which command on on which
box? What do you see when you do so? If you are doing it from another
box are you using a hub or a switch? Does the LAN trace show what you
expect on a box that is working? Have you tried tracing when just
restarting the network service? Does it work when using dhclient (even
though that’s apparently not an option in the long term for you)? Do the
logs on your system show anything?
Good luck.
On 08/02/2010 04:36 AM, gundelgauk wrote:
>
> Hello,
>
> we have a problem with dhcpcd at boot time on any openSUSE version from
> 11.0 to 11.3. It seems that a number of workstations never send out
> DHCPDISCOVER or DHCPREQUEST at boot time, we have verified this with
> packet dumps. The dhcp client progress bar is displayed on the console
> but eventually times out, goes into background and the system continues
> booting. This is a problem because the timeout takes a long time and
> users have to wait. Sometimes the display manager is even started but
> users cannot login yet since we’re using LDAP authentication. Eventually
> these systems just continue to use their old lease and networking works.
> Curiously, when we do a network restart after boot, the clients send
> DHCPDISCOVER/-REQUEST normally, we only have this problem at boot time.
>
> On the server side we’re using ISC dhcpcd-1.3.22pl4-223.13 on SLES 10
> SP2. I have read about others who had the same problem, they switched
> from dhcpcd to dhclient. I have also tried this, but for us dhclient is
> not an option for a number of other reasons. Another thing I have tried
> is setting DHCLIENT_SLEEP (“Some interfaces need time to initialize. Add
> the latency time in seconds”) to up to two minutes to give the interface
> time to initialize. Unfortunately this didn’t change anything.
>
> Does anyone have any other ideas that we could try or research?
>
> Regards
>
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.12 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
this is a switched network. I’m doing the packet dump on the SLES 10 box that is running dhcpd with tcpdump -s 65535 -w dumpfile and later inspect it with Wireshark. Usually (with machines that work normally or after boot on problematic machines) I see the normal DHCP traffic. When one of the problematic machines runs dhcpcd during boot, I see nothing at all. It’s like dhcpcd is trying to send DHCPDISCOVER or -REQUEST (as indicated by the progress bar) but actually doesn’t, for some reason.
Again, if I do a network restart after boot, I see the normal DHCP traffic in the packet dumps. The same is the case with dhclient.
What’s really interesting, at home in my private network, I have seen the same effect with both openSUSE 11.1 and 11.3, here the DHCP server is also ISC dhcpd (I don’t know which version right now) on a Linux based embedded IPFire firewall/router. So I’m guessing this might not be caused by our network equipment.
I have looked through the log files, but nothing that I researched indicated any relation with this problem.
The network interfaces of affected machines are “Intel Corporation 82566DM Gigabit Network Connection (rev 02)”, some others have “Intel Corporation 82578DC Gigabit Network Connection (rev 05)” (both on-board, e1000e kernel module). I have also tried to download and compile the latest Intel drivers instead of using the one that ships with openSUSE - to no avail.
I’d be glad for any thoughts/ideas, I have been on this for months now.
Regards
> How are you doing the LAN trace, meaning with which command on on which
> box? What do you see when you do so? If you are doing it from another
> box are you using a hub or a switch? Does the LAN trace show what you
> expect on a box that is working? Have you tried tracing when just
> restarting the network service? Does it work when using dhclient (even
> though that’s apparently not an option in the long term for you)? Do the
> logs on your system show anything?
>
> Good luck.
On a switched network I would be suspect of the switch just in case it is
being clever somehow. Also, are the machine requesting the IP addresses
only running a single NIC with a single (once assigned) IP or do you have
multiple NICs or are you doing any aliasing on the cards? I’d try to use
a hub between the requesting machine and the switch if possible, or see if
you can set your switch to forward all data to a single connection where
you have a box dedicated to tracing. I’m not trying to say this isn’t a
problem with your machine, but I’ve had switches do quirky things in the
past along these lines and hubs have usually shown what really happened
because of their lack of intelligence.
As a note just for fun, tcpdump can have a ‘0’ (zero) as the parameter
after ‘-s’ to capture all information to save you the trouble of typing
65535 over and over (plus teaching other what 2^16-1 is and why that’s
relevant to tcpdump currently).
Good luck.
On 08/02/2010 08:06 AM, gundelgauk wrote:
>
> Thank you for your reply,
>
> this is a switched network. I’m doing the packet dump on the SLES 10
> box that is running dhcpd with tcpdump -s 65535 -w dumpfile and later
> inspect it with Wireshark. Usually (with machines that work normally or
> after boot on problematic machines) I see the normal DHCP traffic.
> When one of the problematic machines runs dhcpcd during boot, I see
> nothing at all. It’s like dhcpcd is trying to send DHCPDISCOVER or
> -REQUEST (as indicated by the progress bar) but actually doesn’t, for
> some reason.
>
> Again, if I do a network restart after boot, I see the normal DHCP
> traffic in the packet dumps. The same is the case with dhclient.
>
> What’s really interesting, at home in my private network, I have seen
> the same effect with both openSUSE 11.1 and 11.3, here the DHCP server
> is also ISC dhcpd (I don’t know which version right now) on a Linux
> based embedded IPFire firewall/router. So I’m guessing this might not be
> caused by our network equipment.
>
> I have looked through the log files, but nothing that I researched
> indicated any relation with this problem.
>
> The network interfaces of affected machines are “Intel Corporation
> 82566DM Gigabit Network Connection (rev 02)”, some others have “Intel
> Corporation 82578DC Gigabit Network Connection (rev 05)” (both on-board,
> e1000e kernel module). I have also tried to download and compile the
> latest Intel drivers instead of using the one that ships with openSUSE -
> to no avail.
>
> I’d be glad for any thoughts/ideas, I have been on this for months now.
>
>
> Regards
>
>
>> How are you doing the LAN trace, meaning with which command on on
> which
>> box? What do you see when you do so? If you are doing it from
> another
>> box are you using a hub or a switch? Does the LAN trace show what
> you
>> expect on a box that is working? Have you tried tracing when just
>> restarting the network service? Does it work when using dhclient
> (even
>> though that’s apparently not an option in the long term for you)? Do
> the
>> logs on your system show anything?
>>
>> Good luck.
>
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.12 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
Checking with the comms gurus they said to check for a switch feature
called “portfast”. Google finds quite a bit on it if you are not familiar
with it already.
On 08/02/2010 08:21 AM, ab@novell.com wrote:
> On a switched network I would be suspect of the switch just in case it is
> being clever somehow. Also, are the machine requesting the IP addresses
> only running a single NIC with a single (once assigned) IP or do you have
> multiple NICs or are you doing any aliasing on the cards? I’d try to use
> a hub between the requesting machine and the switch if possible, or see if
> you can set your switch to forward all data to a single connection where
> you have a box dedicated to tracing. I’m not trying to say this isn’t a
> problem with your machine, but I’ve had switches do quirky things in the
> past along these lines and hubs have usually shown what really happened
> because of their lack of intelligence.
>
> As a note just for fun, tcpdump can have a ‘0’ (zero) as the parameter
> after ‘-s’ to capture all information to save you the trouble of typing
> 65535 over and over (plus teaching other what 2^16-1 is and why that’s
> relevant to tcpdump currently).
>
> Good luck.
>
>
>
>
>
> On 08/02/2010 08:06 AM, gundelgauk wrote:
>
>> Thank you for your reply,
>
>> this is a switched network. I’m doing the packet dump on the SLES 10
>> box that is running dhcpd with tcpdump -s 65535 -w dumpfile and later
>> inspect it with Wireshark. Usually (with machines that work normally or
>> after boot on problematic machines) I see the normal DHCP traffic.
>> When one of the problematic machines runs dhcpcd during boot, I see
>> nothing at all. It’s like dhcpcd is trying to send DHCPDISCOVER or
>> -REQUEST (as indicated by the progress bar) but actually doesn’t, for
>> some reason.
>
>> Again, if I do a network restart after boot, I see the normal DHCP
>> traffic in the packet dumps. The same is the case with dhclient.
>
>> What’s really interesting, at home in my private network, I have seen
>> the same effect with both openSUSE 11.1 and 11.3, here the DHCP server
>> is also ISC dhcpd (I don’t know which version right now) on a Linux
>> based embedded IPFire firewall/router. So I’m guessing this might not be
>> caused by our network equipment.
>
>> I have looked through the log files, but nothing that I researched
>> indicated any relation with this problem.
>
>> The network interfaces of affected machines are “Intel Corporation
>> 82566DM Gigabit Network Connection (rev 02)”, some others have “Intel
>> Corporation 82578DC Gigabit Network Connection (rev 05)” (both on-board,
>> e1000e kernel module). I have also tried to download and compile the
>> latest Intel drivers instead of using the one that ships with openSUSE -
>> to no avail.
>
>> I’d be glad for any thoughts/ideas, I have been on this for months now.
>>
>
>> Regards
>
>
>>> How are you doing the LAN trace, meaning with which command on on
>> which
>>> box? What do you see when you do so? If you are doing it from
>> another
>>> box are you using a hub or a switch? Does the LAN trace show what
>> you
>>> expect on a box that is working? Have you tried tracing when just
>>> restarting the network service? Does it work when using dhclient
>> (even
>>> though that’s apparently not an option in the long term for you)? Do
>> the
>>> logs on your system show anything?
>>>
>>> Good luck.
>
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.12 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
“portfast” is usually only a feature found on Cisco and some HP switches.
What make/model switches are you using?
HTH.
Simon
Novell Knowledge Partner (NKP)
Do you work with Novell technologies at a university, college or school?
If so, your campus could benefit from joining the Novell Technology
Transfer Partners (TTP) group. See www.novell.com/ttp for more details.
I have been suspecting the switches as well of course. I will try to set up a small test environment as you suggested in the next few days; put the client and a trace box on the same hub in front of the actual switch, or use a mirror port. I will also try some other NICs.
As for “portfast”, we do have HP edge switches that support Spanning Tree Protocol, however we have it globally disabled. Still, I will try to do some tests with this in the next few days as well.
As for the other questions:
All of the client machines use only this single on-board interface.
Usually their IP address says the same, unless the lease is not renewed for a very long time and another machine grabs the old address.
We’re not doing any aliasing, bonding or VLAN tagging on the client machines.
Thank you all for all the suggestions so far, I appreciate that. Also thanks for the tip about tcpdump. I didn’t know that.
