NTP and downloading

On my 11.3 KDE if I leave the computer alone to download something and come back to check after a while I sometimes find that torrents or usenet downloads have been stopped long ago and kdeapplet displays the wrong time, anything from half and hour to ten hours late.

I usually open NTP configuration screen, check if server is accessible and okay out of it and everything goes back to normal by itself, time is corrected and downloads resumed.

What could be the reason for stopping downloads and how can I correct this problem? Is NTP the cause or just another symptom of a deeper problem?

NTP is set to “now and on boot”, interval is set to 5 min - I think those are all default settings.

I do not think NTP is the cause of your problem (why should it), it suffers from the same phenomenon as your downloads, it stopped functioning. Worse even, the time counting in your system stopped (when NTP does not function, that only means that the synchronizing of the system clock with te outside world stops, not that the system clock stops).

Aren’t you sure that your system is not set to going to sleep when there is no action for a certain time on your desktop? This is a wild guess, but to me it looks if the system goes hibernate or something like that.

That could be the reason and I’ve been battling with KDE power management for ages about screen time out, perhaps deserves a new thread when I get really annoyed with it.

But, there’s no sleep or hibernation in those settings, it’s a desktop, not a notebook. It also doesn’t explain the random nature of the problem. Sometimes it goes on for weeks, sometimes it happens every day.

Also, why simply restarting NTP magically solves it?

Just to gather some more information that may lead to somebody saying: “it may be that …”.

Do you do that downloading from the desktop using a GUI application or from a terminal emulator thats stays open. Or do you let it run in the background with the desktop loged out, or from a console session again with no desktop running?

Did you ever have a problem with your hardware clock not working when the system is off? I admit that that is difficult to see if at boot the system is synchronised immediatly by NTP. I am trying to find out if your CMOS battery is still OK.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Going back to the original NTP theory, perhaps (and this is maybe a
stretch, mostly depending on how the various applications are coded) the
downloads don’t get anything new downloaded from the torrent seeds for a
thirty-minute (or ten hour) period and as a result they give up completely
at that point. Seems odd, since torrent apps I’ve used (transmission in
Gnome, along with Azureus/Vuze which is Java-based and pretty good) are
made to try finding seeds until things are done. I’d be less surprised if
non-torrent downloads failed in this case. The lack of data, then, would
come from NTP’s problem changing the time significantly (thirty minutes,
ten hours, etc.) and that’d be the root cause. All of this is dependent
on the applications tracking time themselves rather than just relying on
the OS to handle the lower layers without any application checks… seems
far-fetched.

I, like hcw, would recommend verifying power settings. I usually disable
all of them on my systems, desktops or laptops, and haven’t had this issue
in my Gnome-based environment (11.3 x86_64).

Good luck.


Want to yell at me in person?
Come to BrainShare 2011 in October: http://tinyurl.com/brainshare2011
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.15 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJOOAZCAAoJEF+XTK08PnB574cP/jlqaYOwyYLoxpobxiixDBj+
Yfxz9U+kmnlSnyN3Syuk/Z7bk3jwQ9HFd0/IuEGHD9elFTswy6uRgZJCjGXs9b/i
QO8epz0AxfCSCQR4j9PolKDkpymTyesPLRS9nPqPBXixH3Vs2hPh4pD+kQJAT+2g
fTQkcGtHT8EqWOMAMyt3XtcQ/mQhdOFUuycY2I9Z1h8tuQqqwmFT2yDycPwlX4LD
kIqRBECNm/4Xvpld85xdaAq6eUH7aAP8J+qMd8I3TSDmPxayO69ggC/R9npBpG+a
+LaYSvA6HV1mbBUuG9378yKLfShB7ebnelh5wKiWKHM4xK0K/OVDEsQX2xupYvwZ
UHVEQlPgouvZfUoadpPbscQl/pKtqFJ6wE+2ZgQk5c3NOwD2KiGJbnv7SHo8DB6p
Z9/H7BLn9qw2zv7aobutsrHX72A2rE1YxC34ZBlVsmnaFG3DFX3udyAvYG2xk9J2
0/PH7QoliUnet1YJGFdnStU7W/rd7+wDMa3ykC6kCoyM40JxHshJR7XVIDPyiVtG
+0EGMBIOaG30kuvo9jctKeRknJ4zdOCCoIUhuRY5U3wgyuiiDcu0HYtltGIfUKl5
GFji9OAvGKKTgW5ywVSvDO0VdKC3wqrmCd0G2NUJI82r7drfLoF8ELFXRjEA5mwF
x8WnJLCKdT0/+Dpo3/aZ
=W7Gp
-----END PGP SIGNATURE-----

Today at boot my system run a file system check on a drive and I had a chance to see that hardware clock, before NTP sync, is late by some thirty minutes.

I do all downloading from GUI, desktop is on, and I never log out. I use kTorrent and Grabit for usenet, in wine. Both are affected. Grabit uses ten connections to download files simultaneously. When time stops the existing connections continue until they finish assigned files but won’t reconnect again to get new ones.

Why does NTP stop syncing for hours? Why does it need a manual reset?

Torrents and usenet can be restarted manually, too, but it’s this NTP thing that fixes everything itself is puzzling me.

On 08/02/2011 04:36 PM, Stan Ice wrote:

> Today at boot my system run a file system check on a drive and I had a
> chance to see that hardware clock, before NTP sync, is late by some
> thirty minutes.

are you dual booting with Windows?
do you have your hardware clock set to local or UTC? (if booting with
win it must be local)


DD
Caveat-Hardware-Software
openSUSE®, the “German Engineered Automobiles” of operating systems!

I still guess it it has somethiing to do with your hardware clock. When it gets a kick from NTP it starts running again (for a while). You could do some experiments like:
Stop NTP completely. See what happens. Next time when the system hangs, then set the clock by hand (using YaST, or the date command as root) to see if it also starts everything again. When that is the case, it proves that NTP is not the cause (which I refuse to believe) of the problem, but the cause of the resuming.

Also, when you say your hardware clock fails to keep time when the system is switched off (it may be a few seconds off in about a day, but not that much), the you should replace your CMOS battery or do a likewise hardware repair (I am not that good at hardware).

And as last remark, I do not know anything about MS Windows, thus I can not comment on you using these products, running in Wine on openSUSE (where I can use many Torrent applications directly under openSUSE), but I guess that that is not part of the problem apart from iit all being inside a desktp environment that may do power management.

Yes, there’s Windows here, too, but I haven’t booted into it in, maybe a year.

Should I reboot in Windows and check if it’s local or UTC? I’d rather reformat it altogether and reclaim some more space for Opensuse. What to do with the hardware clock, though. It could only be reset in Bios, right? It would still be wrong, maybe only by a minute or two but wrong, comparing to synchronized time.

Also, my clock is not set back by ten hours. I mean if I leave the machine on overnight it might still show 11PM when I get up in the morning, and no progress on my torrents.

That setting Local/UTC is not in Windows, it is in openSUSE. You can check with YaST > System > Date and Time. It is the checkbox lower left.

But I think you assessment is correct. Even when this setting is wrong, it does not explain the 10 hours difference.

hcvv wrote:
> I still guess it it has somethiing to do with your hardware clock. When
> it gets a kick from NTP it starts running again (for a while).

But the hardware clock is completely irrelevant except at boot-time and
shutdown. The system doesn’t use it while running; the kernel keeps time
itself, optionally with help from ntpd.

As others have said, I’d try disabling all power management and
screensavers etc to see what happens. And be very systematic about
recording all the conditions surrounding the event, to see if some
pattern emerges. Also, it might be possible to increase the log level on
any suspect software.

I agree that the hardware clock is only relevant rom shutdown to boot.

Just looked - UTC box in Date and Time is unchecked.

Re. hardware clock - I admit I have no idea how that works and syncs, I think all my assumptions have been wrong.

I saw it being half hour late when I booted up at 2PM today but that could be a leftover from the time freeze earlier this morning even if I reset NTP before shutting down.

I agree that NTP is probably not the cause but it’s a very curious symptom nevertheless.

So, what should I try next? Set NTP to manual or synchronize without a daemon and wait for the next time freeze?

Grabit is not for torrents, it’s a usenet client, there aren’t any decent usenet clients on Linux, certainly not in repos. I tried manually installing Sabnzb and a couple of others but never succeeded. No alternative to QuickPar for repairing downloaded files either. In Wine they both run like a breeze with minimal footprint. It’s a whole other topic, though.

Speaking of logs, here’s NTP log for my morning session


2 Aug 07:58:01 ntpd[2682]: synchronized to 128.59.16.20, stratum 2
 2 Aug 07:58:01 ntpd[2682]: kernel time sync status change 2001
 2 Aug 10:48:13 ntpd[2682]: ntpd exiting on signal 15
 2 Aug 11:21:18 ntpd[7709]: synchronized to 128.59.16.20, stratum 2
 2 Aug 11:21:18 ntpd[7709]: kernel time sync status change 2001
2 Aug 11:33:38 ntpd[7709]: ntpd exiting on signal 15

This particular part is when there was a time freeze and me resetting NTP half an hour later

2 Aug 10:48:13 ntpd[2682]: ntpd exiting on signal 15
2 Aug 11:21:18 ntpd[7709]: synchronized to 128.59.16.20, stratum 2

On Tue, 02 Aug 2011 14:36:03 +0000, Stan Ice wrote:

> Why does NTP stop syncing for hours?

NTP itself isn’t about regularly updating the clock, it’s about figuring
out an adjustment to the clock to keep it in sync.

Thus, when NTP runs it runs several requests over a short period of time,
and then as the adjustment gets better, it queries the time source less
frequently (because the clock drift becomes better adjusted).

It sounds like the hardware clock isn’t getting updated. Try the
following:

sudo /etc/init.d/ntp stop
sudo sntp -s tick.usno.navy.mil
sudo hwclock -w
sudo /etc/init.d/ntp start

What these commands do is a one-time clock adjustment (you have to
disable the ntp daemon to do this) and then write the current time to the
hardware clock.

As others have suggested, if you’re dual-booting Windows and openSUSE,
then you may need to tweak the settings on one or the other so they both
use either the local time zone or UTC.

Jim

Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

On 08/02/2011 01:07 PM, Jim Henderson wrote:
> On Tue, 02 Aug 2011 14:36:03 +0000, Stan Ice wrote:
>
>> Why does NTP stop syncing for hours?
>
> NTP itself isn’t about regularly updating the clock, it’s about figuring
> out an adjustment to the clock to keep it in sync.
>
> Thus, when NTP runs it runs several requests over a short period of time,
> and then as the adjustment gets better, it queries the time source less
> frequently (because the clock drift becomes better adjusted).
>
> It sounds like the hardware clock isn’t getting updated. Try the
> following:
>
> sudo /etc/init.d/ntp stop
> sudo sntp -s tick.usno.navy.mil
> sudo hwclock -w
> sudo /etc/init.d/ntp start
>
> What these commands do is a one-time clock adjustment (you have to
> disable the ntp daemon to do this) and then write the current time to the
> hardware clock.
>
> As others have suggested, if you’re dual-booting Windows and openSUSE,
> then you may need to tweak the settings on one or the other so they both
> use either the local time zone or UTC.

This problem is clearly not a matter of local versus UTC time. That would be a
fixed offset that depends on the user’s time zone. The OP’s computer seems to be
having trouble with its real-time clock.

Is there anything in the dmesg output that might help understand the problem?

On Tue, 02 Aug 2011 18:43:22 +0000, Larry Finger wrote:

> This problem is clearly not a matter of local versus UTC time. That
> would be a fixed offset that depends on the user’s time zone. The OP’s
> computer seems to be having trouble with its real-time clock.

True, but I would think that getting the hardware clock synced up with
reality certainly would help.

But yes, it wouldn’t be just a question of UTC/local time differences, as
he said it’s 30 minutes to 10 hours difference.

Jim


Jim Henderson
openSUSE Forums Administrator
Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

On 2011-08-02 17:16, hcvv wrote:
>
> I still guess it it has somethiing to do with your hardware clock. When
> it gets a kick from NTP it starts running again (for a while). You could
> do some experiments like:

Even if you remove the battery completely, the system can work and keep
correct time - it would just be very incorrect during boot (I tried this
time ago).

The clock that runs from that battery has no effect when the system is up
and running, it is ignored.


Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” at Telcontar)

On 2011-08-02 22:49, Jim Henderson wrote:
> True, but I would think that getting the hardware clock synced up with
> reality certainly would help.

Not really… sorry.


Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” at Telcontar)

On 2011-08-02 15:46, Stan Ice wrote:
> Also, why simply restarting NTP magically solves it?

No, it corrects the symptoms.

When ntp is running and the clock gets unset by a lot, ntp quits.
Restarting it jump-sets the clock again and then the daemon can keep the
clock again.

But the cause that made the machine stop is unsolved. Stopped downloads,
clock off, ntp quitting… those are simply symptoms, consequences of the
main problem.

So, people, don’t worry about the clock being wrong, the hardware clock,
the system clock, ntp, etc. Those are not causing the problem.

I had that problem, or very similar, for years on another machine: it would
simply stop processing after I stopped typing or moving the mouse. It
froze, clock stopped. We never learnt the cause, I think it is still
happening, but I don’t use that machine nowdays. Some kernels cured it
some, others made it worse.

I could prevent it from happening by making another machine ping it
continuously, because that caused interrupts in the hardware ethernet card.


Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” at Telcontar)