Has this been discussed, on the mailing lists or somewhere? I wonder whether it would actually decrease the servers’ load.
If you install the live CD and update from factory once a week or so through the whole development cycle, that’s got to be in the 10s of gigs of total downloads, per tester. Goodness only knows what it’d be for the DVD.
Presumably the deltas are created automatically, without input from the packagers, so it’s basically just a question of choosing to enable it.
And I realise that this isn’t, in a sense, the optimum way of testing, because you aren’t testing the installer for each milestone. But it has to be accepted that many people will test this way - and more would, to the advantage of the SUSE project, if the download totals weren’t so nutty.
Has anyone actually run the numbers, to see if it’s feasible?
And on a peripheral note, what the heck is content.key? Losing it once or twice would be forgivable - but this is starting to look like carelessness.
Hope calculations aren’t based on that wild assumption. :\
…But it has to be accepted that many people will test this way - and more would, to the advantage of the SUSE project, if the download totals weren’t so nutty.
Erm… Whose calculations? Why is that a wild assumption? How often should one update, as a tester?
Granted, I have no idea what proportion of testers update from factory, rather than reinstalling milestones, but I suspect it’s a fair few - and that seems to be the way most distros do it. And its uncontroversial that more people would test, if there were less downloading involved… isn’t it?
Any calculations (if any are to be made). You cannot be testing milestones by weekly testing between milestones, can you? You cannot reach a milestone inbetween milestones – by definition. In between milestones, you don’t know what the hell you are testing. Why does a distro tester need to update between milestones when the status of a factory repo is indeterminate? :\ Think about the problem of reproducing bugs.
Granted, I have no idea what proportion of testers update from factory, rather than reinstalling milestones, but I suspect it’s a fair few…
How many is a “fair few”?
And its uncontroversial that more people would test, if there were less downloading involved… isn’t it?
Not necessarily. A guess is bound to be controversial.
Having said that, I am always in favour of reducing the size of downloads, but I also like to get my money’s worth from the ISP.
I’m testing the new packages. What should I be testing?
Sure, you have to reinstall before you file a report, because a bug could be spurious (in many ways I think that - or at least discussing it on the forum, is good practise anyway). But that happens - sometimes the updates break it, sometimes the milestones won’t install. That’s testing, and it’s worked OK for me. Maybe I just got lucky?
How many is a “fair few”?
…
Not necessarily. A guess is bound to be controversial.
…
How about I start with “more than 0”, and we can haggle from there?
Correction:
You cannot be testing milestones by weekly testing of updates between milestones, can you? You cannot reach a milestone between milestones – by definition. In between milestones, you don’t know what the hell you are testing. Why does a distro tester need to update between milestones when the status of a factory repo is indeterminate?
Well, firstly, if you wanted to upgrade using deltas only on release of the milestones, you could do. They could plaster a great big “Upgrade or you’ll miss it!” sign at the top of the forums, and leave us to our own moral responsibility.
But if I’m reading you right, what you seem to be saying is that they shouldn’t do this because they would increase the number of spurious bug reports (or perhaps put off testers).
That may be so, but it’s an empirical question, isn’t it? It will also probably add to the number of people starting testing (and therefore the number of good bug reports), so it’s a question of which factor balances out heavier.
All I’m saying is they could try it, with suitable “If you upgrade between milestones, it might well break” warning signs, and see what happens.
Are any new packages introduced between milestones? Or did you mean package updates. Why bother, unless you know exactly what the packager has changed? In any case you were originally talking about milestone testing e.g.
And I realise that this isn’t, in a sense, the optimum way of testing, because you aren’t testing the installer for each milestone.
But that happens - sometimes the updates break it, sometimes the milestones won’t install. That’s testing, and it’s worked OK for me. Maybe I just got lucky?
No, that’s not really testing, that’s just “suck it and see”. Enjoy it by all means, but please don’t call it testing.
It’s a nice idea, as in the case of the cleaner release of milestones 2,3 and 4. For the last two milestones it seemed to take several days and into the weekend to fully populate factory repo(s) with a monday announcement. Is this the shape of future releases, with fewer resources? The images are more determinate for downloading i.e. there or not.
But if I’m reading you right, what you seem to be saying is that they shouldn’t do this because they would increase the number of spurious bug reports (or perhaps put off testers).
Yes, impossible for devs to reproduce bugs. Yes testers may be put off by frustration.
It doesn’t follow that Increasing the number of testers will increase either the number of bug reports or the quality of them.
If they had time, they could try it… The caveats/disclaimers are already explained IIRC in the links on the wiki where the milestones are announced.
There’s a sense in which it’s an artificial distinction. We’re all testing 11.2 - anything else is instrumental. If the devs want us to be testing against a known baseline, then that’s fair enough - what I’m doing doesn’t really help, and they shouldn’t do anything to make it easier. But as I said, for me it really wasn’t that wild a ride - I only had to reinstall once, although I ended up doing so twice. It is my personal, unscientific belief that if they made it easier to do this, they’d end up with more testers. I guess the point of the thread, apart from the discussion (which has been interesting), was to sound out how hard it would actually be.
No, that’s not really testing, that’s just “suck it and see”. Enjoy it by all means, but please don’t call it testing.
It is testing, if I send off bug reports (having tried reinstalling, to check it wasn’t just my installation). Admittedly I’ve only sent one off this cycle, and that went straight to KDE. But hey, I’ve been busy.
Well, I suppose if anything they could set the servers to only generate deltas upon the release of the milestones. Obviously I’m only guessing that that’s possible. Again, you do have the problem that people aren’t testing the installer then, but again, it is my guess that that would be balanced out.
Yes, impossible for devs to reproduce bugs. Yes testers may be put off by frustration.
It doesn’t follow that Increasing the number of testers will increase either the number of bug reports or the quality of them.
If they had time, they could try it… The caveats/disclaimers are already explained IIRC in the links on the wiki where the milestones are announced.
This is all true, I assume there would be more “I told it to upgrade, and it removed three quarters of KDE, now I can’t log in, you should fix it, LOL!” bug reports.
So you may be right - it might be a bad idea. I’m just curious as to how much discussion / thought has actually gone into it - and it has been my experience (not that I have that much of it) that most other distros have chosen the ‘rolling’ development model. They aren’t fools (well… maybe a few of them ;)), so for that reason alone, I would suggest that it’s worth considering.
[ETA: Also, I guess, they could let you file bugs against factory or the milestones separately (assuming they don’t already - I’m not sure). That would help them filter it.]
