Moving /usr to separate partition

I have a dual boot system with Vista and OpenSUSE 11.3 . Linux is distributed over 2 partitions: one for /home and one root partition for all the rest. As this root partition is getting filled, I thought of taking a 10 GB partition from the Vista partition and using this for the /usr folder (= 6 GB). This partition is a primary partition, while the rest of Linux is on secondary partitions.
To be save, I renamed the existing /usr to /usr-old and created an empty /usr as mount point. I changed fstab to load the root partition, the /usr partition and then the /home partition.

But when I started the system, there were a lot of errors about files not found in the /usr folder, althought this folder and is content were clearly present when browsing the filesystem. What went wrong? Hard links? Other system configurations to change? Not possible to put /usr on a separate partition?

Sincerely,
Hubert

On 07/24/2011 06:26 PM, HuibC wrote:
>
> I renamed the existing /usr to /usr-old and created an
> empty /usr as mount point. … But when I started the system, there were a lot of errors about files
> not found in the /usr folder

yep, if the only mounted folder named /usr available is empty, then the
system can’t find the stuff it is looking for in /usr

what i think you should do:
-read caveat in my sig
-boot with a Live CD
-mount the needed partitions
-copy all the contents of /usr-old into /usr
-shut down
-cross your fingers and boot from the hard drive again
-if this time it finds the stuff in /usr all might be well, maybe…
-when you are sure it is, delete /usr-old


DD Caveat

what you describe is what I did. I think that you forgot to read the end of the cited sentence: ’ althought this folder (= /usr) and is content were clearly present when browsing the filesystem’
Hubert

On 07/24/2011 07:46 PM, HuibC wrote:
>
> what you describe is what I did.

i was not trying to describe what i think you should have done, but
rather what i thought was the way to solve the problem you now face…

however, i then read this:

> I think that you forgot to read the end of the cited sentence

and guess i’m probably very confused…and now will back out of this
thread…

but, before i back away

  1. i ask what is the file system on the new /usr

  2. and, since i have to ask that , i guess anyone who can help is going
    to want more info, so please show us the terminal input and output from


cat /etc/SuSE-release
df -h
sudo /sbin/fdisk -l
cat /proc/partitions
cat /etc/fstab
mount

copy/paste the output back to this thread using the instructions here:
http://goo.gl/i3wnr

  1. why do you mention hard links in your initial post?


DD
openSUSE®, the “German Engineered Automobiles” of operating systems!

On 2011-07-24 18:26, HuibC wrote:
> But when I started the system, there were a lot of errors about files
> not found in the /usr folder, althought this folder and is content were
> clearly present when browsing the filesystem. What went wrong? Hard
> links? Other system configurations to change? Not possible to put /usr
> on a separate partition?

You certainly have to copy links as links. If you do it with cp you have to
use certain options I never remember (perhaps -a, dunno). Or use rsync; I
use "–archive --acls --xattrs --hard-links --del --stats --human-readable "

Try again to copy the new usr partition. It should work, I have a a
separate usr.


Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” at Telcontar)

If you plan to upgrade, I don’t think seperating / & /usr is a great idea nowadays.
Unforuntately early system startup rules of udev have packages using things in /usr silently failing, so you might try something like bluetooth one day and have it not work, for no apparent reason.

If you’re short of space in /, did you turn on deletion of old temporary files in /tmp & /var/tmp? With time quite a lot of space can be consumed by things that should have cleaned up, but didn’t. The YaST sysconfig editor can help you enable automatic cleanup (should be default, but some resist change on grounds SUSE Linux didn’t do it in past). Moving /var out of / can also make sense, particularly if you have large amount of cache files. Using bind mounts can splice directories like /srv onto the new partition if there’s significant data stored there.

As to the question, it ought not be “hard links”, they would be copied to the new /usr. I am not sure you copied the data correctly, I would suspect a permissions problem; not everything on startup uses “root” but has other UID/GID for security reasons.

Doing this sort of transfer, I never bothered with Live CD, I’d do a preparatory data copy with rsync -Sax, then take system down to Single User mode, and update with “rsync -cvSax” for /usr, not having to worry about deletions and such; I see Carlos has mentioned “acls” to, which may be needed and could get left off a new mount as they’re an option.

Can’t see how you can get a better answer, without providing error logs of the time which failed, something obviously occured then which got fixed, so script “setperms” may have left you some email.

My 2 cents: don’t. You’d have to be wide aware (no typo) at any upgrade of your system.

On 2011-07-24 23:26, Knurpht wrote:
>
> My 2 cents: don’t. You’d have to be wide aware (no typo) at any upgrade
> of your system.

I have had a separate /usr for 10 years at least, with upgrades.


Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” at Telcontar)

On 2011-07-24 21:16, robopensuse wrote:
> As to the question, it ought not be “hard links”, they would be copied
> to the new /usr.

Mmm, hard links. I think they have to be changed to soft links.


Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” at Telcontar)

I’ve used seperate /usr for very long time, but it’s become senseless. Basically /lib/modules has become so huge, as has /boot/initrd-<kernel-version-flavour> that seperating out / & /usr just doesn’t make sense like it used to, when ‘/’ could be 16 MiB. We used to be able to boot from floppies to.

The same kind of data is stored in /lib as in /usr, and the desktop upstream developers have just assumed /usr is in /. openSUSE 12.1 plans to mount /usr from initrd, to support seperate /usr better than currently, but Kay Sievers & Lennart Poeterring are right, that seperating / & /usr does not make as much sense as it used to. If /usr does not get mounted just after / in early boot, then with time, more things will break; that’s the reason for my change of policy.

I’m having /usr in / now, and looking to bind mount /tmp into a large (emptyish) /var, because it brings more than seperate /usr. / & /usr are then read mostly data, like /boot and with directories like /run it’s becoming feasible for distro’s to support a ro / not just ro /usr.

With “zypper dup” being so heavily used by developers and testers, I’d be surprised if doing a distro update to move to new release, was the bad idea it used to be in SuSE 7 & 8 days.

No, in general it’s considered rude to hard link outside of the directory you’re in, but install relative soft-links in such cases to avoid problems on mounting seperate partitions, or from Live CD where / is /mnt, and /var /mnt/var for example. So that ought not be a problem in practice, in system partitions where ppl are supposed to know what they’re doing. I actually notice, soft links seem to be preferred by install stuff anyway, where hard links could safely be used. Have you example of system files that are hard links these days which are breaking out of /usr or /usr/share to /?

# ls -l /bin/sh /usr/bin/vi
lrwxrwxrwx 1 root root 4 Mar  2 11:43 /bin/sh -> bash
lrwxrwxrwx 1 root root 8 Jul 20 08:10 /usr/bin/vi -> /bin/vim

Hard-links ought thefore to be preserved, but if you’re telling a cp or rsync, to make a copy of a hard linked file in a different filesystem, it is going to copy the data, duplicating it. Just like ‘sparse file’ handling, copying data can cause it to expand disk block usage if you’re not careful.

That was not safe. That was a mistake.

The files were not in /usr but in /usr-old (according to what you did before).

Irrelevant.

Yes. You’ll better mount you new user partition by UUID or LABEL in /etc/fstab.

Possible and even better, IMO. But 6 GB in NOT big enough for /usr. That’s why I would not recommend doing it in your case. OK, I install a lot of stuff, but see:


Filesystem            Size  Used Avail Use% Mounted on
...
/dev/sda11            4.1G  704M  3.2G  18% /
**/dev/sda12             20G  8.8G  9.6G  48% /usr**
/dev/sda13            8.0G  236M  7.4G   4% /local
/dev/sda14            6.0G  724M  4.9G  13% /var
/dev/sda10             35G  1.2G   32G   4% /home
....

tar cvf - -C /usr . | tar xvf - -C /usr.new
  • as root of course!

cpdup](http://linux.die.net/man/1/cpdup) preserves hardlinks and softlinks. It was written by Matt Dillon, the founder of DragonFly BSD, and was the tool originally used to install the system (cpdup from Live CD to hard disk). On Linux, where it is often not available, I use tar (see my previous post) to preserve hard links and soft links.

I am afraid that the problem is indeed that /usr is mounted to late. FYI:

 fdisk -l

Disk /dev/sda: 160.0 GB, 160041885696 bytes
255 heads, 63 sectors/track, 19457 cylinders
Unit = cylinders of 16065 * 512 = 8225280 bytes

Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Schijf-ID: 0x3afc3afc

 Apparaat Opstart   Begin       Einde     Blokken   ID  Systeem
/dev/sda1               1       11075    88956469+   7  HPFS/NTFS
/dev/sda2           18731       19457     5839627+  83  Linux
/dev/sda3   *       12381       18730    51006375    f  W95 extended (LBA)
/dev/sda4           11076       12380    10482412+  83  Linux
/dev/sda5           12381       12572     1542208+  82  Linux swap
/dev/sda6           12573       15048    19888438+  83  Linux
/dev/sda7           15049       16354    10490413+  83  Linux
/dev/sda8           16355       18730    19085188+   c  W95 FAT32 (LBA)

Partitition table items are not in disk order.

cat /proc/partitions
major minor  #blocks  name

   8        0  156290904 sda
   8        1   88956469 sda1
   8        2    5839627 sda2
   8        3          1 sda3
   8        4   10482412 sda4
   8        5    1542208 sda5
   8        6   19888438 sda6
   8        7   10490413 sda7
   8        8   19085188 sda8

partition 7 is / , 6 is /home end 4 should be /usr
The modified fstab:

/dev/disk/by-id/ata-...-part5 swap  swap  defaults     0 0
/dev/disk/by-id/ata-...-part7 /  ext3 acl,user_xattr   1 1
/dev/disk/by-id/ata-...-part4 /usr  ext3   acl,user_xattr   1 2
/dev/disk/by-id/ata-...-part6 /home  ext3  acl,user_xattr   1 2
/dev/disk/by-id/ata-...-part1 /windows/C  ntfs-3g    users,gid=users,fmask=113,dmask=022,locale=en_US.UTF-8 0 0
/dev/disk/by-id/ata-...-part8 /windows/D  vfat       users,gid=users,umask=0002,utf8=true 0 0
proc                 /proc                proc       defaults              0 0
sysfs                /sys                 sysfs      noauto                0 0
debugfs              /sys/kernel/debug    debugfs    noauto                0 0
usbfs                /proc/bus/usb        usbfs      noauto                0 0
devpts               /dev/pts             devpts     mode=0620,gid=5       0 0

please try again wrote:
> robopensuse;2368106 Wrote:
>> Hard-links ought thefore to be preserved, but if you’re telling a cp or
>> rsync, to make a copy of a hard linked file in a different filesystem,
>> it is going to copy the data, duplicating it.
>
> ‘cpdup’ (http://linux.die.net/man/1/cpdup) preserves hardlinks and
> softlinks. It was written by Matt Dillon, the founder of DragonFly BSD,
> and was the tool originally used to install the system (cpdup from Live
> CD to hard disk). On Linux, where it is often not available, I use tar
> (see my previous post) to preserve hard links and soft links.

What does cpdup do that rsync doesn’t?

has a shorter manpage and less options.

Interesting don’t have cpdup installed or a man page at all. Doesn’t allow network copy without involving NFS like rsync -essh though, and the checksumming feature isn’t clean.

Network copying and updating is s killer feature of rsync, so if we were to copy /usr out of /, which I think is a bad idea now a days, then looks like “rsync -SHAXax” / “rsync -cSHAXax --delete” would be best options (and also allows copying to a clone machine over network). It’s only recent years acl’s & xattr’s have been used by applications. If I cared about efficient local copy then I’d be using dd(1) or md(4).

# find /usr -links +1 -type f -xdev -ls |  sort | more

Shows very many hard links in python directories & for locale stuff, in same directory, so that does matter. Fortunately if it’s done wrong there’s “fdupes - Identifying or deleting duplicate files” available to help undo it.

Looks the best you can do. If you give up on moving /usr directory, have you found where the space is being used in / ?
Automatic removal of old temporary files may help you somewhat, as I tried to explain in first post in this thread.

One GUI program I found handy for hunting down “fat” is kdirstat, unfortunately still a KDE3 program, though it works in KDE4 (& ought to under GNOME). It basically does a du of whole filesystem & provides a graphic visualisation to aid target selection.

The actual size you need for /, /var, /tmp & /boot is quite dependant on how you use your system and how much software you install. I find 10 GB is plenty, but I’m taking /boot, /var into seperate partitions as I would /tmp on serious install. That’s getting heavily written areas & initial boot stuff out from /. For test installs, as low as 8 GB has been fine.

Of course I have looked into the length of every folder. /usr occupies 6.2 GB out of 8.5 GB. Next is /lib with only 321 MB!
I have been looking into boot.msg and messages for the precise moment that /home is mounted in my actual system but I could not find it. I found in boot.msg:

Waiting for device /dev/disk/by-id/ata-WDC_WD1600BEVS-60RST0_WD-WXE207F75760-part7 to appear:  ok
fsck from util-linux-ng 2.17.2
[/sbin/fsck.ext3 (1) -- /] fsck.ext3 -a -C0 /dev/sda7 
/dev/sda7: clean, 249525/655776 files, 2144897/2622603 blocks
fsck succeeded. Mounting root device read-write.
Mounting root /dev/disk/by-id/ata-WDC_WD1600BEVS-60RST0_WD-WXE207F75760-part7
mount -o rw,acl,user_xattr -t ext3 /dev/disk/by-id/ata-WDC_...-part7 /root

I found also a lot of lines containing something like:

ROOTFS_BLKDEV=/dev/disk/by-id/ata-WDC_...-part7 _=/sbin/startproc DAEMON=/usr/sbin/cron 

which indicates that /usr should be loaded and supposed to be on the root partition?
Any idea were I can find the loading of /home for my actual system?

On 2011-07-25 10:56, robopensuse wrote:
> I’ve used seperate /usr for very long time, but it’s become senseless.
> Basically /lib/modules has become so huge, as has
> /boot/initrd-<kernel-version-flavour> that seperating out / & /usr just
> doesn’t make sense like it used to, when ‘/’ could be 16 MiB. We used
> to be able to boot from floppies to.

I currently have:



/dev/sdb8              21G   12G  8,7G  57% /usr
/dev/sdc6              10G  5,5G  4,6G  55% /usr/src
/dev/sdc7              13G  4,0G  8,1G  33% /usr/local
/dev/sdb9             9,1G  1,2G  7,9G  13% /opt


I would not make it the same nowdays, but…


Cheers / Saludos,

Carlos E. R.
(from 11.4 x86_64 “Celadon” at Telcontar)