Cloning an old drive to a new drive

English Forums Feedback
1

This is a basic guide/example for cloning an old small hard EIDE (PATA) internal drive to a new larger EIDE (PATA) internal hard drive, using the linux “dd” command where both hard drives are 255 heads, 63 sectors/track.

Edit: The reviewed how-to has been copied to here:
Cloning an old drive to a new drive - openSUSE Forums
any/all future edits will be made on the Linked how-to, and not on this post. This thread is being maintained for comments, suggestions, and discussion of corrections for the “reviewed” how-to.

**OVERVIEW: **

In the example that I did in January 2009, the old drive was 40GB and the new drive was 160GB. This was done on an old Dell Dimension 2100 PC (1.1 GHz CPU with 512MB of RAM and the latest A4 BIOS) that was purchased in 2001. After a successful cloning, the new drive will have only 40 GB of formatted bits and bytes, but it can be easily expanded to use the fully 160 GB with the appropriate utility.

In the case where I did this operation, I used an
(a) openSUSE-11.1 live CD
(b) parted Magic Live CD
(c) external 60 GB USB hard drive (for data backup).

**RESEARCH: **

Before purchasing the external hard drive, try to ensure that it is compatible with your PC (via research on the Internet). This is especially true if one intends to use the hard drive on an older PC. This guide assumes both old and new drives are 255 heads, 63 sectors/track, so attempt to confirm that prior to purchasing the new drive.

Also before starting, check the web site of your motherboard manufacturer to see if there is a BIOS update. Sometimes there are BIOS fixes associated with new drives (and newer drive technology).

If possible, it is useful to ensure the new hard hard drive can be returned to the computer store, if the cloning does not work. However finding a computer store that will allow that return policy is not so easy nor common.

** BACKUP FIRST **

Before cloning, it is best to backup all data. External USB drives are inexpensive, and they make great platforms for external backed up data.

** CONNECT BOTH OLD AND NEW DRIVE**

After backing up the data, switch OFF the PC, and leave the old (smaller) hard drive connected. Connect the new hard drive to the same cable as the old hard drive, but have the jumpers on the new hard drive set as “slave”. Now switch ON the PC and boot to the BIOS, and confirm that that BIOS recognizes both the old hard drive, and the new hard drive.

In the example I did (with the old Dell Dimension 2100) the BIOS recognized both old and new drives, although the Dell recognized the new 160GB drive (a Seagate Barracuda) as a 137,447 MB drive. The fact the drive is recognized was positive, although it worried me that the 160 GB drive was recognized as a 137 GB, as opposed to 160 GB. I read on our forum that this was due to a 142 GB BIOS limit. Purportedly (from what I was told) that only means that any GRUB files should be in partitions under this limit. Linux has no problems reading the whole disk, since it doesn’t rely on the BIOS to access the disk.

Put the openSUSE Live CD in the CD drive, and with the BIOS set to boot to CD ROM first, continue booting to the openSUSE live CD. Once boot is complete, open a gnome terminal (for Gnome live CD) or KDE konsole (for KDE live CD) and type “su” (no quotes, enter root password when prompted) followed by fdisk -lto check openSUSE can see both old hard drive and new hard drive. Once may see something like this for the old hard drive:

fdisk -l

Disk /dev/hda: 40.0 GB, 40000000000 bytes
255 heads, 63 sectors/track, 4863 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/hda1 * 1 959 7703136 b W95 FAT32
/dev/hda2 960 2043 8707230 7 HPFS/NTFS
/dev/hda3 2943 4863 15430432+ f W95 Ext’d (LBA)
/dev/hda4 2044 2942 7221217+ 83 Linux
/dev/hda5 2943 4033 8763426 b W95 FAT32
/dev/hda6 4034 4103 562243+ 82 Linux swap / Solaris
/dev/hda7 4104 4863 6104668+ 83 Linuxand something like this for the new hard drive: Disk /dev/hdb: 160.0 GB, 160041885696 bytes
255 heads, 63 sectors/track, 19457 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

 Disk /dev/hdb doesn't contain a valid partition table

So clearly here in the above example, hda was the old drive and hdb is the new drive. In this example, hda1 was a winME partition, hda2 is a winXP partition, hda5 is a fat32 data partition, and hda4, 6, 7 were all old openSUSE-10.2 Linux partitions (for /, swap, /home respectively). Note that hdb “doesn’t contain a valid partition table” because it is brand new, and never was partitioned before. Note also that both drives were 255 heads and 63 sectors/track. That is important, and provides confidence that the cloning will work.

** CLONE **

Then, in the same terminal / konsole (with root permissions) type:dd if=/dev/hda of=/dev/hdb bs=32256BE VERY CAREFUL when typing that. If you get that wrong, you could erase your existing hard drive !! THAT LINE IS KEY TO THE CLONING. Note in this example, hda is the original hard drive, and hdb is the new hard drive. Dependant on how the live CD recognizes your drives, this may instead be different. For example if the old drive was sda and the new sdb, this could instead be: **dd if=/dev/sda of=/dev/sdb bs=32256 ** I have read that bs=32256 represents one complete track of 63 sectors, each with 512 bytes (I also read that omitting bs=32256 purportedly still works, but that would significantly slow down dd cloning).

In the case of cloning a 40 GB drive, that could take about 2 hours for an old PC (time dependant on the PC age). There will be no messages during the cloning.

** PREPARE FOR FIRST BOOT **

Once the cloning is complete, switch OFF the PC, remove the old drive (40 GB drive in this example) and move the new drive (160GB in this example) to the same place on the cable as the old drive, and ensure that you change the jumpers on the new drive to make it the primary drive. This is important. Do NOT power on with new drive still setup as a slave.

** FIRST BOOT **

Then switch ON the PC. It should boot properly to Grub, which was cloned as part of the cloning process. In my example case, I selected to boot to openSUSE-10.2, and the PC booted properly to 10.2 (hda4). I then rebooted to winME (hda1) properly as a test. I then rebooted to winXP (hda2) as a test. WinXP gave an error the first time, and refused to boot properly. I’m told this is typical winXP behaviour when it notes a new drive ID in the place of the old drive. I then rebooted winXP (hda2) and on the second boot winXP booted properly and functioned properly.

** RECLAIM FULL SPACE OF NEW DRIVE **

I then inserted the Parted Magic live CD (this would work just as well with the gparted live CD) and repartitioned the new hard drive. There are many ways to go from this point, but the purpose here by booting to Parted Magic is to expand the 160 GB drive (which is currently only using 40 GB) to use its full capacity.

At this point, one is complete. (In my case, I “shelved” the old 40GB drive, and did not use it any more).

** EXAMPLE OF ADDITIONAL THINGS **

Also in my case I decided to go a step further to completely remove openSUSE-10.2 and replace it with openSUSE-11.1, so I removed all partitions except for the hda1 (winME) and hda2 (winXP). By doing that the drive would no longer boot, as the MBR had the old openSUSE-10.2 grub pointing to /hda4 (where the boot files on /hda4 had been removed by me). With parted Magic I expanded the /hda2 NTFS partition to be larger (for winXP), I created a new /hda3 (fat32 data partition) and created a new /hda4 extended partition where upon I created an hda5 swap partition, a hda6 / (root) partition area and a hda7 /home partition area for an openSUSE-11.1 install. I then rebooted from the Parted Magic live CD to the openSUSE-11.1 live CD, and during the openSUSE-11.1 live CD install I was careful to ensure openSUSE-11.1 installed on the /hda5 (swap), /hda6 (root) and /hda7 /home, and careful to ensure openSUSE-11.1 placed grub on the MBR (it was not going to do this by default, and I had to edit the install configuration).

The openSUSE-11.1 was successful, and after a successful openSUSE-11.1 install, the new grub then gave me the choice upon booting to go to winME, winXP, openSUSE-11.1 or openSUSE-11.1 safe.

Overall, the operation ran quite smooth.

For the example I gave above, the following is the thread where I sought help from our forum experts for conducing the cloning operation: Looking for new hard drive advice - openSUSE Forums

2

Good stuff.

3

Well done oldcpu. I see the OpenSUSE world is benefiting from your filial IT support. :slight_smile:

4

I understand that for drives of different geometry it is possible to clone by partition, for example,dd if=/dev/hda1 of=/dev/hdb3. This will copy partition hda1 to partition hdb3.

Pros: After copy you can resize the new partition before copying the next one, if any. This may be very desirable, as after a full drive dump (copy) you’re only be able to resize the last partition.

Cons: The boot sector(s) is(are) not copied, it has to be done manually. dd can be used for this, or another partitioner. Also grub or lilo may have to be edited if partitions are copied out of order, as in the example above.

5

Further to my above post, … to say that I did a large amount of research into dd before applying it for a specific purpose of cloning a “healthy” 40 GByte drive image on to a new “healthy” 160 GB drive, would be an understatement. I did a LOT of research.

For those users who wish to follow any debate that may follow on this thread, and I think such a debate would be VERY HEALTHY, then I recommend users first bring themselves up to speed technically. By being up to speed technically, it will make any technical debate much more useful in coming to the truth as to the utility of the how-to.

I recommend a good place to start is to read this very large thread quoted here, from start to finish:
How to migrate XP, Vista, Linux, BSD and Solaris to a bigger hard disk - JustLinux Forums

And finally a plea to all. No insults. Just the specific technical facts, without generalisations. I am “guilty” of complaining about FUD, although if one reads the thread I quote, and then read also this initial thread Looking for new hard drive advice - openSUSE Forums about the warnings I got saying it would never work (until I examined each one in detail, and figured out why the warnings were not applicable if things were done properly) then one might understand why I say there is a lot of FUD.

6

As I have been quoted. In the thread, the guy with the flakey disk wants to go from XFS to ext3, and dd does not provide a convenient interface to copy files. Cloning also will not result in the cleanest disk layout, whereas filling a newly built filesystem would.

That said, there’s nothing wrong with cloning a drive. It’s a perfectly reasonable and efficient technique, if you understand the PC Partitioning concepts.

The point I would make about the HOWTO is that disk geometry & track sizes are not very relevant and that personally, I’d use some small multiple of 4KiB block sizes eg) 32768 last time I did some tests, large blocks didn’t improve performance.

7

That makes me wonder about the whole importance of the original point.

8

An LBA disk is just effectively a big array of blocks, with no guarantee about geometry.

A Linux block device, gets cached by the OS, unless you take special steps to use a raw device.

So, when you copy… I think we are just doing a serial read on a very big file. readadhead should kick in and read the old drive as fast as poss. The output drive is larger, and very likely faster, so most likely we are limitted by read speed, with Linux doing a whole load of buffering in the block cache.

Doesn’t this defeat any attempt to do a track by track copy? The OS and drive electronics, should effectively do the same (even drives have large buffers these days).

If this is right, do we gain anything by talking about geometry? Isn’t the main thing that after cloning, you need to find & use the unused space?

9

In fact in modern drives, the track, head, and sector numbers used to interface to the OS in CHS addressing are just fictions that are materialised by the onboard controller. Internally CHS will map to however the controller wants it. There are in fact spare sectors that are automatically mapped in when the controller detects a bad block (with attendent cost in seeking to it). That’s why CHS has no correspondence to geometric reality these days and LBA is the easier way to address a block.

I can only think of floppies and DVD-RAMs where there tracks and sectors have geometric reality. DVDs and CDs just have one long spiral track.

10

On the new drive possibly having bad sectors it would be possible to suggest using badblocks(8) as an optional procedure to aid reliability.

That’s not going to be enough for a critical system though. Suitable ‘burn in’ tests belong to a different HOWTO though.

We could suggest mounting the clone partitions under /mnt, and using rsync -nca to verify the copy.

11

I noted in another thread on this forum, a user attempted to clone with dd and made two fundamental mistakes (in what is to me a common sense area), indicating they did not read the links I provided, and they ignored the research recommendation provided. < sigh >

Accordingly, I added some very specific technical common sense cautions and explanations to the “reviewed” how to. Note the edits I made were very technically specific, and are not wide open generalities. (Wide open generalities are mostly not helpful). Further review of those added sections are welcome. Please, when it comes to making changes, I really need helpful specific suggestions of edits to the text, and not non-technical generalities.

Its easy to see what has been added by looking at the initial (unedited) guide on this page, against the edited guide in the how-to. There is also a brief change tracking summary at the bottom of the reviewed how-to guide.

There are some excellent comments on cloning geometry in this thread, but I don’t know enough to succinctly and accurately convert those comments to specific edits, … so edit suggests there are also welcome.

12

I’m trying to understand this. So after the cloning is complete, with the original drive still hda and the new cloned drive hdb, while the PC is still powered, one would mount the new cloned drive by creating a directory /mnt/new (ie su -c ‘mkdir /mnt/new’ ) and mount the new drive with:
mount /dev/hdb /mnt/newfollowed by
rsync -nca /dev/hda /mnt/newto verify the cloning?

Do I have that correct?

13

I’ve never used badblocks before. I just finished typing ‘man badblocks’ and was pleasantly surprised to find a succinct description of its use. I need to research this and educate myself a bit more before I can propose anything specific here.

Thank you for the recommendation.

14

Further to this, I note the man page for badblocks recommends instead use of the -c option of the e2fsck and mke2fs programs.

Checking into e2fsck and cloning, I noted this old fedora thread (see the 2005-07-11 08:57 post): Clone Hd Linux Fedora Core 3 (raw) [Archive] - FedoraForum.org where the recommendation given that one may (if they wish) reduce “warts” (so to speak) prior to cloning, and to do this by first boot to the old drive, unmount the nominal linux drives, then with them unmounted run “debugfs” and “e2fsck” with some rather specific options, and then clone.

But having typed that, I’ve also read else where, that one can successful clone a drive, “warts and all” and then after the cloning run such utilities to clean up. I have proven that defragmentation can be successfully conducted after cloning.

I am assuming then that running programs such as badblocks or e2fsck (prior to cloning) is to identify (and possibly ensure) there are no critical badblocks that would cause the cloning process to fail …

15

Modern drives conceal bad blocks by internal remapping to spare sectors. By the time these bad blocks can be seen by the badblocks program, there are probably too many errors for the disk to be worth using. So I have my doubts whether badblocks is worth running. Still I don’t have enough experience to categorically say that no purpose is served by badblocks. But I would recommend a SMART self-test first and then not using the disk if the error stats look bad.

16

Thats a good point. One fundamental assumption I made was the old hard drive was still reasonably healthy before the cloning. If the old hard drive is bad, then I can not recommend any approach, due to a lack of experience on my part.

I also know very very little about using smartctl disk monitoring. I note this article:
Monitoring Hard Disks with SMART
…
presumeably then, prior to cloning, one could run this command smartctl -a /dev/hda, using the correct path to the old disk, as root. If SMART is not enabled on the disk, one must enable it with the -s on option.

Then one could type (to confirm the disk is in the smartctl database):
smartctl -i /dev/hda
And then do a health status inquiry with: smartctl -Hc /dev/hda… but I don’t know if I have that correct.

17

You can look at the stored stats with smartctl, and also initiate self-tests of various lengths from smartctl. I think the number of spare sectors used is in one of the counters.

18

Hi oldcpu,

I’ve had a look at your howto and I would recommend one modification to the dd command.

You have:

dd if=/dev/sda of=/dev/sdb bs=32256

I would recommend changing that to:

dd if=/dev/sda of=/dev/sdb bs=32256 conv=sync,noerror

The reason is this: If the input disk has errors, you would like to recover as much as possible from the disk. However if you allow errors to cause chunks of data to be omitted, that will throw off all the references to blocks that are after the bad block. So what we do is ask dd to fill up the rest of a partial read due to read errors with NULs, and to continue in the face of errors. That way, the output file is always the same size as the input file, even if there are blocks patched with NULs.

Now the obvious question is, can we reduce the amount of loss to the minimum? If say the middle of the 31.5k chunk is bad, we lose the second half, even though there may be only one bad 512 byte block. You can if you reduce the block size to 512, the minimum. But that will cost copying speed.

However there is a program that tries to get good speed with minimal loss, and that is ddrescue. The package is ddrescue and the program is dd_rescue. It will copy in chunks of 64k but fall back to 512 bytes when an error is hit. The option to always write a block is -A. It’s so new that there isn’t even a man page but dd_rescue --help will get you a summary. Here’s a web page about it:

dd_rescue

Even though it has rescue in the name, it’s worth using even when there is nothing wrong with the input and output disks.

19

ddrescue is interesting. Back in late December-2008, early January-2009, I briefly looked at it, but I decided I did not know enough to employ its use. I had researched dd extensively, and had read of many successes with “dd” (with the correct approach being adopted) but had not find many examples of ddrescue, so in the end I went with dd.

I agree it would be useful to learn more about ddrescue.

20

Thanks. I applied your edit.

I have not tried that additional “argument” to the dd command, so I’m hoping the syntax is ok.