Problem cloning (dd) TW partitions in extended Win partition

suse_rasputin · June 27, 2021, 3:19pm

Hello again!

I have 1TB HDD with Win 7 and TW (installed in an extended Win partition, running fine as grub dual boot for years):

Disk /dev/sdf: 931.51 GiB, 1000204886016 bytes, 1953525168 sectors
Disk model: JFCX-68N6GN0    
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0xd5f6c85d

Device     Boot      Start        End    Sectors   Size Id Type
/dev/sdf1             2048     206847     204800   100M  7 HPFS/NTFS/exFAT
/dev/sdf2  *        206848  184319999  184113152  87.8G  7 HPFS/NTFS/exFAT
/dev/sdf3        184320000  225279999   40960000  19.5G  7 HPFS/NTFS/exFAT
/dev/sdf4        225280000 1953521663 1728241664 824.1G  f W95 Ext'd (LBA)
/dev/sdf5        225282048 1789681663 1564399616   746G  7 HPFS/NTFS/exFAT
/dev/sdf6       1789683712 1793898495    4214784     2G 82 Linux swap / Solaris
/dev/sdf7       1793900544 1835843583   41943040    20G 83 Linux
/dev/sdf8       1835845632 1953503231  117657600  56.1G 83 Linux

The 824 GB sdf4 is the extendend Win partition, containing at first a large Win 7 data partition (sf5), followed by swap (sdf6), / of TW (sdf7) and /home of TW (sdf8)

I tried to dd the whole HDD, but inside the (not really relevant) sdf5 data partition there is a read error that stopps dd.

So I cloned the partition layout with Yast Partitioner to another HDD of same size and make.

Disk /dev/sdg: 931.51 GiB, 1000204886016 bytes, 1953525168 sectors
Disk model: JFCX-68N6GN0    
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0x6405976c

Device     Boot      Start        End    Sectors   Size Id Type
/dev/sdg1             2048     206847     204800   100M  7 HPFS/NTFS/exFAT
/dev/sdg2           206848  184319999  184113152  87.8G  7 HPFS/NTFS/exFAT
/dev/sdg3        184320000  225279999   40960000  19.5G  7 HPFS/NTFS/exFAT
/dev/sdg4        225280000 1953521663 1728241664 824.1G  f W95 Ext'd (LBA)
/dev/sdg5        225282048 1789681663 1564399616   746G  7 HPFS/NTFS/exFAT
/dev/sdg6       1789683712 1793898495    4214784     2G 82 Linux swap / Solaris
/dev/sdg7       1793900544 1835843583   41943040    20G 83 Linux
/dev/sdg8       1835845632 1953503231  117657600  56.1G 83 Linux

and started dd-ing over sdf1, sdf2, sdf3, sdf4, (not! sdf5 with the read error) and sdf6. Afterwards the output of fdisk -l looks like:

Disk /dev/sdg: 931.51 GiB, 1000204886016 bytes, 1953525168 sectors
Disk model: JFCX-68N6GN0    
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0x6405976c

Device     Boot      Start        End    Sectors   Size Id Type
/dev/sdg1             2048     206847     204800   100M  7 HPFS/NTFS/exFAT
/dev/sdg2           206848  184319999  184113152  87.8G  7 HPFS/NTFS/exFAT
/dev/sdg3        184320000  225279999   40960000  19.5G  7 HPFS/NTFS/exFAT
/dev/sdg4        225280000 1953521663 1728241664 824.1G  f W95 Ext'd (LBA)
/dev/sdg5        225282048 1789681663 1564399616   746G  7 HPFS/NTFS/exFAT
/dev/sdg6       1789683712 1793898495    4214784     2G 82 Linux swap / Solaris

So apparently 2 partition on sdf are gone after copying the swap with dd.

If I dd sdf7 and sdf8 then the whole thing won’t change, these partitions won’t appear afterwards and the HDD doesn’t boot.

I use SATA-USB3 adapters for this whole dd thing, might this be the problem? I tried 2 different makes

Bus 007 Device 003: ID 174c:55aa ASMedia Technology Inc. ASM1051E SATA 6Gb/s bridge, ASM1053E SATA 6Gb/s bridge, ASM1153 SATA 3Gb/s bridge, ASM1153E SATA 6Gb/s bridge

and

Bus 007 Device 006: ID 152d:0578 JMicron Technology Corp. / JMicron USA Technology Corp. JMS578 SATA 6Gb/s

Any ideas if it’s feasible to copy over such a HDD setup and get functional grub dual boot?

OrsoBruno · June 27, 2021, 4:05pm

I guess that what you see is linked with the very structure of the “Extended partition”, where each logical partition points to the beginning of the next one where an Extended Boot Record is to be found ( see here ).
Maybe with your dd you overwrote one of those links and now the system doesn’t know where to look for the last two EBRs and doesn’t “see” the last two logical partitions that should still be there.

suse_rasputin · June 27, 2021, 4:24pm

Hi!

Thanks for replying!

I tried again and noticed that sdf7 and sdf8 disappear already after dd-ing sdf4, shich contains nearly no info

sudo dd bs=4M if=/dev/sdf4 of=/dev/sdg4 status=progress conv=fsync  
[sudo] password for root: 
0+1 records in
0+1 records out
1024 bytes (1.0 kB, 1.0 KiB) copied, 0.504114 s, 2.0 kB/s

Afterwards I get:

Disk /dev/sdg: 931.51 GiB, 1000204886016 bytes, 1953525168 sectors
Disk model: JFCX-68N6GN0    
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0x27249890

Device     Boot      Start        End    Sectors   Size Id Type
/dev/sdg1             2048     206847     204800   100M  7 HPFS/NTFS/exFAT
/dev/sdg2           206848  184319999  184113152  87.8G  7 HPFS/NTFS/exFAT
/dev/sdg3        184320000  225279999   40960000  19.5G  7 HPFS/NTFS/exFAT
/dev/sdg4        225280000 1953521663 1728241664 824.1G  f W95 Ext'd (LBA)
/dev/sdg5        225282048 1789681663 1564399616   746G  7 HPFS/NTFS/exFAT
/dev/sdg6       1789683712 1793898495    4214784     2G 82 Linux swap / Solaris

sigh

No ideas left here. I though about my old Win 7 tool for cloning disks (old Paragon Partition Manager), but I guess it will fail with the damaged block in the data partition…

OrsoBruno · June 27, 2021, 4:42pm

I guess that you can still recreate sdg7 and sdg8 with the partition manager of your choice after having cloned the other logical partitions and then clone them as well.
It looks as a slight misalignement between the two disks causes the 512 bytes with the EBR of sdg7 to be overwritten, maybe only partially, and the system doesn’t know where to look for sdg7 (and then sdg8, impossible to find if you cannot see sdg7).

caf4926 · June 27, 2021, 5:13pm

I usually dd the entire drive to a .iso and save it to a backup device
Then dd write the .iso to my new drive
Never fails

Use ionice ddrescue if the source is a bit dodgy

suse_rasputin · June 27, 2021, 5:15pm

I try right now to only dd sdf1, 2, 3, 7 and 8 and see how this ends…

Is there anything more than the partition info in sdf4?!?

Next try would be to have both HDDs connected via native SATA ports to see if this helps…

hcvv · June 27, 2021, 5:21pm

It is nonsense to copy the Extended partition. It is has no other contents then the pointer to the first Logical Partition.

The way to go (when you want to copy partition by partition) is:

Create the same partitions (table) on the new disk as was on the old disk;
Copy one by one the Primary and Logical partitions. Do not copy the Extended Partitions, it is already correct.

suse_rasputin · June 27, 2021, 8:37pm

After the procedure described above, I have only a black screen with a blinking cursor in the left upper corner.

What’s next?

mrmazda · June 27, 2021, 9:03pm

What hcvv wrote, which is entirely correct (except for mispelling), doesn’t copy important sectors containing bootloader code. The bootloader must be installed by conventional method unless you have cloned the entire disk sector by sector as dd would do.

You should be able to install the bootloader by using rescue media to boot the installed system, or by chrooting into the installed system.

The extended “partition” isn’t really a partition. It’s simply a placeholder table entry that points, as hcvv wrote, to the first logical partition, whose own EBR does nothing but point to the next logical partition, etc. It’s the chain of placeholder entry plus EBR entries that make up what collectively comprise an extended “partition”. No other “partition” has sectors scattered among various places separated by filesystems on their own individual partitions on the media.

Sensible partitioning tools manage the extended partition automatically, making it unnecessary to take it into account other than via the designation of partitions it necessarily “contains” as “logical”. Those that make extended partition management discrete step(s) serve primarily to confuse anyone but partition experts.

suse_rasputin · June 27, 2021, 10:14pm

I followed this instruction in the second post of this thread:

https://forums.opensuse.org/showthread.php/496983-grub2-drops-to-quot-grub-rescue-quot-with-quot-unknown-filesystem-quot-error

but now the bootloader is in sdX1, not in sdX2, as in the original disk. And the system boots only to “Reached target: Basic System” and times out after some minutes with errors on the UUID of the root (iirc) partition, although the UUIDs on both HDDs (original and clone) are identical, according to blkid…

/dev/sdf1: LABEL="System-reserviert" BLOCK_SIZE="512" UUID="16581E8F581E6E2D" TYPE="ntfs" PARTUUID="8d2a7d1a-01"
/dev/sdf2: LABEL="Donald" BLOCK_SIZE="512" UUID="5AD0272BD0270CB7" TYPE="ntfs" PARTUUID="8d2a7d1a-02"
/dev/sdf3: LABEL="Daisy" BLOCK_SIZE="512" UUID="765AA6485AA604C9" TYPE="ntfs" PARTUUID="8d2a7d1a-03"
/dev/sdf7: UUID="ad83ae2c-5882-483d-ae85-6e0c7a58655a" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="8d2a7d1a-07"
/dev/sdf8: UUID="9c39d5ee-aa47-4034-94b2-76ba518b3a0c" BLOCK_SIZE="4096" TYPE="xfs" PARTUUID="8d2a7d1a-08"
/dev/sda1: PARTUUID="ce842ab1-41aa-4ef3-ac93-2ed83dcc183d"
/dev/sdf5: PARTUUID="8d2a7d1a-05"
/dev/sdf6: PARTUUID="8d2a7d1a-06"

/dev/sdg1: LABEL="System-reserviert" BLOCK_SIZE="512" UUID="16581E8F581E6E2D" TYPE="ntfs" PARTUUID="d5f6c85d-01"
/dev/sdg2: LABEL="Donald" BLOCK_SIZE="512" UUID="5AD0272BD0270CB7" TYPE="ntfs" PARTUUID="d5f6c85d-02"
/dev/sdg3: LABEL="Daisy" BLOCK_SIZE="512" UUID="765AA6485AA604C9" TYPE="ntfs" PARTUUID="d5f6c85d-03"
/dev/sdg5: LABEL="Goofy" BLOCK_SIZE="512" UUID="E0CEBA1CCEB9EAC4" TYPE="ntfs" PARTUUID="d5f6c85d-05"
/dev/sdg6: UUID="c4f07dd9-ccef-4a1f-aed1-b00dafa4f0f3" TYPE="swap" PARTUUID="d5f6c85d-06"
/dev/sdg7: UUID="ad83ae2c-5882-483d-ae85-6e0c7a58655a" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="d5f6c85d-07"
/dev/sdg8: UUID="9c39d5ee-aa47-4034-94b2-76ba518b3a0c" BLOCK_SIZE="4096" TYPE="xfs" PARTUUID="d5f6c85d-08"

For two days now I’m trying to fix this mess. Why is it that complicated to simply clone a dual boot HDD?

caf4926 · June 27, 2021, 10:17pm

Nevermind partitions just copy everything
eg

dd if=/dev/sdf of=/image.iso

Obviously your instructions might vary for the target or for additional instructions

If you really struggle getting it booting again. See if you can grab a bit of space using a Live distro of any flavour you like and install it temp
Recover to your install and run grub edits
Reboot to test
Remove the temp install

mrmazda · June 27, 2021, 10:32pm

Use ddrescue instead.

mrmazda · June 27, 2021, 10:41pm

Are you trying to do anything post-cloning without:

first shutting down the PC and removing either the original or the clone before restarting, or
*]immediately making all UUIDs and LABELs on one or the other of original and clone unique, then conforming all files dependent on UUIDs and/or LABELS to the changes
???

suse_rasputin · June 28, 2021, 8:09am

The cloning is with HDDs (original + target) attached to a different TW machine. Booting is on a different machine.

Why would UUIDs need to be unique? If they are identical (orginal and clone) anything should work fine. On embedded linux devices I clone complete installs (usually on SD-cards) via dd with identical UUIDs all over the place and that works just fine.

suse_rasputin · June 28, 2021, 8:11am

caf4926:

Nevermind partitions just copy everything
eg
dd if=/dev/sdf of=/image.iso
Obviously your instructions might vary for the target or for additional instructions

If you really struggle getting it booting again. See if you can grab a bit of space using a Live distro of any flavour you like and install it temp
Recover to your install and run grub edits
Reboot to test
Remove the temp install

The dd stops due to a read error in the data partition (described in first post), therefore I have to go the partition-by-partition way…

karlmistelberger · June 28, 2021, 8:33am

As sdf5 is irrelevant have dd ignore the read error:

**3400G:~ #** dd if=/dev/sdd of=/dev/null conv=noerror 
^C68513+0 records in 
68512+0 records out 
35078144 bytes (35 MB, 33 MiB) copied, 3.48656 s, 10.1 MB/s 

**3400G:~ #**

caf4926 · June 28, 2021, 8:49am

I did suggest that earlier

It might go

ionice -c3 ddrescue /dev/sdf  /*target-location/MyHDDBackup.iso

caf4926 · June 28, 2021, 8:50am

I would add
I always work from a live system copying and then writing

suse_rasputin · June 28, 2021, 9:35am

I skipped dd-ing the swap partition, but as I see now, the UUID is not the same as for the source disk (see post above). As swap is not in fstab I didn’t consider this to be of relevance.

Is it necessary to dd the swap partition?

ddrescue sounds good in principle. Will use it for next try…

suse_rasputin · June 28, 2021, 9:56am

I do dd images from SD-cards on the TW I used on regular basis without any problems. Why would a live system be of any advantage?