In my way to start learning dd in order to get rid of USB writing tools like Rufus, UNetbootin, etc (possibly even of CD/DVD burning software as well?), I found in several articles and forums around internet that dd is kind of outdated nowadays. And seemingly more “confirmed” by this Stackexchange question and answer.
Initially I wanted to learn dd for all general use: create bootable USB/SDcard from OpenSUSE, Debian, Raspberry Pi, any other distro live/installer ISOs; clone hard disks or partitions like in Norton Ghost or Clonezilla (specially for usage with virtual machine conversions), rip ISO image from a physical CD/DVD, etc.
So, can all of this be actually done with cat, cp or pv nowadays? Even without somehow specifying “buffer size” like in dd?
dd is best for making a byte to byte exact copy of a file and remember that “everything” is a file in Unix/Linux. Thus dd can also copy whole partitions or mass-storage devices (by using the device file related to it like /dev/sde) and is mostly used for these sort of actions.
Calling something outdated is nonsense. It works for already more then 40 years. Stable as a rock.
cp is more for copying on the file level then on the device level, but there is considarable overlap.
cat is in fact created for showing text files on the terminal (CLI before it was called CLI). But again, as long as you do not realy try to dump with cat on a terminal, but redirect or pipe it elsewhere, it will handle all bytes.
I’m no specialist of the internals of the commands you are looking at, but if you read carefully the reference you provided you’ll find a good explanation of why “dd” with its several options might be an invaluable tool when dealing with byte or physical media manipulation.
The short story is that “dd” can operate on any sequence of bytes stored on a device and that saved my day a few times when trying to copy/rescue/correct records (including boot sectors) damaged by other tools or accidents.
With that in mind, be careful since “dd” even copies byte errors if you have a damaged file or sector on a disk, without any attempt at “repairing”; but then it allows you to change that single byte if you want to, a thing no other tool can do.
If you are only interested in copying files from a directory to another, “cp” may be easier to use and for “looking” at the content of a file “cat” is a handy choice. As Henk suggests, there may be considerable overlap between those tools.
All of them are of about the same age, the beginnings of *nix, and not “outdated”.
And, in my opinion they are subject of the old adagium that goes about:
You can do something in *nix in probably in 100 different ways, of which about 80 are sound (some exaggeration here).
May not sound encouraging, but it is just experience and thus reading man pages, using the tools, stand-alone and in scripts, again and again, that will grow a habit for you to use instinctively the correct one for the case on hand.
I find it very difficult to believe that anything could be significantly faster than dd, done correctly a dd copy can approach nearly instantaneous no matter the size of whatever is being copied.
The drawback to using dd which is not mentioned in your reference is that there is no integrity check, but that would probably be true of anything that does byte level copying.
Also, “outdated” in Unix usually only means out of fashion, not that the command doesn’t or can’t do its task well.
Thanks very much everyone for your advise. Even during confusion I kind of felt that dd was still the “orthodox” way; you helped confirming with arguments.
I was also wondering if I could use dd to rip ISO images from physical CDs/DVDs, and found this page.
Certainly I didn’t expect this: it seems that, if using dd with just the default values, dd will add some couple of blocks for free to the output ISO file, thus checksum will not match the original CD/DVD source? Will this actually make the output ISO unbootable or uninstallable?
And, even if “correctly” ripping ISO image with dd, is there difference in the process between dd itself and common burning software such as K3b and Brasero?
Regarding dd parameters, well, as already implied in the thread here, one could just use all dd defaults and it will work, but by specifying correct parameters it will work just better. For the “bs” parameter, how can one tell the best optimal value for each case? A search in internet also threw the link posted in OP as part of the many results, implying with a “benchmark” that it’s not very relevant nowadays and just use one preferred value in general?
Finally, if trying to write openSUSE Leap installer DVD ISO into a USB stick, will USB be able to boot either BIOS and UEFI, or only one of them and I must specify it somehow during dd process?
It is also my impression that blocksize isn’t that important anymore (for performance). Todays revolving disks have firmware that probably produce another blocksize then the real one on the disk.
A very rare usage I know of is when you want to copy the Master Boot record, which is the first 440 bytes of the first block of the disk. Thus bs=440 and count=1 will address that without touuching the partition table taht comes after those 440 bytes.
If you dd the iso file to the /dev/sdx of the USB stick you do a byte for byte copy of the iso image, which is structured to boot both in UEFI and BIOS mode; the same might not be true if you “burn” the image with other Win* tools that try to guess the special structure of openSUSE images, often doing a mess of the few initial bytes.
[Sorry the other questions are too much for me to answer on the fly ]
As I understand it, adding a couple of zero blocks at the end of the image will not prevent booting or installing the copy made from that image; after all, no USB stick exactly matches the size of a disk, so there is always plenty of zero blocks at the end of a stick anyway.
I never recall using K3b and seldom use Brasero these days, but when I had to duplicate a physical disk I did just that (insert the master, copy, insert the blank DVDs and make copies) without any attempt at “ripping”.
For hassle-free burning to USB sticks I use the SUSE Studio Imagewriter openSUSE Software which is guaranteed not to mess with the hybrid openSUSE images BTW.
“Ripping” might mean a few different things… likely extracting file data.
If that’s your intention, I doubt that dd can be your preferred tool, at least without more work than makes it worthwhile to try.
If you simply want to create an ISO file from your DVD,
The following is a recommended procedure from the Arch Wiki
Actually block size is supremely important today for performance (and management).
Traditionally, 256 bytes was the standard block size for practically everything… data storage, transfer, etc.
When gigabyte (and now today’s terabyte) drives started appearing the Advanced Format (4k) block size was proposed and largely implemented everywhere (I’ve never verified whether it was formally standardized). Of course the larger block size reduces overhead and enables mapping large amounts of data with far fewer pointers. But, it can also result in mis-alignment.
if creating an ISO file from physical CD/DVD by using dd with no parameters -i.e., all defaults-, will I actually get an ISO file of full 700 Mb size (or 4.7 Gb for DVD) regardless of how much data physical CD actually has stored?
@tsu2:
then, for whatever activity I need to do with dd (writing to some disk, disk/partition cloning, etc), is there a way to deduce the optimal “correct” values for “bs” and “count” parameters?
How can one tell?
to see how many bytes you get when reading it. And I’m pretty sure that’s what “dd” will copy to the generated file.
then, for whatever activity I need to do with dd (writing to some disk, disk/partition cloning, etc), is there a way to deduce the optimal “correct” values for “bs” and “count” parameters?
You primarily need the “bs” and “count” parameters when you are trying to explicitly control how much data you are copying. Apart from that, the use of “bs” is mostly about optimizing performance, and that depends on your hardware.
Originally dd was created for working with tapes where block size mattered - it was exactly the underlying device block size. Using wrong block size could return incomplete data (because every IO would have fetched new physical tape block and if you requested smaller size, the rest of block was discarded). dd was also used to exchange data between Unix and other systems using fixed-size records (here is where conv=block/unblock options originate from initially) where block size had to match record size.
Linux does not really have character disk devices in the old Unix sense, so here block sizes are hidden by driver and it does not really matter indeed. It still matters for tapes though.
That dpeneds of course on what is on that CD/DVD. There should be an ISO file system on it to start with.
And then it depends on what you tell dd to do. You want to stickuse no parameters and stick to the defaults, but the defaults for if is stdin and for of is stdout. Thus the defaults will not work on any CD/DVD.
I meant, if creating an ISO file from physical CD/DVD by using dd with no parameters --i.e., all defaults except input and output files of course–, will I actually get an ISO file of full 700 Mb size (or 4.7 Gb for DVD) regardless of how much data physical CD actually has stored?
And, for writing the openSUSE DVD ISO into USB stick, does it not need bs nor count parameters?
There is no “of course” when talking to computers. The system will do what you tell it to do, not what you think you told it to do.
And I try to emulate this to avoid confusion as much as possible.
Thus when you say you specify if= and of=, as long as you do not tell what the values are, I can not confirm what most probably will happen.
Also (again), when there is no ISO file system on the CD/DVD (e.g. when it is a music CD), the result will not be a so called ISO file.
And no, you do not need a bs and/or count parameter when you copy the openSUSE ISO file to a USB mass-meory device.
Didn’t you read the instructions on SDB:Live USB stick - openSUSE Wiki?
There it says:
]