Compressing on openSUSE: How to split into 4.7GB volumes

I’m trying to compress a 16GB folder into 2 DVD’s, but I’m new to openSUSE and I’ve never compressed anything. Plus, I can’t find any option that allows me to split it into volumes, is there a way of doing so?

http://img856.imageshack.us/img856/9861/snapshot2ub.png

Hi, n easy way to do that would b to compress the folder into a sinle file first, using the archiver of your choice. Then split the resulting file with the “split” command (type split --help"). At your destination recreate the file (join it again) with the “cat” command. HTH Lenwolf

Custom splitting based on media is possible through using J7Z from openSUSE Software

Floppy,CD,DVD etc
Also support custom sizes like spitting into 1MB sizes

http://paste.opensuse.org/images/49354616.png

Directory compressed and split into floppy size files

http://paste.opensuse.org/images/23476136.png

Thank you all very much :smiley:

On 2013-05-01 08:56, amarildojr wrote:
>
> I’m trying to compress a 16GB folder into 2 DVD’s, but I’m new to
> openSUSE and I’ve never compressed anything. Plus, I can’t find any
> option that allows me to split it into volumes, is there a way of doing
> so?

This is a method:


> https://forums.opensuse.org/english/get-technical-help-here/how-faq-forums/unreviewed-how-faq/444240-procedure-backup-large-disk-image-across-several-dvds.html

Other methods: There are proprietary compressors, like rar, which can
split the archive on several files of a given size. The Linux version is
CLI only. I don’t know if it keeps Linux file attributes, I have not
tested it and it is not mentioned in the documentation. Correction,
there is a brief note, it might work).

Opensource zip can not do this. However, you can create a big archive,
then use it split to break it in pieces. To reconstruct the contents,
you have to join the pieces into a single archive first, you can not
just enter one DVD and uncompress it.

(zip can split at an approximate size, but only on complete files
inside. The proprietary msdos zip can split anywhere).


Cheers / Saludos,

Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)

Carlos E. R. wrote:
> On 2013-05-01 08:56, amarildojr wrote:
>> I’m trying to compress a 16GB folder into 2 DVD’s, but I’m new to
>> openSUSE and I’ve never compressed anything. Plus, I can’t find any
>> option that allows me to split it into volumes, is there a way of doing
>> so?
>
> This is a method:
>
>


>> https://forums.opensuse.org/english/get-technical-help-here/how-faq-forums/unreviewed-how-faq/444240-procedure-backup-large-disk-image-across-several-dvds.html
> 

Interesting. I like the idea of dvdisaster

Could you compress to a single large file, then use split into 3.7 GB
lumps and run dvdisaster on those? Slightly simpler if you can.

> Other methods: There are proprietary compressors, like rar, which can
> split the archive on several files of a given size. The Linux version is
> CLI only. I don’t know if it keeps Linux file attributes, I have not
> tested it and it is not mentioned in the documentation. Correction,
> there is a brief note, it might work).

amanda can do mult-volume backup, as can various other progs. I expect
amanda has all kinds of bells and whistles to tweak the process.

Am 01.05.2013 17:28, schrieb Carlos E. R.:
> Other methods: There are proprietary compressors, like rar, which can
> split the archive on several files of a given size. The Linux version is
> CLI only. I don’t know if it keeps Linux file attributes, I have not
> tested it and it is not mentioned in the documentation. Correction,
> there is a brief note, it might work).

file-roller can do it from the GUI if that is more pleasant. Select rar
as archive type and in the advanced settings tell it how large the junks
should be.
I looked if Ark can do it as well (of course it can make a rar if rar is
installed) but it seems to lack the setting to split the archive.


PC: oS 12.3 x86_64 | i7-2600@3.40GHz | 16GB | KDE 4.10.2 | GTX 650 Ti
ThinkPad E320: oS 12.3 x86_64 | i3@2.30GHz | 8GB | KDE 4.10.2 | HD 3000
HannsBook: oS 12.3 x86_64 | SU4100@1.3GHz | 2GB | KDE 4.10.2 | GMA4500

On 2013-05-01 17:54, Dave Howorth wrote:
> Carlos E. R. wrote:
>> On 2013-05-01 08:56, amarildojr wrote:
>>> I’m trying to compress a 16GB folder into 2 DVD’s, but I’m new to
>>> openSUSE and I’ve never compressed anything. Plus, I can’t find any
>>> option that allows me to split it into volumes, is there a way of doing
>>> so?
>>
>> This is a method:
>>
>>


>>> https://forums.opensuse.org/english/get-technical-help-here/how-faq-forums/unreviewed-how-faq/444240-procedure-backup-large-disk-image-across-several-dvds.html
>> 

>
> Interesting. I like the idea of dvdisaster
>
> Could you compress to a single large file, then use split into 3.7 GB
> lumps and run dvdisaster on those? Slightly simpler if you can.

The problem is achieving compressed files of a certain size, like the
3.7 GB you say. You can, as you say, compress to a large single file,
and then split it in sizeable chunks. However, to reconstruct, you need
reconstructing the entire large single file first, then expand it; you
can not simply plug in the DVD and expand what it contains on the go.

My method (which is complicated, I agree) was made for imaging large
partitions, not directories with files.

What I do is split the image in smaller chunks which I then compress
with zisofs; then burn to a DVD a number of compressed chunks that
approximate a DVD size (or less with dvdisaster info).

The difference when using zisofs is that the kernel decompresses
transparently the DVD, so that the DVD may apear to have, say, 7 GiB of
files: no need to run a tool to decompress them. You can thus simple
concatenate the chunks directly on the destination, no need for an
intermediate space for decompression or joining of the splited chunks.

My method is complicated for backup creation, but easier on
reconstruction. However, with current hard disk sizes, it is just easier
to image directly to another hard disk. Unfortunately, I do not know of
a filesytem that can do compression on the fly and transparently.

The unfortunate thing is that you can not, in Linux, use zip (or gzip)
an archive to an arbitrary chunk size.

>> Other methods: There are proprietary compressors, like rar, which can
>> split the archive on several files of a given size. The Linux version is
>> CLI only. I don’t know if it keeps Linux file attributes, I have not
>> tested it and it is not mentioned in the documentation. Correction,
>> there is a brief note, it might work).
>
> amanda can do mult-volume backup, as can various other progs. I expect
> amanda has all kinds of bells and whistles to tweak the process.

I tried it once, but I found difficulties. It is made for network,
enterprise, setups. I did not find it easier to backup to DVDs or hard
disks via usb.

I made a list of backup tools; not all of them I tried:

dar
rdiff-backup current copy is a mirror; older are rdifss.
rsnapshot current copy is a mirror; older are hardlinks
and new files.
gadmin-rsync?
http://www.dirvish.org/
pdumpfs (http://0xcc.net/pdumpfs)
duplicity
duply
Back-In-Time (http://backintime.le-web.org/)
LuckyBackup
deja-dup
dropbox
duplicity

I don’t know if there is a tool that can do backups of files or
partitions, split the result on DVDs, and do all that with forward
recovery data - ie, protection to recover the data on a DVD even in the
case of damaged sectors (dvdisaster is for that).

Decades ago, I used PCtools backup for MSdos. It did backups to
floppies, and very fast. It created reconstruction data, so that
floppies with damaged sectors could still be recovered. I believe that
my 80 floppy backup sets are still usable - on old machines of the time.
Unfortunately, this proprietary tool only worked with the same software
version, and on certain hardware. It worked on my 386 machine, but not
on my pentium machine, because it used a proprietary floppy low level
format method that was both fast, safe, and large capacity, but it
depended on the hardware.

Anyway, I have not seen backup software for Linux capable of achieve
similar results with, say, DVDs.


Cheers / Saludos,

Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)

J7z is by far the best one, it gives me an GUI and it works fine =)

On 2013-05-01 22:06, amarildojr wrote:
>
> J7z is by far the best one, it gives me an GUI and it works fine =)

It is not distributed on openSUSE except by a home repo for 12.3 and 12.2.


Cheers / Saludos,

Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)

On 2013-05-01 18:34, Martin Helm wrote:
> Am 01.05.2013 17:28, schrieb Carlos E. R.:
>> Other methods: There are proprietary compressors, like rar, which can
>> split the archive on several files of a given size. The Linux version is
>> CLI only. I don’t know if it keeps Linux file attributes, I have not
>> tested it and it is not mentioned in the documentation. Correction,
>> there is a brief note, it might work).
>
> file-roller can do it from the GUI if that is more pleasant. Select rar
> as archive type and in the advanced settings tell it how large the junks
> should be.

It is a frontend, so it can only do what the archiver CLI can do. Of
course, it makes things easier.


Cheers / Saludos,

Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)

Am 01.05.2013 23:13, schrieb Carlos E. R.:
> It is a frontend, so it can only do what the archiver CLI can do. Of
> course, it makes things easier.

I never said something else.


PC: oS 12.3 x86_64 | i7-2600@3.40GHz | 16GB | KDE 4.10.2 | GTX 650 Ti
ThinkPad E320: oS 12.3 x86_64 | i3@2.30GHz | 8GB | KDE 4.10.2 | HD 3000
HannsBook: oS 12.3 x86_64 | SU4100@1.3GHz | 2GB | KDE 4.10.2 | GMA4500

On 2013-05-01 23:15, Martin Helm wrote:
> Am 01.05.2013 23:13, schrieb Carlos E. R.:
>> It is a frontend, so it can only do what the archiver CLI can do. Of
>> course, it makes things easier.
>
> I never said something else.

Ah, ok :slight_smile:

But what I want is archivers that can automatically split at a given
size, with error recovery, and expand also automatically. The only one I
know is rar, and it is not free.

And none can create DVDs on the fly with error recovery of the type done
by dvdisaster.


Cheers / Saludos,

Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)

Am 02.05.2013 00:03, schrieb Carlos E. R.:
> But what I want is archivers that can automatically split at a given
> size,
whixh is what file-roller does

> with error recovery,
this is a feature of the rar format and I would expect that this is then
also what file-roller does when I choose the rar format

> and expand also automatically.
this is what file-roller does when I simply click on the first rar file
from the series it produced when compressing

> The only one I
> know is rar, and it is not free.
rar is a compression format, I do not see inhowfar a free GPL
implementation is not free
>
> And none can create DVDs on the fly with error recovery of the type done
> by dvdisaster.
>
that is of course a different requirement which is not fulfilled by
something as simple as file-roller or the command line rar.
But it was not asked for that in the original post.


PC: oS 12.3 x86_64 | i7-2600@3.40GHz | 16GB | KDE 4.10.2 | GTX 650 Ti
ThinkPad E320: oS 12.3 x86_64 | i3@2.30GHz | 8GB | KDE 4.10.2 | HD 3000
HannsBook: oS 12.3 x86_64 | SU4100@1.3GHz | 2GB | KDE 4.10.2 | GMA4500

On 2013-05-02 00:29, Martin Helm wrote:
> Am 02.05.2013 00:03, schrieb Carlos E. R.:
>> But what I want is archivers that can automatically split at a given
>> size,
> whixh is what file-roller does
>
>> with error recovery,
> this is a feature of the rar format and I would expect that this is then
> also what file-roller does when I choose the rar format

For doing that it needs that the rar program is installed in the system,
and as I say, it is not free:

  1. The software is distributed as try before you buy. This means that
    anyone may use the software during a test period of a maximum of 40
    days
    at no charge. Following this test period, the user must purchase
    a license to continue using the software.

>> The only one I
>> know is rar, and it is not free.
> rar is a compression format, I do not see inhowfar a free GPL
> implementation is not free

There is no GPL implementation of RAR. Have a look at the license file
of the rar program obtainable from packman. There is only an opensource
decompressor, and I think it is for version 2 only. The author of rar
published it long ago.

The rar format itself is also proprietary.

>> And none can create DVDs on the fly with error recovery of the type done
>> by dvdisaster.
>>
> that is of course a different requirement which is not fulfilled by
> something as simple as file-roller or the command line rar.
> But it was not asked for that in the original post.

True.

But that is what I would want if I need to backup a big folder into
several DVDs.


Cheers / Saludos,

Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)

Ouch - you are right that is nonfree software.
I confused that now completely.

But that does not look bad as a gui program
http://peazip.sourceforge.net/peazip-free-archiver.html
at least for just creating the splitted archives of arbitrary size.

dvdisaster looks also good for a different purpose, I read the homepage
now (I can only find a home build for it though).

Maybe if one combines just the 7z command compress+split (split to
volumes with -v) with a dvdisaster error checking and automatic burn in
a simple script and an analog script for recovery you have the best of
all worlds.
Shouldn’t that be easy and it contains then only free software (p7zip is
LGPL).


PC: oS 12.3 x86_64 | i7-2600@3.40GHz | 16GB | KDE 4.10.2 | GTX 650 Ti
ThinkPad E320: oS 12.3 x86_64 | i3@2.30GHz | 8GB | KDE 4.10.2 | HD 3000
HannsBook: oS 12.3 x86_64 | SU4100@1.3GHz | 2GB | KDE 4.10.2 | GMA4500

On 2013-05-02 01:10, Martin Helm wrote:
> Ouch - you are right that is nonfree software.
> I confused that now completely.
>
> But that does not look bad as a gui program
> http://peazip.sourceforge.net/peazip-free-archiver.html
> at least for just creating the splitted archives of arbitrary size.

I think it is also a frontend. This is not clear from the page, till you
get to the section “Third party technologies”.

>
> dvdisaster looks also good for a different purpose, I read the homepage
> now (I can only find a home build for it though).

I don’t remember if I built it myself or found it ready made. I think
the later.

It is written in Pascal (Lazarus) :wink:

> Maybe if one combines just the 7z command compress+split (split to
> volumes with -v) with a dvdisaster error checking and automatic burn in
> a simple script and an analog script for recovery you have the best of
> all worlds.
> Shouldn’t that be easy and it contains then only free software (p7zip is
> LGPL).

Yes, it is possible to do, I know.

For imaging, there is a software I was recently told of, “ewftools”, in
the security repo. It creates CRC checksums of the images for
verification. I want to test it, but I haven’t yet.

What I have used is “partimage”. It has compression and split (for
writing to DVDs), but no error protection. It does not store empty
sectors, thus it needs knowledge of the filesystem. ext4/btrfs are not
supported.

I feel safer with a plain dd, uncompressed, somehow…


Cheers / Saludos,

Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)

peazip is more or less a frontend for p7zip, that is the reason I
mentioned later 7z (the command line utility which supports the same
features).
So you only need the free 7z archiving utilities not the proprietary rar.

I do not use myself DVDs for backup or archiving (except pictures and
documents which I do not compress and “split” manually, my error
protection is trivial, I keep several copies of important files on
different media), so I do not have much use for all that splitting and
compressing things.

Sometimes I have for rare and different purposes such a need (last time
maybe 3 or 4 years ago).


PC: oS 12.3 x86_64 | i7-2600@3.40GHz | 16GB | KDE 4.10.2 | GTX 650 Ti
ThinkPad E320: oS 12.3 x86_64 | i3@2.30GHz | 8GB | KDE 4.10.2 | HD 3000
HannsBook: oS 12.3 x86_64 | SU4100@1.3GHz | 2GB | KDE 4.10.2 | GMA4500

On 2013-05-02 02:34, Martin Helm wrote:
> peazip is more or less a frontend for p7zip, that is the reason I
> mentioned later 7z (the command line utility which supports the same
> features).
> So you only need the free 7z archiving utilities not the proprietary rar.

I see.

> I do not use myself DVDs for backup or archiving (except pictures and
> documents which I do not compress and “split” manually, my error
> protection is trivial, I keep several copies of important files on
> different media), so I do not have much use for all that splitting and
> compressing things.

I know.

Hard disks are so big nowdays, that making backups on DVDs is horribly
tedious and just not done. It is easier, and perhaps cheaper, to connect
an external big enough hard disk via eSATA or USB and just do it. You
can leave it overnight running if you must.

At the heyday of floppies, there were automatic floppy feeders. You
could start a backup session, pile a bunch of a hundred floppies in the
stack, and click enter on the software. Write a floppy, eject it, take
the next one… all automatic. I think some could eject bad floppies to
another pile.

Other people used tapes instead. They were too expensive for me (and the
floppy feeders too).


Cheers / Saludos,

Carlos E. R.
(from 12.1 x86_64 “Asparagus” at Telcontar)

I remember the time when I had myself only floppies to backup the most
important data. A second hard disk or tape drive was way too much money
and even floppies were at the beginning really expensive.

Well today if I have 16GB to store I take a flash card for a few bucks
and store it on that if it has not to last for the next 50 years.

But I am afraid we are way off topic now as the OP found a tool which
fits the needs so I will stop here.


PC: oS 12.3 x86_64 | i7-2600@3.40GHz | 16GB | KDE 4.10.2 | GTX 650 Ti
ThinkPad E320: oS 12.3 x86_64 | i3@2.30GHz | 8GB | KDE 4.10.2 | HD 3000
HannsBook: oS 12.3 x86_64 | SU4100@1.3GHz | 2GB | KDE 4.10.2 | GMA4500