Page 1 of 4 123 ... LastLast
Results 1 to 10 of 37

Thread: couple rsync with tar.bzip2

  1. #1

    Default couple rsync with tar.bzip2

    Hi everyone,
    I am using rsync as a way of having duplicates of my files in an external hard disk (in case something goes wrong).

    I would like to ask you though if there is a way to couple rsync with thar bzip2 in a way that all the copied files at the rsync destination are packed and compressed together to a single file.

    Would that be possible?

    Regards
    Alex

  2. #2
    Join Date
    Feb 2009
    Location
    Spain
    Posts
    25,547

    Default Re: couple rsync with tar.bzip2

    On 2013-02-05 12:36, alaios wrote:
    >
    > Hi everyone,
    > I am using rsync as a way of having duplicates of my files in an
    > external hard disk (in case something goes wrong).
    >
    > I would like to ask you though if there is a way to couple rsync with
    > thar bzip2 in a way that all the copied files at the rsync destination
    > are packed and compressed together to a single file.
    >
    > Would that be possible?


    You can compress the copy, of course, but then a second rsync as done to
    refresh or update the backup would copy absolutely everything, the
    compressed files or archives are not recognized.

    So the answer is "no".

    If Linux had a compressed read/write filesystem (as is NTFS in Windows)
    then we could use that transparently. But, quite unbelievably, Linux
    does not have a compressed, r/w filesystem.

    You can, on the other hand, use a program like "rdiff-backup". The
    current backup is an rsync, the old ones are rdifs, which should be smaller.

    --
    Cheers / Saludos,

    Carlos E. R.
    (from 12.1 x86_64 "Asparagus" at Telcontar)

  3. #3

    Default Re: couple rsync with tar.bzip2

    Thanks a lot for the answer.
    Would a pipelining work like

    rsync -rav -e ssh /home/user/Documents/Documents-Sensitive/ user@server:/home/user/Documents/ | tar -jvf myfile.tar.bz2 ?

    Regards
    Alex

  4. #4
    Join Date
    Feb 2009
    Location
    Spain
    Posts
    25,547

    Default Re: couple rsync with tar.bzip2

    On 2013-02-05 14:56, alaios wrote:
    >
    > Thanks a lot for the answer.
    > Would a pipelining work like
    >
    > rsync -rav -e ssh /home/user/Documents/Documents-Sensitive/
    > user@server:/home/user/Documents/ *| *tar -jvf myfile.tar.bz2 ?


    No, nothing would.

    Rsync needs to compare the source and destination files, and they have
    to be exactly the same. At least same name and timestamp, and it can
    also verify the internal checksum. You can not alter in any way the copy.

    Only a transparent, compressed, r/w, filesystem would work. And it does
    not exist in Linux, only in Windows. Maybe btfrs, perhaps.

    --
    Cheers / Saludos,

    Carlos E. R.
    (from 12.1 x86_64 "Asparagus" at Telcontar)

  5. #5

    Default Re: couple rsync with tar.bzip2

    I guess though that I can split it with a bash script
    with the first command doing the rsync and the second one taking the destination folder and compressing it to a single file.

    Alex

  6. #6

    Default Re: couple rsync with tar.bzip2

    alaios wrote:
    > I guess though that I can split it with a bash script
    > with the first command doing the rsync and the second one taking the
    > destination folder and compressing it to a single file.


    If you want to do that, why would you use rsync in the first place? Why
    not just use tar?

  7. #7
    Join Date
    Jun 2008
    Location
    Groningen, Netherlands
    Posts
    19,649
    Blog Entries
    14

    Default Re: couple rsync with tar.bzip2

    Take a look at luckybackup. It has advanced options, where you can add a tar command.

    But, have you considered using tar all the way?

    Code:
    tar -cjvf /media/Backup/`date +%d%m%y`.tar.bz2 /home/knurpht/Test
    This generates a compressed tar file from the Test folder in my homedir with a datestamp in the generated tarfile. Luckbackup allows you to use a command like this one before and after the actual rsync operation.

    If you want to stick to CLI, you could create a bash script that does the tarring before or after rsync.
    ° Appreciate my reply? Click the star and let me know why.

    ° Perfection is not gonna happen. No way.

    https://en.opensuse.org/openSUSE:Board#Members
    http://en.opensuse.org/User:Knurpht
    http://nl.opensuse.org/Gebruiker:Knurpht

  8. #8

    Default Re: couple rsync with tar.bzip2

    Quote Originally Posted by djh-novell View Post
    alaios wrote:
    > I guess though that I can split it with a bash script
    > with the first command doing the rsync and the second one taking the
    > destination folder and compressing it to a single file.


    If you want to do that, why would you use rsync in the first place? Why
    not just use tar?
    For some reason I trust more rsync for doing decent copies .....
    Nothing more to reason here

    Alex

  9. #9
    Join Date
    Jul 2008
    Location
    Seattle, WA
    Posts
    17,043

    Default Re: couple rsync with tar.bzip2

    On Tue, 05 Feb 2013 11:36:01 +0000, alaios wrote:

    > Hi everyone,
    > I am using rsync as a way of having duplicates of my files in an
    > external hard disk (in case something goes wrong).
    >
    > I would like to ask you though if there is a way to couple rsync with
    > thar bzip2 in a way that all the copied files at the rsync destination
    > are packed and compressed together to a single file.
    >
    > Would that be possible?


    As Carlos has said, not really, no - because rsync depends on comparing
    the source and destination files, so if you change the data in transit,
    then the comparison will fail.

    But it sounds like you're focused on a "how" rather than a "what". It
    sounds like what you want to do is create a compressed backup. *That* is
    something that can be done, but not with the tools you're specifying.

    A couple of possibilities:

    1. Use the tar command to create incremental/differential backups with a
    weekly backup schedule. tar can compress using bzip2 or gzip, and by
    backing up incremental or differential backups periodically and a full
    backup weekly (for example), you can accomplish this sort of thing.

    The difference between incremental and differential has to do with how
    much data is backed up - if the comparison is against the last full or
    the last full+the last partial backup.

    2. You could create a compressed filesystem using FUSE - something like
    compFUSEd would do this - it creates a layer over an existing filesystem
    that stores the files in a compressed format (from reading the brief
    description, it sounds like what encfs does for encryption - you see the
    files in the directories, but they're stored compressed and need to be
    accessed through the FUSE layer. That would be an efficient option for
    storage, because otherwise you'd have to allocate a file and mount that
    as a compressed filesystem using FUSE or a loop filesystem.

    Jim
    --
    Jim Henderson
    openSUSE Forums Administrator
    Forum Use Terms & Conditions at http://tinyurl.com/openSUSE-T-C

  10. #10
    Join Date
    May 2010
    Location
    Space Colony Lagrange Point 22° à, 77° Ƅ, 56° ɤ, 99° ɜ
    Posts
    3,166

    Default Re: couple rsync with tar.bzip2

    I found this in YaST
    Duplicity incrementally backs up files and directories by encrypting tar-format volumes with GnuPG and uploading them to a remote (or local) file server. In theory many remote backends are possible; right now local, ssh/scp, ftp, rsync, HSI, WebDAV, and Amazon S3 backends are written. Because duplicity uses librsync, the incremental archives are space efficient and only record the parts of files that have changed since the last backup. Currently duplicity supports deleted files, full unix permissions, directories, symbolic links, fifos, etc., but not hard links.
    GNOME Version 3.20.2
    openSUSE Leap 42.3 64-bit

    www.vazhavandan.blogspot.com

Page 1 of 4 123 ... LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •