Results 1 to 7 of 7

Thread: lzma compared to bzip2 for opensuse linux

  1. #1

    Default lzma compared to bzip2 for opensuse linux

    Dear all,
    I have been reading the following link, that is about lzma performance compred to bzip2.

    Lzma Vs Bzip2 – Better Compression than bzip2 on UNIX / Linux

    I would like to ask you if you have the same feeling that the article claims that lzma can achieve a better of compressed ratio. Moreover, I would like to ask you if the lzma is well supported in linux (in terms that I can download it and start using it, without major and minor bugs floating around).I

    I would like to thank you in advance for your reply

    Regards
    Alex

  2. #2
    Join Date
    Mar 2009
    Location
    United States
    Posts
    612

    Default Re: lzma compared to bzip2 for opensuse linux

    Personally, I use gzip. With the multi-threaded pigz utility it is almost i/o bound on compressing large files, and nearly as good as bzip2 at reducing size. Lzma, now called xz, is probably the best solution if squeezing the last bit of space is more useful than compression time.

    I did some tests a few months ago on this to make the decision. I do not have the results but they were easy to reproduce if someone doesn't want to take me at my word. For compressing and decompressing files I found:


    • Gzip compresses nearly as well as Bzip2 but compresses and decompresses a lot faster.
    • Xz compresses usually much better than either, but takes much longer to compress. Decompression is fast.

  3. #3
    Join Date
    May 2010
    Location
    Space Colony Lagrange Point 22° à, 77° Ƅ, 56° ɤ, 99° ɜ
    Posts
    3,166

    Default Re: lzma compared to bzip2 for opensuse linux

    gzip is probably the best format. Even browsers use gzip to render compressed pages(some websites use gzip to support faster page loading at client end)
    GNOME Version 3.20.2
    openSUSE Leap 42.3 64-bit

    www.vazhavandan.blogspot.com

  4. #4

    Default Re: lzma compared to bzip2 for opensuse linux

    Hi all,
    I have a big large tar file of size 1.2TB and it should be highly compressible (contains large sequences of 0s and 1s). I would like to squeeze it as much as possible so I was thinking for the xz.

    1. My fear is if xz is well tested (I learned about it today and I thought it might be a new untested utility).
    2. I wonder if all these utilities like tar,bzip2 and xz are well tested for large files. I found the following in xz that made me sceptical, as the file sizes reported there are way smaller than mine file


    Level xz LZMA Utils 4.32.x
    -0 3 MiB N/A
    -1 9 MiB 2 MiB
    -2 17 MiB 12 MiB
    -3 32 MiB 12 MiB
    -4 48 MiB 16 MiB
    -5 94 MiB 26 MiB
    -6 94 MiB 45 MiB
    -7 186 MiB 83 MiB
    -8 370 MiB 159 MiB
    -9 674 MiB 311 MiB

    .....
    Column descriptions:

    · DictSize is the LZMA2 dictionary size. It is waste of memory to use a dictionary bigger than the size of the uncompressed file.
    This is why it is good to avoid using the presets -7 ... -9 when there's no real need for them. At -6 and lower, the amount of
    memory wasted is usually low enough to not matter.


    3. How much extra memory should I need to compress my a big file ? I guess that these compression utilities would be able to work on it piece by piece, and do not need to load the whole 1.2TB file on to the memory (that of course I do not have).

    I would like to thank you in advance for your replies

    Regards
    Alex

  5. #5

    Default Re: lzma compared to bzip2 for opensuse linux

    On 2013-02-28, alaios <alaios@no-mx.forums.opensuse.org> wrote:
    >
    > Hi all,


    Well, since your post isn't really lzma vs bzip2, you might get better answers if you posted a new thread.

    > I have a big large tar file of size 1.2TB and it should be highly
    > compressible (contains large sequences of 0s and 1s).


    Sorry to break this to you, but all files (including text files) ultimately consist of sequences of 0s and 1s. The
    capacity to compress depends on the number of repeated sequences. You have not mentioned the source of your big tar file
    so it is impossible to gauge how compressible it is since YMMV massively depending on whether it's completely
    uncompressed media data or not. I'm sure Lempel-Ziv-based compression algorithms are reliable and failures are more
    likely to arise from your hardware media than any compression program itself. I use gzip because it's fast and proven;
    sure - the compression could be 10% greater but you're probably up to no good if 10% makes a huge difference to you.


  6. #6
    Chris Maaskant NNTP User

    Default Re: lzma compared to bzip2 for opensuse linux

    alaios wrote:

    > I would like to ask you if you have the same feeling that the article
    > claims that lzma can achieve a better of compressed ratio


    It realy depens on what you want to compress.
    Example:
    I have a wave audio file of 575.9MB.
    With rar i can compress it to 427.4MB.
    With 7zip only to 537.3MB

    I also have a text file of 1.8MB
    With i can compress it to 110.5KB
    With 7zip i can compress it to 96.8KB

    So you realy can't say which one is better if you test only with one
    type of file to compress.
    In other words, it just depends on what you need to do.

    --
    Chris Maaskant

  7. #7
    Chris Maaskant NNTP User

    Default Re: lzma compared to bzip2 for opensuse linux

    Chris Maaskant wrote:

    > With i can compress it to 110.5KB


    Should've been:
    With rar i can compress it to 110.5KB
    --
    Chris Maaskant

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •