3

My problem is: I want to store a array of bytes in compressed file, and then I want to read it with a good performance.

So I create a array of bytes then pass to a ZLIB algorithm then store it in the file. For my surprise the algorithm doesn't work well., probably because the array is a random sample. Using this approach, it will will be ber easy to read. Just copy the stream to memory, decompress them and copy it to a array of bytes.

But i need to compress the file. Do I have to use a algorithm, like RLE, for compresse the byte array? I think that I can store the byte array like a string and then compress it. But i think I am going to have a poor performance on reading data.

Sorry for my poor english.

Thanks

3
  • My file is like a binary file. I think because of it my compression was very poor. Am I right? Jun 5, 2010 at 15:21
  • Instead of create a file of Array of Bytes, i create a file of Array of Integer. And them compress it. For my surprise, the file of array of integer was very well compressed! Does anyone know why? thanks Jun 5, 2010 at 15:50
  • So, I investigate a little more. The file that i was creating using the function random(255). So when I compressed it the size increases. Jun 6, 2010 at 0:14

3 Answers 3

1

It's not clear whether you mean "random sample" in the statistical sense (a sample of real data chosen at random), or just randomly generated data. The first should compress fine.

However, truly random data can not be compressed (well). In fact, compressibility is one of the measures of randomness.

3
  • I made a test. I create a file that store an array of bytes with random bytes. Then i tryed to compress it but didn´t work. The compression was very poor. It is like a binary file, that is why the compression is poor? Jun 5, 2010 at 15:17
  • 1
    @Pedro, all compression algorithms work by exploiting non-randomness in the input. Thus, random data compresses poorly. Jun 5, 2010 at 15:22
  • So I need an algorith that compress my array of bytes. It seems simple to me. I want to pass my array of bytes to an algorithm and then receive a new array or stream compacted. Jun 5, 2010 at 15:26
1

If the data is random, it will have a high entropy. There is no algorithm that can compress such data with much success. That is, since it's random, you might get lucky on a specific case, but generally it's useless to try to compress it.

0

Are you sure that you really need to compress random data? Best compression can be achieved if you know what is source of data and some other specifications that can be used to choose and optimize compression algorithms. First how big your arrays are? If this are values of characters there is difference in frequency of each letter. If this array is sorted or it has unique values this can be also used to optimize space. So what is the source of your data?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.