4

I am developing a web application using php and MySQL. I am compressing the text using gzcompress() php function before storing in MySQL database / storing compressed form of text in database. My Question is that is this OK? to store compressed form? Or this method will create trouble or not for me? I am compressing text for saving disk space.

5
  • Why did you feel that it would be beneficial to compress the text before storing? Commented Mar 20, 2012 at 7:03
  • @IgnacioVazquez-Abrams for save disk space Commented Mar 20, 2012 at 7:05
  • Disk space is cheap. Compressed text can't be searched. Doesn't seem like it would often be a good trade off.
    – Quentin
    Commented Mar 20, 2012 at 7:06
  • @Quentin Is there any way to save large text with small size? Commented Mar 20, 2012 at 7:12
  • Don't worry about it and get a bigger disk.
    – Quentin
    Commented Mar 20, 2012 at 7:14

3 Answers 3

16

My approach. I needed to store email's text in MySQL table. Column type: TEXT.

Compress:

$compressed_string_for_db = base64_encode(gzcompress('big email text', 9));

Uncompress:

$email_text = gzuncompress(base64_decode($compressed_string_from_db));

My compression result:

income string length: 41072

compressed string length: 5312

1
  • 1
    Why do you do a base64_encode ? You use gzip level 9, which is generally a bad trade-off between CPU cycles & storage gain. If the fields should not be searched compression seems fine;
    – SvennD
    Commented Feb 10, 2020 at 14:13
8

I can definitely see a LOT of practical benefits of storing compressed text ...
gz compression is fast, can be streamed and saves a LOT of space in some scenarios.

In my practical real life use I have data usage of about 50MB per 1000 records and I am getting millions of records.
Compressed I am able to reduce this to 10MB per 1000 records which means I pay 1/5th of the usual data storage fee to my cloud provider.
If that's not a practical benefit, what else ?;)

Also, did you consider that the CPU load of gzip is minimal ? gzip is already a HTTP standard, stackoverflow is being sent to your browser in a gzip stream just like almost any website if you are not using a browser from 1985.
I would guess that disk access (especially hard disks) have more impact than gzip on data retrieval. So by reducing the data through compression size you might actually gain performance.

Here is the code I use to compress the data in a mysql compatible format:
Here is my code for that scenario, for uncompress you can also use PHP and just substr the first 4 bytes away.
Output of mysql:

mysql : "select hex(compress('1234512345'))"
0A000000789C3334323631350411000AEB01FF

The php equivalent:

$string="1234512345";
$data=gzcompress($string);
$len=mb_strlen($string);
$head=pack('V',$len);

echo($head);
echo($data);

Output of PHP:

php test.php  | hexdump -C
00000000  0a 00 00 00 78 9c 33 34  32 36 31 35 04 11 00 0a
5

My approach is significantly faster than sNICkerssss aproach.

Column type: BLOB. The reason for using BLOB because:

A BLOB is a binary large object that can hold a variable amount of data. BLOB values are treated as binary strings (byte strings).

Compress:

$compressed = gzdeflate('big email text');

Decompress:

$email_text = gzinflate($compressed);

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.