I have binary data inside a bytearray that I would like to gzip first and then post via requests. I found out how to gzip a file but couldn't find it out for a bytearray. So, how can I gzip a bytearray via Python?
4 Answers
Have a look at the zlib
-module of Python.
Python 3: zlib
-module
A short example:
import zlib
compressed_data = zlib.compress(my_bytearray)
You can decompress the data again by:
decompressed_byte_data = zlib.decompress(compressed_data)
Python 2: zlib
-module
A short example:
import zlib
compressed_data = zlib.compress(my_string)
You can decompress the data again by:
decompressed_string = zlib.decompress(compressed_data)
As you can see, Python 3 uses bytearrays while Python 2 uses strings.
-
2For me it wanted a string, instead of bytearray..
compressedData = zlib.compress(mystring)
– mgearMay 27, 2016 at 3:24 -
3@mgear That's because you are using Python 2 which expects an input string - in Python 3, the function expects an bytearray... I've added this to my answer.– mozzbozzJun 5, 2016 at 13:53
-
From stackoverflow.com/a/8507012/8046487: "Note that this method is incompatible with the gzip command-line utility in that gzip includes a header and checksum, while this mechanism simply compresses the content." ; "If you want to produce a complete gzip-compatible binary string, with the header etc, you could use gzip.GzipFile together with StringIO" Apr 18, 2020 at 8:02
In case the bytearray is not too large to be stored in memory more than once and known as b
, you can just:
b_gz = str(b).encode('zlib')
If you need to do deocding first, have a look at the decode()
method of the bytearray.
-
1I'm not sure if this is correct? If you call
str()
on a bytearray you get something like"bytearray(b'test')"
- but he want's to compress the bytearray and not some string describing the bytearray (I think this could also result in a loss of data in some special circumstances?).– mozzbozzNov 5, 2014 at 9:29 -
1I just see that the code works in Python 2 only. In Python 3 there have been some changes in that area.– Klaus D.Nov 5, 2014 at 9:43
-
2Ok, that's what I've thought (I'm mainly working with Python 3). You might put a note in your post for people not reading the comments on the "first try".– mozzbozzNov 5, 2014 at 12:34
The zlib module of Python Standard Library should meet your requirements :
>>> import zlib
>>> a = b'abcdefghijklmn' * 10
>>> ca = zlib.compress(a)
>>> len(a)
140
>>> len(ca)
25
>>> b = zlib.decompress(ca)
>>> b == a
True
>>> b
b'abcdefghijklmnabcdefghijklmnabcdefghijklmnabcdefghijklmnabcdefghijklmnabcdefghijklmnabcdefghijklmnabcdefghijklmnabcdefghijklmnabcdefghijklmn'
This is the output under Python3.4, but it works same under Python 2.7 -
import zlib
import binascii
def compress_packet(packet):
return zlib.compress(buffer(packet),1)
def decompress_packet(compressed_packet):
return zlib.decompress(compressed_packet)
def demo_zlib() :
packet1 = bytearray()
packet1.append(0x41)
packet1.append(0x42)
packet1.append(0x43)
packet1.append(0x44)
print "before compression: packet:{0}".format(binascii.hexlify(packet1))
cpacket1 = compress_packet(packet1)
print "after compression: packet:{0}".format(binascii.hexlify(cpacket1))
print "before decompression: packet:{0}".format(binascii.hexlify(cpacket1))
dpacket1 = decompress_packet(buffer(cpacket1))
print "after decompression: packet:{0}".format(binascii.hexlify(dpacket1))
def main() :
demo_zlib()
if __name__ == '__main__' :
main()
This should do. The zlib requires access to bytearray content, use buffer() for that.