I am trying to encode signed values ranging from -256 <-> 255 (i.e. 9 bit data represented by short) with arithmetic encoder, however I have discovered that existing implementations of Arithmetic coding (such as dlib, rANS) usually reads the file in the form of a string and treat the data as 8-bit.
The problem with this technique is that this splitting of signed data (shown in 3) in the form of string destroys the underlying histogram (shown in 4). I believe that such splitting may also be degrading the compression ratios (but I may be wrong).
I tested my hypothesis by implementing a Huffman encoding with 8-bit and 16-bit data and found that I was right, this maybe due to Huffman's dependence on making the tree by using probabilities.
(EDITED)My question is: How to encode/model symbols (which cannot be contained in a conventional 8-bit container) so that resulting symbols can be easily compressed with traditional arithmetic compressor implementations without affecting compression ratios.
Signed histogram:
Splitted histogram:
0x00
and0x01
's which wouldn't impact the compression (very much.... i think... maybe).... please tell if this makes no sense XD