This isn't really an answer to your question, but the results of the investigation that I've carried out. I used to struggle with CRC when I was building a simple implementation of an Ethernet controller in VHDL, and I do realize that implementations can be sometimes very varying due to reasons unknown.
Okay, let's begin. The typedef that you found in boost/crc.hpp :
typedef crc_optimal<32, 0x04C11DB7, 0xFFFFFFFF, 0xFFFFFFFF, true, true> crc_32_type;
Is a simple declaration of a CRC generator that will produce the CRC for Ethernet. The parameters of the template are as follows : Bits (the number of bits output by the generator), TruncPoly (the polynomial used by the generator), InitRem (the initial remainder to be fed into the algorithm before processing the first byte of input), FinalXor (the value that the output value should be XORed with after processing all the bytes of the input), ReflectIn and ReflectRem (if the input bytes and/or the output should be bit-reflected, e.g. bit 0 becomes bit 7, and so on). Ethernet not only requires the output to be calculated with the given polynomial, but it also requires the constraints that you can read from that typedef.
According to the specification of cksum, the typedef for the CRC generator for it should look something like this :
typedef crc_optimal<32, 0x04C11DB7, 0, 0xFFFFFFFF, false, false> cksum_crc_type;
This is because :
- The specification doesn't specify a starting value for the generator, thus 0.
- The output value should be complemented, according to number 4 of the specification. The same result can be achieved by XORing the value with ones.
- Bit reflecting is not mentioned anywhere, and thus will not be performed.
However, there is one significant difference when it comes to cksum as opposed to ordinary CRCs :
(...) followed by one or more octets representing the length of the
file as a binary value, least significant octet first. The smallest
number of octets capable of representing this integer shall be used.
Ordinary CRC generators don't take into account the number of octets that were processed. This would also explain why you got a good result when processing a zero-length file, but a bad one when processing larger files.
Unfortunately, I don't see an easy solution for that problem. I guess what you could do is modify the process_bytes method in a following way :
template < std::size_t Bits, BOOST_CRC_PARM_TYPE TruncPoly,
BOOST_CRC_PARM_TYPE InitRem, BOOST_CRC_PARM_TYPE FinalXor,
bool ReflectIn, bool ReflectRem >
inline
void
BOOST_CRC_OPTIMAL_NAME::process_bytes
(
void const * buffer,
std::size_t byte_count
)
{
unsigned char const * const b = static_cast<unsigned char const *>(
buffer );
process_block( b, b + byte_count );
for(; byte_count; byte_count >>= 8)
rem_ = (rem_ << 8) ^ crc_table_type::table_[((rem_ >> 24) ^ byte_count) & 0xFF];
}
With such an implementation, the method gives the same result as cksum. for loop courtesy of GNU coreutils.
Hope I helped.