0

Let say I am trying to read data from a file. The data is stored as binary data and can be read easily using compiler packing extension (using C99 notation for clarity):

#pragma pack(push, 1) /* visual studio */
struct S
{
  int16_t v1;
  uint32_t v2; /* gcc would use: __attribute__((packed)) */
  int16_t v3;
};
#pragma pack(pop)
void read( std::istream & is )
{
  S s;
  assert( sizeof(s) == 8 ); // packed !
  is.read( (char*)&s, sizeof(s) );
  std::cout << s.v1 << " " << s.v2 << " " << s.v3 << std::endl;
}

It becomes much messier to write the same code but with portability in mind:

struct S2
{
  unsigned char buf1[2];
  unsigned char buf2[4];
  unsigned char buf3[2];
};
static inline uint16_t makenum(const unsigned char (&x)[2])
{
  return x[0] | (x[1] << 8);
}
static inline uint32_t makenum(const unsigned char (&x)[4])
{
  return
      ((uint32_t)x[0] <<  0)
    | ((uint32_t)x[1] <<  8)
    | ((uint32_t)x[2] << 16)
    | ((uint32_t)x[3] << 24);
}
void read( std::istream & is )
{
  S2 s2;
  assert( sizeof(s2) == 8 ); // garanteed !
  is.read( (char*)&s2, sizeof(s2) );
  std::cout << makenum(s2.buf1) << " " << makenum(s2.buf2) << " " << makenum(s.buf3) << std::endl;
}

Is there anything else (smarter) to do ? I guess the bit-shift and bitwise inclusive OR should not impact much the execution, but I could not find a generic solution with union to avoid the computation. Eg: pseudo-solution (non working):

struct S3
{
  union { char buf1[2]; int16_t v1; } uv1;
  union { char buf2[4]; uint32_t v2; } uv2;
  union { char buf3[2]; int16_t v3; } uv3;
};
3
  • 1
    I doubt there's a real portable solution for your problems (especially cross platform) usinng simple structs. You should have a look at boost::serialization or gooble protocol buffers to achieve what you want. Dec 8, 2014 at 15:45
  • What about functions like htonl() and ntohl()?
    – Julian
    Dec 8, 2014 at 15:54
  • @AtlasC1 data alignement != endianness
    – malat
    Dec 8, 2014 at 15:59

2 Answers 2

1

I highly recommend do not use struct to map fields 1:1 with protocols.

One of the reasons is that compilers are allowed to add padding between fields.

Another is that you want the structure to be efficient with your processor. Hopefully, you will manipulate the data in the structure more often that you will perform I/O with it.

For example, given a 32-bit processor is very efficient with 32-bit numbers, but not as efficient with 16 bit. The protocol requires 16-bit integers. So, do you map the fields in the structure as 16-bit or 32-bit?

Answer: Use 32-bit fields in the structure and write methods to convert to and from the protocol. For example, to load a 16-bit variable from memory, a 32-bit processor may have to perform a fetch and a shift, depending on where the 16-bits are located in the 32-bit register. If the structure field is 32-bits, no shift is required; thus more efficient.

Also, writing protocol conversion functions allow you to handle Big-Endian versus Little-Endian issues without changing the structure.

1
  • while I understand your general comment, a C compiler will not make any difference (release mode) in between char buf[512] and typedef struct { char buf1[512/2]; char buf2[512/2]; } S;, the second syntax is much easier to read and maintain in the long term.
    – malat
    Dec 9, 2014 at 11:05
0

Here is the solution I came up with:

#include <cstring>
#include <stdint.h>

template <typename T>
struct Fast;
template <>
struct Fast<uint16_t> {
  typedef uint_fast16_t Type;
};
template <>
struct Fast<int16_t> {
  typedef int_fast16_t Type;
};
template <>
struct Fast<uint32_t> {
  typedef uint_fast32_t Type;
};
template <>
struct Fast<int32_t> {
  typedef int_fast32_t Type;
};
template <>
struct Fast<uint64_t> {
  typedef uint_fast64_t Type;
};
template <>
struct Fast<int64_t> {
  typedef int_fast64_t Type;
};

template <typename T>
struct Helper {
  typedef typename Fast<T>::Type RetType;
  typedef char (VecType)[sizeof(T)];
  typedef union { VecType vec; T val; } UType;
};

template <typename T>
struct MakeNum {
  typedef typename Helper<T>::RetType RetType;
  typedef typename Helper<T>::UType UType;

  static RetType Get(const char (&x)[sizeof(T)]) {
    UType u;
    memcpy( u.vec, x, sizeof(T) );
    return u.val;
  }
};


#define AddField( type, name ) \
  char name ## _[sizeof(type)]; \
  typename MakeNum<type>::RetType \
    name () const { return MakeNum<type>::Get(name ## _); }

struct S
{
  AddField( uint16_t, name1 );
  AddField(  int32_t, name2 );
  AddField( uint16_t, name3 );
};

int main()
{
  S s = { 0, 1, 0, 1, 0, 0, 0, 1 };
  return s.name1() + s.name2() + s.name3();
}

It does generate exactly the same code as this (non-portable) one, using gcc 4.9.1:

#include <stdint.h>

struct S2
{
  uint16_t v1;
   int32_t v2 __attribute__((packed));
  uint16_t v3;
};

int main()
{
  S2 u = { 256, 256, 256 };
  return u.v1 + u.v2 + u.v3;
}

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.