4

I have a message class which was previously a bit of a pain to work with, you had to construct the message class, tell it to allocate space for your object and then populate the space either by construction or memberwise.

I want to make it possible to construct the message object with an immediate, inline new of the resulting object, but to do so with a simple syntax at the call site while ensuring copy elision.

#include <cstdint>

typedef uint8_t id_t;
enum class MessageID { WorldPeace };

class Message
{
    uint8_t* m_data;         // current memory
    uint8_t m_localData[64]; // upto 64 bytes.
    id_t m_messageId;
    size_t m_size; // amount of data used
    size_t m_capacity; // amount of space available
    // ...

public:
    Message(size_t requestSize, id_t messageId)
        : m_data(m_localData)
        , m_messageId(messageId)
        , m_size(0), m_capacity(sizeof(m_localData))
    {
        grow(requestSize);
    }

    void grow(size_t newSize)
    {
        if (newSize > m_capacity)
        {
            m_data = realloc((m_data == m_localData) ? nullptr : m_data, newSize);
            assert(m_data != nullptr); // my system uses less brutal mem mgmt
            m_size = newSize;
        }
    }

    template<typename T>
    T* allocatePtr()
    {
        size_t offset = size;
        grow(offset + sizeof(T));
        return (T*)(m_data + offset);
    }

#ifdef USE_CPP11
    template<typename T, typename Args...>
    Message(id_t messageId, Args&&... args)
        : Message(sizeof(T), messageID)
    {
        // we know m_data points to a large enough buffer
        new ((T*)m_data) T (std::forward<Args>(args)...);
    }
#endif
};

Pre-C++11 I had a nasty macro, CONSTRUCT_IN_PLACE, which did:

#define CONSTRUCT_IN_PLACE(Message, Typename, ...) \
    new ((Message).allocatePtr<Typename>()) Typename (__VA_ARGS__)

And you would say:

Message outgoing(sizeof(MyStruct), MessageID::WorldPeace);
CONSTRUCT_IN_PLACE(outgoing, MyStruct, wpArg1, wpArg2, wpArg3);

With C++11, you would use

Message outgoing<MyStruct>(MessageID::WorldPeace, wpArg1, wpArg2, wpArg3);

But I find this to be messy. What I want to implement is:

    template<typename T>
    Message(id_t messageId, T&& src)
        : Message(sizeof(T), messageID)
    {
        // we know m_data points to a large enough buffer
        new ((T*)m_data) T (src);
    }

So that the user uses

Message outgoing(MessageID::WorldPeace, MyStruct(wpArg1, wpArg2, wpArg3));

But it seems that this first constructs a temporary MyStruct on the stack turning the in-place new into a call to the move constructor of T.

Many of these messages are simple, often POD, and they are often in marshalling functions like this:

void dispatchWorldPeace(int wpArg1, int wpArg2, int wpArg3)
{
    Message outgoing(MessageID::WorldPeace, MyStruct(wpArg1, wpArg2, wpArg3));
    outgoing.send(g_listener);
}

So I want to avoid creating an intermediate temporary that is going to require a subsequent move/copy.

It seems like the compiler should be able to eliminate the temporary and the move and forward the construction all the way down to the in-place new.

What am I doing that is causing it not to? (GCC 4.8.1, Clang 3.5, MSVC 2013)

9
  • 2
    Message w(id, in_place<MyStruct>(a,b,c)); do? Oh and a std::forward<T>(src) in the new would not hurt. Jan 31, 2014 at 23:59
  • It might, I'll take a look at that.
    – kfsone
    Feb 1, 2014 at 0:05
  • Oh - the missing forward might be it.
    – kfsone
    Feb 1, 2014 at 0:05
  • I might be wrong, but shouldn't it be template<typename T, typename... Args> Message(id_t messageId, Args&&... args)? Doesn't seem to compile otherwise.
    – user1508519
    Feb 1, 2014 at 0:09
  • @remyabel Oops, should be fixed.
    – kfsone
    Feb 1, 2014 at 0:21

2 Answers 2

3

You won't be able to elide the copy/move in the placement new: copy elision is entirely based on the idea that the compiler knows at construction time where the object will eventually end up. Also, since copy elision actually changes the behavior of the program (after all, it won't call the respective constructor and the destructor even if they have side-effects) copy elision is limited to a few very specific cases (listed in 12.8 [class.copy] paragraph 31: essentially when returning a local variable by name, when throwing a local variable by name, when catching an exception of the correct type by value, and when copying/moving a temporary variable; see the clause for exact details). Since [placement] new is none of the contexts where the copy can be elided and the argument to constructor is clearly not a temporary (it is named), the copy/move will never be elided. Even adding the missing std::forward<T>(...) to your constructor will cause the copy/move to be elided:

template<typename T>
Message(id_t messageId, T&& src)
    : Message(sizeof(T), messageID)
{
    // placement new take a void* anyway, i.e., no need to cast
    new (m_data) T (std::forward<T>(src));
}

I don't think you can explicitly specify a template parameter when calling a constructor. Thus, I think the closest you could probably get without constructing the object ahead of time and getting it copied/moved is something like this:

template <typename>
struct Tag {};

template <typename T, typename A>
Message::Message(Tag<T>, id_t messageId, A... args)
    : Message(messageId, sizeof(T)) {
    new(this->m_data) T(std::forward<A>(args)...);
}

One approach which might make things a bit nicer is using the id_t to map to the relevant type assuming that there is a mapping from message Ids to the relevant type:

typedef uint8_t id_t;
template <typename T, id_t id> struct Tag {};
struct MessageId {
    static constexpr Tag<MyStruct, 1> WorldPeace;
    // ...
};
template <typename T, id_t id, typename... A>
Message::Message(Tag<T, id>, A&&... args)
    Message(id, sizeof(T)) {
    new(this->m_data) T(std::forward<A>)(args)...);
}
0

Foreword

The conceptual barrier that even C++2049 cannot cross is that you require all the bits that compose your message to be aligned in a contiguous memory block.

The only way C++ can give you that is through the use of the placement new operator. Otherwise, objects will simply be constructed according to their storage class (on the stack or through whatever you define as a new operator).

It means any object you pass to your payload constructor will be first constructed (on the stack) and then used by the constructor (that will most likely copy-construct it).

Avoiding this copy completely is impossible. You may have a forward constructor doing the minimal amount of copy, but still the scalar parameters passed to the initializer will likely be copied, as will any data that the constructor of the initializer deemed necessary to memorize and/or produce.

If you want to be able to pass parameters freely to each of the constructors needed to build the complete message without them being first stored in the parameter objects, it will require

  • the use of a placement new operator for each of the sub-objects that compose the message,
  • the memorization of each single scalar parameter passed to the various sub-constructors,
  • specific code for each object to feed the placement new operator with the proper address and call the constructor of the sub-object.

You will end up with a toplevel message constructor taking all possible initial parameters and dispatching them to the various sub-objects constructors.

I don't even know if this is feasible, but the result would be very fragile and error-prone at any rate.

Is that what you want, just for the benefit of a bit of syntactic sugar?

If you're offering an API, you cannot cover all cases. The best approach is to make something that degrades nicely, IMHO.

The simple solution would be to limit payload constructor parameters to scalar values or implement "in-place sub-construction" for a limited set of message payloads that you can control. At your level you cannot do more than that to make sure the message construction proceeds with no extra copies.

Now the application software will be free to define constructors that take objects as parameters, and then the price to pay will be these extra copies.

Besides, this might be the most efficient approach, if the parameter is something costly to construct (i.e. the construction time is greater than the copy time, so it is more efficient to create a static object and modify it slightly between each message) or if it has a greater lifetime than your function for any reason.

a working, ugly solution

First, let's start with a vintage, template-less solution that does in-place construction.

The idea is to have the message pre-allocate the right kind of memory (local buffer of dynamic) depending on the size of the object.
The proper base address is then passed to a placement new to construct the message contents in place.

#include <cstdint>
#include <cstdio>
#include <new>

typedef uint8_t id_t;
enum class MessageID { WorldPeace, Armaggedon };

#define SMALL_BUF_SIZE 64

class Message {
    id_t     m_messageId;
    uint8_t* m_data;
    uint8_t  m_localData[SMALL_BUF_SIZE];

public:

    // choose the proper location for contents
    Message (MessageID messageId, size_t size)
    {
        m_messageId = (id_t)messageId;
        m_data = size <= SMALL_BUF_SIZE ? m_localData : new uint8_t[size];
    }

    // dispose of the contents if need be
    ~Message ()
    {
        if (m_data != m_localData) delete m_data;
    }

    // let placement new know about the contents location
    void * location (void)
    {
        return m_data;
    }
};

// a macro to do the in-place construction
#define BuildMessage(msg, id, obj, ...   )       \
        Message msg(MessageID::id, sizeof(obj)); \
        new (msg.location()) obj (__VA_ARGS__);  \

// example uses
struct small {
    int a, b, c;
    small (int a, int b, int c) :a(a),b(b),c(c) {}
};
struct big {
    int lump[1000];
};

int main(void)
{
    BuildMessage(msg1, WorldPeace, small, 1, 2, 3)
    BuildMessage(msg2, Armaggedon, big)
}

This is just a trimmed down version of your initial code, with no templates at all.

I find it relatively clean and easy to use, but to each his own.

The only inefficiency I see here is the static allocation of 64 bytes that will be useless if the message is too big.

And of course all type information is lost once the messages are constructed, so accessing their contents afterward would be awkward.

About forwarding and construction in place

Basically, the new && qualifier does no magic. To do in-place construction, the compiler needs to know the address that will be used for object storage before calling the constructor.

Once you've invoked an object creation, the memory has been allocated and the && thing will only allow you to use that address to pass ownership of the said memory to another object without resorting to useless copies.

You can use templates to recognize a call to the Message constructor involving a given class passed as message contents, but that will be too late: the object will have been constructed before your constructor can do anything about its memory location.

I can't see a way to create a template on top of the Message class that would defer an object construction until you have decided at which location you want to construct it.

However, you could work on the classes defining the object contents to have some in-place construction automated.

This will not solve the general problem of passing objects to the constructor of the object that will be built in place.

To do that, you would need the sub-objects themselves to be constructed through a placement new, which would mean implementing a specific template interface for each of the initializers, and have each object provide the address of construction to each of its sub-objects.

Now for syntactic sugar.

To make the ugly templating worth the while, you can specialize your message classes to handle big and small messages differently.

The idea is to have a single lump of memory to pass to your sending function. So in case of small messages, the message header and contents are defined as local message properties, and for big ones, extra memory is allocated to include the message header.

Thus the magic DMA used to propell your messages through the system will have a clean data block to work with either way.

Dynamic allocations will still occur once per big message, and never for small ones.

#include <cstdint>
#include <new>

// ==========================================================================
// Common definitions
// ==========================================================================

// message header
enum class MessageID : uint8_t { WorldPeace, Armaggedon };
struct MessageHeader {
    MessageID id;
    uint8_t   __padding; // one free byte here
    uint16_t  size;
};

// small buffer size
#define SMALL_BUF_SIZE 64

// dummy send function
int some_DMA_trick(int destination, void * data, uint16_t size);

// ==========================================================================
// Macro solution
// ==========================================================================

// -----------------------------------------
// Message class
// -----------------------------------------
class mMessage {
    // local storage defined even for big messages
    MessageHeader   m_header;
    uint8_t         m_localData[SMALL_BUF_SIZE];

    // pointer to the actual message
    MessageHeader * m_head;
public:  
    // choose the proper location for contents
    mMessage (MessageID messageId, uint16_t size)
    {
        m_head = size <= SMALL_BUF_SIZE 
            ? &m_header
            : (MessageHeader *) new uint8_t[size + sizeof (m_header)];
        m_head->id   = messageId;
        m_head->size = size;
   }

    // dispose of the contents if need be
    ~mMessage ()
    {
        if (m_head != &m_header) delete m_head;
    }

    // let placement new know about the contents location
    void * location (void)
    {
        return m_head+1;
    }

    // send a message
    int send(int destination)
    {
        return some_DMA_trick (destination, m_head, (uint16_t)(m_head->size + sizeof (m_head)));
    }
};

// -----------------------------------------
// macro to do the in-place construction
// -----------------------------------------
#define BuildMessage(msg, obj, id, ...   )       \
        mMessage msg (MessageID::id, sizeof(obj)); \
        new (msg.location()) obj (__VA_ARGS__);  \

// ==========================================================================
// Template solution
// ==========================================================================
#include <utility>

// -----------------------------------------
// template to check storage capacity
// -----------------------------------------
template<typename T>
struct storage
{
    enum { local = sizeof(T)<=SMALL_BUF_SIZE };
};

// -----------------------------------------
// base message class
// -----------------------------------------
class tMessage {
protected:
    MessageHeader * m_head;
    tMessage(MessageHeader * head, MessageID id, uint16_t size) 
        : m_head(head)
    {
        m_head->id = id;
        m_head->size = size;
    }
public:
    int send(int destination)
    {
        return some_DMA_trick (destination, m_head, (uint16_t)(m_head->size + sizeof (*m_head)));
    }
};

// -----------------------------------------
// general message template
// -----------------------------------------
template<bool local_storage, typename message_contents>
class aMessage {};

// -----------------------------------------
// specialization for big messages
// -----------------------------------------
template<typename T>
class aMessage<false, T> : public tMessage
{
public:
    // in-place constructor
    template<class... Args>
    aMessage(MessageID id, Args...args) 
        : tMessage(
            (MessageHeader *)new uint8_t[sizeof(T)+sizeof(*m_head)], // dynamic allocation
            id, sizeof(T))
    {
        new (m_head+1) T(std::forward<Args>(args)...);
    }

    // destructor
    ~aMessage ()
    {
        delete m_head;
    }

    // syntactic sugar to access contents
    T& contents(void) { return *(T*)(m_head+1); }
};

// -----------------------------------------
// specialization for small messages
// -----------------------------------------
template<typename T>
class aMessage<true, T> : public tMessage
{
    // message body defined locally
    MessageHeader m_header;
    uint8_t       m_data[sizeof(T)]; // no need for 64 bytes here

public:
    // in-place constructor
    template<class... Args>
    aMessage(MessageID id, Args...args) 
        : tMessage(
            &m_header, // local storage
            id, sizeof(T))
    {
        new (m_head+1) T(std::forward<Args>(args)...);
    }

    // syntactic sugar to access contents
    T& contents(void) { return *(T*)(m_head+1); }
};


// -----------------------------------------
// helper macro to hide template ugliness
// -----------------------------------------
#define Message(T) aMessage<storage<T>::local, T>
// something like typedef aMessage<storage<T>::local, T> Message<T>

// ==========================================================================
// Example
// ==========================================================================
#include <cstdio>
#include <cstring>

// message sending
int some_DMA_trick(int destination, void * data, uint16_t size)
{
    printf("sending %d bytes @%p to %08X\n", size, data, destination);
    return 1;
}

// some dynamic contents
struct gizmo {
    char * s;
    gizmo(void) { s = nullptr; };
    gizmo (const gizmo&  g) = delete;

    gizmo (const char * msg)
    {
        s = new char[strlen(msg) + 3];
        strcpy(s, msg);
        strcat(s, "#");
    }

    gizmo (gizmo&& g)
    {
        s = g.s;
        g.s = nullptr;
        strcat(s, "*");
    }

    ~gizmo() 
    { 
        delete s;
    }

    gizmo& operator=(gizmo g)
    {
        std::swap(s, g.s);
        return *this;
    }
    bool operator!=(gizmo& g)
    {
        return strcmp (s, g.s) != 0;
    }

};

// some small contents
struct small {
    int a, b, c;
    gizmo g;
    small (gizmo g, int a, int b, int c)
        : a(a), b(b), c(c), g(std::move(g)) 
    {
    }

    void trace(void) 
    { 
        printf("small: %d %d %d %s\n", a, b, c, g.s);
    }
};

// some big contents
struct big {
    gizmo lump[1000];

    big(const char * msg = "?")
    { 
        for (size_t i = 0; i != sizeof(lump) / sizeof(lump[0]); i++)
            lump[i] = gizmo (msg);
    }

    void trace(void)
    {
        printf("big: set to ");
        gizmo& first = lump[0];
        for (size_t i = 1; i != sizeof(lump) / sizeof(lump[0]); i++)
            if (lump[i] != first) { printf(" Erm... mostly "); break; }
        printf("%s\n", first.s);
    }
};

int main(void)
{
    // macros
    BuildMessage(mmsg1, small, WorldPeace, gizmo("Hi"), 1, 2, 3);
    BuildMessage(mmsg2, big  , Armaggedon, "Doom");
    ((small *)mmsg1.location())->trace();
    ((big   *)mmsg2.location())->trace();
    mmsg1.send(0x1000);
    mmsg2.send(0x2000);

    // templates
    Message (small) tmsg1(MessageID::WorldPeace, gizmo("Hello"), 4, 5, 6);
    Message (big  ) tmsg2(MessageID::Armaggedon, "Damnation");
    tmsg1.contents().trace();
    tmsg2.contents().trace();
    tmsg1.send(0x3000);
    tmsg2.send(0x4000);
}

output:

small: 1 2 3 Hi#*
big: set to Doom#
sending 20 bytes @0xbf81be20 to 00001000
sending 4004 bytes @0x9e58018 to 00002000
small: 4 5 6 Hello#**
big: set to Damnation#
sending 20 bytes @0xbf81be0c to 00003000
sending 4004 bytes @0x9e5ce50 to 00004000

Arguments forwarding

I see little point in doing constructor parameters forwarding here.

Any bit of dynamic data referenced by the message contents would have to be either static or copied into the message body, otherwise the referenced data would vanish as soon as the message creator would go out of scope.

If the users of this wonderfully efficient library start passing around magic pointers and other global data inside messages, I wonder how the global system performance will like that. But that's none of my business, after all.

Macros

I resorted to a macro to hide the template ugliness in type definition.

If someone has an idea to get rid of it, I'm interested.

Efficiency

The template variation requires an extra forwarding of the contents parameters to reach the constructor. I can't see how that could be avoided.

The macro version wastes 68 bytes of memory for big messages, and some memory for small ones (64 - sizeof (contents object)).

Performance-wise, this extra bit of memory is the only gain the templates offer. Since all these objects are supposedly constructed on the stack and live for a handful of microseconds, it is pretty neglectible.

Compared to your initial version, this one should handle message sending more efficiently for big messages. Here again, if these messages are rare and only offered for convenience, the difference is not terribly useful.

The template version maintains a single pointer to the message payload, that could be spared for small messages if you implemented a specialized version of the send function.
Hardly worth the code duplication, IMHO.

A last word

I think I know pretty well how an operating system works and what performances concerns might be. I wrote quite a few real-time applications, plus some drivers and a couple of BSPs in my time.

I also saw more than once a very efficient system layer ruined by too permissive an interface that allowed application software programmers to do the silliest things without even knowing.
That is what triggered my initial reaction.

If I had my say in global system design, I would forbid all these magic pointers and other under-the-hood mingling with object references, to limit non-specialist users to an inoccuous use of system layers, instead of allowing them to inadvertently spread cockroaches through the system.

Unless the users of this interface are template and real-time savvies, they will not understand a bit what is going on beneath the syntactic sugar crust, and might very soon shoot themselves (and their co-workers and the application software) in the foot.

Suppose a poor application software programmer adds a puny field in one of its structs and crosses unknowingly the 64 bytes barrier. All of a sudden the system performance will crumble, and you will need Mr template & real time expert to explain the poor guy that what he did killed a lot of kittens.
Even worse, the system degradation might be progressive or unnoticeable at first, so one day you might wake up with thousands of lines of code that did dynamic allocations for years without anybody noticing, and the global overhaul to correct the problem might be huge.

If, on the other hand, all people in your company are munching at templates and mutexes for breakfast, syntactic sugar is not even required in the first place.

16
  • So, yeah. Not really sure where to start. So I'll just pick the most obvious and egregious. if (m_data == m_localData) m_data = new uint8_t[new_size]; ... else m_data = (uint8_t*) realloc (m_data, new_size);. Lets trim that down: if X new ELSE realloc and the ternary variant I wrote allows GCC, ICC, Clang and MSVC to use conditional assignment rather than a branch. And how do you consider allocating an object with "new" and memcpying it into the message not a copy? Plus the minor point that your code doesn't provide for in-place construction.
    – kfsone
    Feb 5, 2014 at 7:55
  • In my sscce I accidentally left grow as public - I didn't see the point in specifically going and marking it private for an sscce. In terms of the default "local buffer", the actual code pools these objects and a large percentage of the messages, especially in high pressure situations, are generally small enough to fit in those buffers. The overarching reason for the complexity in this system is the large number of downstream users. The intent here is to work with the compiler sufficiently so that the immediate users of the API don't have to learn this kind of optimization nastiness.
    – kfsone
    Feb 5, 2014 at 8:00
  • The grow() method is never called in the constructor. I only included it to match what I understood of the original request. Objects less than 64 bytes in size are indeed copied (which takes a neglectible time), but larger ones are not. If you want to "grow" the message afterwards (which is an ungodly thing to do IMHO), since all typing information is lost anyway, there is little choice but to resort to a copy.
    – kuroi neko
    Feb 5, 2014 at 8:01
  • Lastly, the system I'm working on is a refactoring of my own code. The system is incredibly stable, incredibly efficient network message passing library. It's been running for 10 years, the current revision for nearly 6. The complexity in the under-the-hood has largely prevented the devs working with it from making silly mistakes that kill the clients or servers due to networking, excepting a few cases, and I've had success in employing C++11 techniques to help the compiler work those out for them.
    – kfsone
    Feb 5, 2014 at 8:03
  • Your variant requires the caller to allocate an object and pass a pointer to it, and then memcpy()s the contents into the object. My variant marshals the data directly into the object. If we were constructing a X byte object with yours, you'd have to malloc X bytes of heap, populate them, then pass the address to Message::Message(), which would then either malloc() a second X bytes and transfer the object into that or it would copy it onto the heap.
    – kfsone
    Feb 5, 2014 at 8:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.