35

Is there a version of memset() which sets a value that is larger than 1 byte (char)? For example, let's say we have a memset32() function, so using it we can do the following:

int32_t array[10];
memset32(array, 0xDEADBEEF, sizeof(array));

This will set the value 0xDEADBEEF in all the elements of array. Currently it seems to me this can only be done with a loop.

Specifically, I am interested in a 64 bit version of memset(). Know anything like that?

1

7 Answers 7

39
void memset64( void * dest, uint64_t value, uintptr_t size )
{
  uintptr_t i;
  for( i = 0; i < (size & (~7)); i+=8 )
  {
    memcpy( ((char*)dest) + i, &value, 8 );
  }  
  for( ; i < size; i++ )
  {
    ((char*)dest)[i] = ((char*)&value)[i&7];
  }  
}

(Explanation, as requested in the comments: when you assign to a pointer, the compiler assumes that the pointer is aligned to the type's natural alignment; for uint64_t, that is 8 bytes. memcpy() makes no such assumption. On some hardware unaligned accesses are impossible, so assignment is not a suitable solution unless you know unaligned accesses work on the hardware with small or no penalty, or know that they will never occur, or both. The compiler will replace small memcpy()s and memset()s with more suitable code so it is not as horrible is it looks; but if you do know enough to guarantee assignment will always work and your profiler tells you it is faster, you can replace the memcpy with an assignment. The second for() loop is present in case the amount of memory to be filled is not a multiple of 64 bits. If you know it always will be, you can simply drop that loop.)

1
  • This implementation is more than I bargained for with the question :) Thanks! It would have been nice if you explained the implementation. For example, I can't understand why use a function call to memcpy() instead of an assignment.
    – gnobal
    Sep 21, 2008 at 8:20
10

There's no standard library function afaik. So if you're writing portable code, you're looking at a loop.

If you're writing non-portable code then check your compiler/platform documentation, but don't hold your breath because it's rare to get much help here. Maybe someone else will chip in with examples of platforms which do provide something.

The way you'd write your own depends on whether you can define in the API that the caller guarantees the dst pointer will be sufficiently aligned for 64-bit writes on your platform (or platforms if portable). On any platform that has a 64-bit integer type at all, malloc at least will return suitably-aligned pointers.

If you have to cope with non-alignment, then you need something like moonshadow's answer. The compiler may inline/unroll that memcpy with a size of 8 (and use 32- or 64-bit unaligned write ops if they exist), so the code should be pretty nippy, but my guess is it probably won't special-case the whole function for the destination being aligned. I'd love to be corrected, but fear I won't be.

So if you know that the caller will always give you a dst with sufficient alignment for your architecture, and a length which is a multiple of 8 bytes, then do a simple loop writing a uint64_t (or whatever the 64-bit int is in your compiler) and you'll probably (no promises) end up with faster code. You'll certainly have shorter code.

Whatever the case, if you do care about performance then profile it. If it's not fast enough try again with more optimisation. If it's still not fast enough, ask a question about an asm version for the CPU(s) on which it's not fast enough. memcpy/memset can get massive performance increases from per-platform optimisation.

1
  • @Steve Jessop, Please explain to me the 64-bit Windows or Linux alignment considerations.
    – Frank
    Aug 8, 2015 at 8:30
8

Just for the record, the following uses memcpy(..) in the following pattern. Suppose we want to fill an array with 20 integers:

--------------------

First copy one:
N-------------------

Then copy it to the neighbour:
NN------------------

Then copy them to make four:
NNNN----------------

And so on:
NNNNNNNN------------

NNNNNNNNNNNNNNNN----

Then copy enough to fill the array:
NNNNNNNNNNNNNNNNNNNN

This takes O(lg(num)) applications of memcpy(..).

int *memset_int(int *ptr, int value, size_t num) {
    if (num < 1) return ptr;
    memcpy(ptr, &value, sizeof(int));
    size_t start = 1, step = 1;
    for ( ; start + step <= num; start += step, step *= 2)
        memcpy(ptr + start, ptr, sizeof(int) * step);

    if (start < num)
        memcpy(ptr + start, ptr, sizeof(int) * (num - start));
    return ptr;
}

I thought it might be faster than a loop if memcpy(..) was optimised using some hardware block memory copy functionality, but it turns out that a simple loop is faster than the above with -O2 and -O3. (At least using MinGW GCC on Windows with my particular hardware.) Without the -O switch, on a 400 MB array the code above is about twice as fast as an equivalent loop, and takes 417 ms on my machine, while with optimisation they both go to about 300 ms. Which means that it takes approximately the same number of nanoseconds as bytes, and a clock cycle is about a nanosecond. So either there is no hardware block memory copy functionality on my machine, or the memcpy(..) implementation does not take advantage of it.

3
  • 1
    Modern processors can run a simple loop fast enough to saturate the memory bus, making block move/copy instructions redundant. Oct 28, 2014 at 22:30
  • 1
    @MarkRansom I was hoping there was some single-instruction way to e.g. set a whole page to all-zeros. Because it's conceivable that a given design of a RAM chip will allow to add that functionality in for free. But it's so much easier to check whether that's happening than to find the specification for the technology, or even the correct technical term for this. Oct 30, 2014 at 1:27
  • @evgeniSergeev, you could ask the OS to remap the page to zeros, which on some platforms may be faster than doing memory accesses. Also this assumes the OS has some pre-zeroed pages to hand you.
    – rsaxvc
    Feb 23, 2018 at 17:26
6

Check your OS documentation for a local version, then consider just using the loop.

The compiler probably knows more about optimizing memory access on any particular architecture than you do, so let it do the work.

Wrap it up as a library and compile it with all the speed improving optimizations the compiler allows.

5

wmemset(3) is the wide (16-bit) version of memset. I think that's the closest you're going to get in C, without a loop.

1
  • 10
    -1 for 16-bit. It's wchar_t which is 32-bit on any implementation that supports Unicode properly. It's only 16-bit on windows, which ignores the C standard and stores UTF-16 in wchar_t. Aug 7, 2010 at 18:45
2

If you're just targeting an x86 compiler you could try something like (VC++ example):

inline void memset32(void *buf, uint32_t n, int32_t c)
{
  __asm {
  mov ecx, n
  mov eax, c
  mov edi, buf
  rep stosd
  }
}

Otherwise just make a simple loop and trust the optimizer to know what it's doing, just something like:

for(uint32_t i = 0;i < n;i++)
{
  ((int_32 *)buf)[i] = c;
}

If you make it complicated chances are it will end up slower than simpler to optimize code, not to mention harder to maintain.

1
  • 2
    I'd throw a cld in there to make sure you aren't iterating backwards by accident. Also, worth noting that ecx is the number of qwords, not bytes, so the original question would be wrong to pass sizeof array here.
    – riv
    Jul 1, 2015 at 10:27
1

You should really let the compiler optimize this for you as someone else suggested. In most cases that loop will be negligible.

But if this some special situation and you don't mind being platform specific, and really need to get rid of the loop, you can do this in an assembly block.

//pseudo code
asm
{
    rep stosq ...
}

You can probably google stosq assembly command for the specifics. It shouldn't be more than a few lines of code.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.