26

As answered elsewhere, calling functions like memcpy with invalid or NULL pointers is undefined behaviour, even if the length argument is zero. In the context of such a function, especially memcpy and memmove, is a pointer just past the end of the array a valid pointer?

I'm asking this question because a pointer just past the end of an array is legal to obtain (as opposed to, e.g. a pointer two elements past the end of an array) but you are not allowed to dereference it, yet footnote 106 of ISO 9899:2011 indicates that such a pointer points to into the address space of the program, a criterion required for a pointer to be valid according to §7.1.4.

Such usage occurs in code where I want to insert an item into the middle of an array, requiring me to move all items after the insertion point:

void make_space(type *array, size_t old_length, size_t index)
{
    memmove(array + index + 1, array + index, (old_length - index) * sizeof *array);
}

If we want to insert at the end of the array, index is equal to length and array + index + 1 points just past the end of the array, but the number of copied elements is zero.

9
  • 1
    It still seems UB to me. array+index+1 may or may not be a valid address. You probably won't have any problems with many implementations but you still can't depend on it.
    – holgac
    Apr 24, 2015 at 9:57
  • 1
    @holgac The standard explicitly says that it's legal to create and use a pointer just past the end of an object, but you are not allowed to dereference it. The wording in the standard seems unclear in this regard.
    – fuz
    Apr 24, 2015 at 10:00
  • @FUZxxl You can also point to NULL but not dereference. I think that sentence is meaningless. I mean, I can perfectly point to 0x12345678 and I won't have a single problem until I dereference it. Anyway, if memcpy or memmove implementation performs some cache-friendly operation before actually checking length, they would dereference your pointer and boom.
    – holgac
    Apr 24, 2015 at 10:03
  • 2
    @holgac That is implementation defined behaviour in general. For the specific case where the value stored in the long is equal to a value obtained by casting a valid pointer into an uintptr_t, it is well-defined.
    – fuz
    Apr 24, 2015 at 10:10
  • 3
    @holgac I think I understand your confusion now. Casting the integer 0x12345678 is implementation defined behaviour. If the resulting pointer is invalid, your program exhibits undefined behaviour and may crash immediately, independently of whether you dereference the pointer or not. This is because some machines allow you to place only valid pointers in pointer registers.
    – fuz
    Apr 24, 2015 at 10:48

3 Answers 3

10

3.15 object

  1. object region of data storage in the execution environment, the contents of which can represent values

The memory, pointer to one past the last element points to, of an array object or an object cannot represent values, since it cannot be dereferenced ( 6.5.6 Additive operators, paragraph 8 ).

7.24.2.1 The memcpy function

  1. The memcpy function copies n characters from the object pointed to by s2 into the object pointed to by s1. If copying takes place between objects that overlap, the behavior is undefined.

Pointers passed to memcpy must point to an object.

6.5.3.4 The sizeof and _Alignof operators

  1. When sizeof is applied to an operand that has type char, unsigned char, or signed char, (or a qualified version thereof) the result is 1. When applied to an operand that has array type, the result is the total number of bytes in the array. When applied to an operand that has structure or union type, the result is the total number of bytes in such an object, including internal and trailing padding.

sizeof operator doesn't count the one-past element as the object, since it doesn't count towards the size of the object. Yet it clearly gives the size of the entire object.

6.3.2.1 Lvalues, arrays, and function designators

  1. An lvalue is an expression (with an object type other than void) that potentially designates an object; 64) if an lvalue does not designate an object when it is evaluated, the behavior is undefined.

I argue that the one past pointer to an array object or an object, both of which are otherwise allowed to point to, does not represent an object.

int a ;
int* p = a+1 ; 

p is defined, but it does not point to an object since it cannot be dereferenced, the memory it points to cannot represent a value, and sizeof doesn't count that memory as a part of the object. Memcpy requires a pointer to an object.

Therefore the passing one past pointer to memcpy causes undefined behavior.

Update:

This part also support the conclusion:

6.5.9 Equality operators

  1. Two pointers compare equal if and only if both are null pointers, both are pointers to the same object (including a pointer to an object and a subobject at its beginning) or function, both are pointers to one past the last element of the same array object, or one is a pointer to one past the end of one array object and the other is a pointer to the start of a different array object that happens to immediately follow the first array object in the address space.

This implies that pointer to an object if incremented to one past an object, can point to a different object. In that case, it certainly cannot point to the object it pointed to originally, showing that pointer one past an object doesn't point to an object.

1
  • 1
    This answer is very precise.
    – fuz
    Apr 24, 2015 at 16:58
10

Passing the past the end pointer to the first argument of memmove has several pitfalls, probably resulting in a nasal demon attack. Strictly speaking, there is no impermeable guarantee for that to be well defined.

(Unfortunatelly, there is not much information about the "past the last element" conecpt in the standard.)

Note: Sorry about having the other direction now...

The question basicially is whether the "one past the end pointer" is a valid first function argument for memmove if 0 bytes are moved:

T array[length];
memmove(array + length, array + length - 1u, 0u);

The requirement in question is the validity of the first argument.

N1570, 7.1.4, 1

If a function argument is described as being an array, the pointer actually passed to the function shall have a value such that all address computations and accesses to objects (that would be valid if the pointer did point to the first element of such an array) are in fact valid.

If an argument to a function has an invalid value (such as a value outside the domain of the function, or a pointer outside the address space of the program, or a null pointer, or a pointer to non-modifiable storage when the corresponding parameter is not const-qualified) or a type (after promotion) not expected by a function with variable number of arguments, the behavior is undefined.

Making the argument valid if the pointer

  1. is not outside the address space,
  2. is not a null pointer,
  3. is not a pointer to const memory

and if the argument type

  1. is not of array type.

1. Address space

N1570, 6.5.6, 8

Moreover, if the expression P points to the last element of an array object, the expression (P)+1 points one past the last element of the array object, and if the expression Q points one past the last element of an array object, the expression (Q)-1 points to the last element of the array object.

N1570, 6.5.6, 9

Moreover, if the expression P points either to an element of an array object or one past the last element of an array object, and the expression Q points to the last element of the same array object, the expression ((Q)+1)-(P) has the same value as ((Q)-(P))+1 and as -((P)-((Q)+1)), and has the value zero if the expression P points one past the last element of the array object, even though the expression (Q)+1 does not point to an element of the array object.106

106 Another way to approach pointer arithmetic is first to convert the pointer(s) to character pointer(s): In this scheme the integer expression added to or subtracted from the converted pointer is first multiplied by the size of the object originally pointed to, and the resulting pointer is converted back to the original type. For pointer subtraction, the result of the difference between the character pointers is similarly divided by the size of the object originally pointed to.

When viewed in this way, an implementation need only provide one extra byte (which may overlap another object in the program) just after the end of the object in order to satisfy the "one past the last element" requirements.

Eventhough the footnote is not normative -as pointed out by Lundin- we have an explanation here that "an implementation need only provide one extra byte". Although, I can't proove by quoting I suspect that this is a hint that the standard means to require the implementation to included memory inside of the programs address space at the location pointed to by the past the end pointer.

2. Null Pointer

The past the end pointer is not a null pointer.

3. Pointing to const memory

The standard imposes no further requirements on the past the end pointer other than giving some information about the result of several operations and the (again non-normaltive ;)) footnote clarifies that it can overlap with another object. Thus, there is no guarantee that the memory the past the end pointer points at is non constant. Since the first argument of memove is a pointer to non-constant memory, passing the past the end pointer is not guaranteed to be valid and potentially undefined behaviour.

4. Validity of array arguments

Chapter 7.21.1 describes the string handling header <string.h> and the first clause states:

The header declares one type and several functions, and defines one macro useful for manipulating arrays of character type and other objects treated as arrays of character type.

I don't think that the standard is very clear here whether the "objects treated as arrays of character type" refers to the functions or to the macro only. If this sentence actually implies that memove treats the first argument as an array of characters, the behaviour of passing the past the end pointer to memmove is undefined behaviour as per 7.1.4 (which requires a pointer to a valid object).

9
  • It would be useful to also quote the text " If an argument to a function has an invalid value (such as a value outside the domain of the function, or a pointer outside the address space of the program,or a null pointer, or a pointer to non-modifiable storage when the corresponding parameter is not const-qualified)". The start of your answer obviously refers to that section but without showing the text so it is a bit confusing.
    – M.M
    Apr 24, 2015 at 11:54
  • 1
    Foot notes in ISO standards are never normative, so you can completely disregard note 106.
    – Lundin
    Apr 24, 2015 at 11:54
  • 2
    @Lundin they can be used as a clue to the intent of the authors when the standard has unclear or defective text
    – M.M
    Apr 24, 2015 at 11:56
  • 2
    One could argue that this extra byte is not writeable since dereferencing it is undefined behaviour, ergo writing to it is undefined behaviour.
    – fuz
    Apr 24, 2015 at 12:11
  • 1
    @Pixelchemist: I find the analysis of such minutiae interesting, given that the only plausible scenario where such a thing could matter would be if a compiler went to the trouble of inferring that such a thing could occur for the express purpose of being able to ignore its normal obligations (and--with 99% certainty--programmer intent!) in such a case. I hope a group with some clout is able to spin off a variation of the C standard which would largely mirror the old one but would define silly corner cases like this so programmers wouldn't have to worry about them.
    – supercat
    Apr 24, 2015 at 16:21
6

If we look at the C99 standard, there is this:

7.21.1.p2

Where an argument declared as size_t n specifies the length of the array for a function, n can have the value zero on a call to that function. Unless explicitly stated otherwise in the description of a particular function in this subclause, pointer arguments on such a call shall still have valid values, as described in 7.1.4. On such a call, a function that locates a character finds no occurrence, a function that compares two character sequences returns zero, and a function that copies characters copies zero characters. ...

There is no explicit statement in the description of memcpy in 7.21.2.1

7.1.4.p1

... If a function argument is described as being an array, the pointer actually passed to the function shall have a value such that all address computations and accesses to objects (that would be valid if the pointer did point to the first element of such an array) are in fact valid.

Emphasis added. It seems the pointers have to point to valid locations (in the sense of dereferencing), and the paragraphs about pointer arithmetic allowing to point to the end + 1 do not apply here.

There is the question if the arguments to memcpy are arrays or not. Of course they are not declared as arrays, but

7.21.1.p1 says

The header string.h declares one type and several functions, and defines one macro useful for manipulating arrays of character type and other objects treated as arrays of character type.

and memcpy is in string.h.
So I would assume memcpy does treat the arguments as arrays of characters. Because the macro mentioned is NULL, the "useful for..." part of the sentence clearly applies to the functions.

14
  • 2
    The phrase “all address computations and accesses to objects” seems to pertain the address computations and accesses done by the function you pass the pointer to. Since memmove with a zero argument is not allowed to dereference the pointer (as it moves zero bytes), the valid accesses are an empty set. The NULL pointer is still excluded by the previous sentence. I might be wrong with this notion though.
    – fuz
    Apr 24, 2015 at 11:01
  • I'm not sure that memmove is not allowed to dereference the pointer when the size is 0. Is it described like this somewhere?
    – alain
    Apr 24, 2015 at 11:05
  • 1
    All of the cited text refers to the case of void func (size_t n, int [n]) so I don't see how this is relevant or answers the question.
    – Lundin
    Apr 24, 2015 at 11:42
  • 1
    @Lundin That's not right... The preamble to C11 draft 1570's section on string.h states (7.24.1.1): The header <string.h> declares one type and several functions, and defines one macro useful for manipulating arrays of character type and other objects treated as arrays of character type. 307) The type is size_t and the macro is NULL (both described in 7.19). Various methods are used for determining the lengths of the arrays, but in all cases a char * or void * argument points to the initial (lowest addressed) character of the array. Apr 24, 2015 at 12:17
  • 1
    @Lundin Furthermore, §7.24.1/2 says: “Where an argument declared as size_t n specifies the length of the array for afunction, n can have the value zero on a call to that function. [...] On such a call, [...] a function that copies characters copies zero characters.” Furthermore, footnote 189 affirms that memcpy is not allowed to copy bytes beyond the specified length even if it undoes the effect afterwards with respect to concurrency.
    – fuz
    Apr 24, 2015 at 12:20

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.