6

I'm working with integers and SSE and have become very confused about how endianness affects moving data in and out of registers.

My initial, wrong, understanding

Initially my understanding was as follows. If I have an array of 4 byte integers the memory would be laid out as follows since x86 architectures are little endian:

0D 0C 0B 0A 1D 1C 1B 1A 2D 2C 2B 2A .... nD nC nB nA

Where the letters A, B, C and D index the bytes within an integer element, and numbers index the element.

In an XMM register, my understanding is that four integers would be laid out as follows:

0A 0B 0C 0D 1A 1B 1C 1D 2A 2B 2C 2D 3A 3B 3C 3D

However, I'm pretty sure this picture is wrong for several reasons. The first is the documentation for the mm_load_si128 intrinsic, which is supposed to work for any integer data, but in the above picture should only work for one word size. Similarly there is this (archived) thread. Finally I see people writing code like the following:

__declspec(align(16)) int32_t A[N];
__m128i* As = (__m128i*)A;

A potentially correct picture

The Wikipedia article on endianness says I should think of memory addresses increasing right to left. How about the following picture for memory then?

nA nB nC nD ... 2A 2B 2C 2D 1A 1B 1C 1D 0A 0B 0C 0D

And then in a register:

3A 3B 3C 3D 2A 2B 2C 2D 1A 1B 1C 1D 0A 0B 0C 0D
7
  • 2
    The easiest way to think about this is that the byte ordering of the whole XMM register is little endian. So the least significant byte of the whole 16 bytes is byte 0. Individual elements are also little endian. Note also that we only care about endianness with SIMD when doing any horizontal operations, such as packing or unpacking - the rest of the time it makes no difference.
    – Paul R
    Jun 5, 2014 at 7:57
  • @PaulR, if you have four 32-bit integers or eight 16-bit shorts what is the least significant byte? I would have expected for 32-bit ints the 128-bit register is 0A0B0C0D 1A1B1C1D 2A2B2C2D 3A3B3C3D (or 3A3B3C3D...0A0B0C0D). But the OP makes a good point that since the way to load integers values (8-bit, 16-bit, 32-bit, or 64-bit) is with one instruction which does not depend on the size of the integer then it must fill them the same way independent of the integer size. So how is it represented in the register?
    – Z boson
    Jun 5, 2014 at 8:27
  • If you think of little endian byte ordering as always starting at the lowest address, which corresponds to byte 0 in the XMM register, then for 32 bits ints the first int is as 0..3, the second int at 4..7, etc. For 16 bits shorts the first short is at 0..1, the second short is at 2..3, etc. So the ordering of the vector, and the ordering of the elements is the same - the least significant byte is at the lowest index within the element/vector. If I can find the time I will draw some diagrams and post it as an answer.
    – Paul R
    Jun 5, 2014 at 8:32
  • I don't see how it's possible for mm_load_si128 to translate from memory to 0..3 4..7 or 0..1 2..3 unless it knows the size of the integers that it's loading. If there was a separate instruction for each possible integer size (like there is for float and double) this would make sense but there's only one instruction.
    – Z boson
    Jun 5, 2014 at 9:53
  • 1
    @PaulR, BTW, I don't think there is any sense talking about the Endianness of a register it only makes sense for memory addresses. You said "whole XMM register is little endian. So the least significant byte of the whole 16 bytes is byte 0". But that's true of a 32-bit register as well (the least significant byte is byte 0) but I don't think anyone would call a 32-bit register little endian. You write the digits of a 32-bit register on a little endian and big endian system the same way: bit-32, bit-31,...bit-0. Endianness only matters on how the register is read/written to memory.
    – Z boson
    Jun 5, 2014 at 13:37

2 Answers 2

7

Elaborating on Peter's comment, there is most certainly an implicit endianness inherent to the SIMD implementation, owing to the fact that there are instructions that can:

  • Read and write 128-bit SIMD registers from memory. Since memory must always be accessed by byte offset (regardless of the instruction or how many bytes it stores or fetches), and can subsequently be examined by other non-SIMD means, the movaps, movdqa, movdqu, etc. instructions inherently imply an endianness.
  • Index vector elements with instructions like pshufd and even runtime-variable indexing with pshufb that use an integer index to select elements. This means that elements have something like addresses, and wide elements contain multiple independently-addressable narrow elements. (Not part of memory address space, of course, but unlike scalar registers we have a 2nd way to talk about position other than left/right shift within a wide element. This is the same thing that makes endianness an issue for memory.) The indexing of elements within a register is chosen to match the order in memory (little-endian), but it could have been different.
  • Shift bits across byte boundaries of a SIMD register with pslld, psrld, or whole-vector byte-shifts like pslldq etc. Note that crossing "byte boundaries" includes within the individual word, dword, or qword components, because (for the same reason noted in the previous point), the register can be subsequently be imaged to memory. A byte-shift of a whole vector groups the low byte of one word with the high byte of the adjacent word, in a way that depends on endianness.
  • Re-interpret the component size (byte, word, dword, or qword) of an existing SIMD register's contents. This is the analogue of reading the bytes of a dword in memory: they have an order. Shuffling around qwords using pshufd requires you to consider the endianness when choosing the shuffle control, to keep the right high:low pairs of dwords grouped in the right order.

So while it's true that if you never do any of these things, meaning you exclusively use SIMD memory images with SIMD registers, with matching component sizes and never examine that memory otherwise, and also maintain consistent component sizes in operations on those SIMD registers, then you don't have to worry about SIMD endianness. Otherwise, read on...

Knowing now that the SIMD operations listed above expose an endianness, what then is it? Well, we already know that Intel architecture is little-endian, meaning word, dword, and qword (respectively, 16-, 32-, and 64-bit memory accesses) are recursively swapped. For example, storing a single qword swaps the stores of its two dwords, each of which swaps its two words, each of which swaps its two bytes. This results in the memory image of a CPU register having a reversed byte order overall.

For compatibility with non-SIMD instructions operating on the same size, the memory image of each individual component of a SIMD register should be bit-identical, for every component size, with existing (little-endian) format. The prior existence of non-SIMD instructions for word, dword, and qword accesses thus represent hard constraints, and those SIMD components must manifest little-endian images.

But there are no prior non-SIMD instructions for 128-bit memory access, so there isn't a prior constraint on the (qword, qword) layout of the dqword SIMD register itself. That leaves really just the one possible question we could be asking here: does the recursive little-endian swapping pattern (word, dword, qword, ...?) continue, applying to dqword values as well? In other words, in the 16-byte memory image of a SIMD register, which of its two qword components—the numerically least-significant, or the more-significant—is stored in the lower-addressed 8-bytes?

ANSWER: The least-significant qword is stored at the l̲o̲w̲e̲r̲-address 8-bytes.

This preserves the symmetry of the "little-endian" recursive swapping, by extending the pattern to include dqword values as well. To summarize, a 128-bit SIMD register is little-endian, because its memory image at [esi] has:

  • the less-significant qword (SIMD index 0) at the lower address qword ptr [esi],
  • the more-significant qword (SIMD index 1) at the higher address qword ptr [esi + 8].
2
  • Your edit bumped the question; that's what led me to look at it. This looks good, thanks for writing this up since I only took the time to comment, not answer. Jul 1, 2020 at 21:53
  • I did end up making an edit to highlight the fact that vector elements can be indexed at runtime. I think this is key to having an endianness for wider elements that span multiple indexable elements. Jul 1, 2020 at 22:13
6

It's just a question of interpretation. We read/write digits of a number from left to right and highest digit to lowest digit. So for a 32-bit number with the highest byte A then B then C and lowest byte D we would read/write ABCD. We do the same notating a 128-bit integer.

3A3B3C3D 2A2B2C2D 1A1B1C1D 0A0B0C0D

But in a little endian system it reads and writes digits from the lowest address to the highest like this

0D0C0B0A 1D1C1B1A 2D2C2B2A 3D3C3B3A

For 16-bit integers it's the same logic. We could read/write it as

7A7B 6A6B 5A5B 4A4B 3A3B 2A2B 1A1B 0A0B

and the little endian computer read/stores it from lowest to highest address as

0B0A 1B1A 2B2A 3B3A 4B4A 5B5A 6A6B 7B7A

That's why there is only one instruction to read/write 32-bit, 16-bit and 8-byte integers int a 128-bit register: namely movdqa and movaps (or the unaligned variants movdqu and movups).

3
  • 1
    SIMD elements have numbers, e.g. for pshufd / pshufb, and direction for pslld / psrld bit-shift within 32-bit elements vs. psl/rldq byte shifts, so SIMD does introduce the issue of endianness to registers: lowest numbered element at the left, loaded/stored to the lowest memory address. Scalar integer registers don't have an endianness and there it truly is just an issue of interpretation. Jul 1, 2020 at 9:09
  • Related: Convention for displaying vector registers Jul 1, 2020 at 9:12
  • I thought the little endian will have to be "0A0B0C0D 1A1B1C1D 2A2B2C2D 3A3B3C3D" why do you have the bit ordering also changed aswell? Nov 20, 2022 at 17:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.