16

It seems to be a recurring problem that many Intel processors (up until Skylake, unless I'm wrong) exhibit poor performance when mixing AVX-256 instructions with SSE instructions.

According to Intel's documentation, this is caused by SSE instructions being defined to preserve the upper 128 bits of the YMM registers, so in order to be able to save power by not using the upper 128 bits of the AVX datapaths, the CPU stores those bits away when executing SSE code and reloads them when entering AVX code, the stores and loads being expensive.

However, I can find no obvious reason or explanation why SSE instructions needed to preserve those upper 128 bits. The corresponding 128-bit VEX instructions (the use of which avoids the performance penalty) work by always clearing the upper 128 bits of the YMM registers instead of preserving them. It seems to me that, when Intel defined the AVX architecture, including the extension of the XMM registers to YMM registers, they could have simply defined that the SSE instructions, too, would clear the upper 128 bits. Obviously, since the YMM registers were new, there could have been no legacy code that would have depended on SSE instructions preserving those bits, and it also appears to me that Intel could have easily seen this coming.

So, what is the reason why Intel defined the SSE instructions to preserve the upper 128 bits of the YMM registers? Is it ever useful?

2

2 Answers 2

14

In order to move external resources in-site, I've extracted the relevant paragraphs from the link Michael provided in the comments.

All credits go to him.
The link points to a very similar question Agner Fog asked on the Intel's forum.

[Fog in respone to Intel's answer] If I understand you right, you decided that it is necessary to have two versions of all 128-bit instructions in order to avoid destroying the upper part of the YMM registers in case an interrupt calls a device driver using legacy XMM instructions.

Intel were concerned that by making legacy SSE instructions zeroing the upper part of the XMM registers the ISRs would now suddenly affect the new YMM registers.
Without support for saving the new YMM context this would make the use of AVX impossible under any circumstances.

However Fog was not completely satisfied and pointed out that by simply recompiling a driver with an AVX aware compiler (so that VEX instruction were used) would result in the same outcome.

Intel replied that their goal was to avoid forcing existing software to be rewritten.

There is no way we could compel the industry to rewrite/fix all of their existing drivers (for example to use XSAVE) and no way to guarantee they would have done so successfully. Consider for example the pain the industry is still going through on the transition from 32 to 64-bit operating systems! The feedback we have from OS vendors also precluded adding overhead to the ISR servicing to add the state management overhead on every interrupt. We didn't want to inflict either of these costs on portions of the industry that don't even typically use wide vectors.

By having two versions of the instructions, support for AVX in drivers can be achieved like it has been for FPU/SSE:

The example given is similar to the current scenario where a ring-0 driver (ISR) vendor attempts to use floating-point state, or accidentally links it in some library, in OSs that do not automatically manage that context at Ring-0. This is a well known source of bugs and I can suggest only the following:

  • On those OSs, driver developers are discouraged from using floating-point or AVX

  • Driver developers should be encouraged to disable hardware features during driver validation (i.e. AVX state can be disabled by drivers in Ring-0 through XSETBV()

4
  • 1
    If I read Agner's comments correctly the reason the extra complexity is due to the 64-bit calling convention of Windows " The ABI for 64-bit Windows specifies that register XMM6 - XMM15 have callee-save status" and "extra complexity is made only for the sake of compatibility with existing x64 Windows device drivers. This problem could have been solved with very little performance cost by forcing Microsoft to implement the lazy saving solution I outlined above"...so it's Microsoft's fault.
    – Z boson
    Jan 25, 2017 at 10:24
  • 2
    @Zboson I agree. With Microsoft things work backwards: the hardware must accommodate the software. Anyway, feel free to edit the answer, it is a community wiki and your comment is a nice add-on :) Jan 25, 2017 at 10:35
  • @Zboson: not exactly. During an interrupt handler, all the YMM regs potentially hold valuable user-space state. Kernel code often avoids executing any FP or SIMD instructions, especially in an interrupt handler, so only the integer registers need to be saved/restored. Nov 23, 2018 at 18:03
  • A driver that wants to use some XMM regs in an interrupt handler might manually save/restore them instead of calling a kernel function to trigger save/restore. This is the source of the problem. Adding ISR overhead to always save/restore SIMD state is bad. See also the bottom of my answer on What is the penalty of mixing EVEX and VEX encoded scheme?/ Nov 23, 2018 at 18:04
1

Background: the decision was made early to make KeSaveFloatingPointState do nothing on Windows x64 and to allow XMM registers to be used without extra save/restore calls even in drivers. Obviously these drivers would not be aware of AVX or the YMM registers.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.