2

I need to enable x2apic on Intel(R) Xeon(R) CPU E3-1225 v5 @ 3.30GHz, and I find x2apic was supported in cpuinfo:

enter image description here

But when kernel started, I find error message:

[ 0.138328] IRQ remapping doesn't support X2APIC mode, disable x2apic.

I have checked my kernel config:

CONFIG_X86_X2APIC=y

What can I do to fix this problem?

1

2 Answers 2

4

Personally I find this error message to be misleading.

What it says: IRQ remapping doesn't support X2APIC mode, disable x2apic.

What it actually means: IRQ remapping isn't enabled, so x2APIC was disabled.

The solution is to turn on IRQ remapping. Set CONFIG_IRQ_REMAP=y in your kernel config and recompile it.

x2APIC requires that the IOMMU be enabled and that IRQ remapping is enabled.

  1. Enable IOMMU in the BIOS. (It's disabled by default on many systems. Even on my brand new (July 2020) Gigabyte R282-Z93 with EPYC 7742 CPUs!)
  2. Enable x2APIC in the BIOS.
  3. Enable IOMMU support in the kernel:
    • For Intel CPUs: CONFIG_INTEL_IOMMU=y
    • For AMD CPUs: CONFIG_AMD_IOMMU=y and CONFIG_AMD_IOMMU_V2=y
  4. Enable x2APIC support in the kernel: CONFIG_X86_X2APIC=y
  5. Enable IRQ remapping in the kernel: CONFIG_IRQ_REMAP=y

So if x2APIC requires IRQ remapping then how exactly does one end up with a kernel that has x2APIC support but no IRQ remapping? Well, if you look at the dependencies for CONFIG_X86_X2APIC you'll see that it requires X86_LOCAL_APIC && X86_64 && (IRQ_REMAP || HYPERVISOR_GUEST)

So if your kernel was built to support running under a hypervisor then you could very well fail to enable the option for IRQ remapping. (I did exactly this. Oops! It was very confusing to figure out what went wrong until I grepped the kernel source for the "IRQ remapping doesn't support X2APIC mode, disable x2apic" message, at which point it became relatively straightforward to figure out the problem.)

3

x2apic in /proc/cpuinfo shows capabilities of your CPU chip. The message "IRQ remapping doesn't support X2APIC mode, disable x2apic" is generated by kernel function `try_to_enable_x2apic() when there is environment where x2apic mode can't be enabled for the required mode. Current code is

static __init void try_to_enable_x2apic(int remap_mode)
{
    if (x2apic_state == X2APIC_DISABLED)
        return;

    if (remap_mode != IRQ_REMAP_X2APIC_MODE) {
        /* IR is required if there is APIC ID > 255 even when running
         * under KVM
         */
        if (max_physical_apicid > 255 ||
            !x86_init.hyper.x2apic_available()) {
            pr_info("x2apic: IRQ remapping doesn't support X2APIC mode\n");
            x2apic_disable();
            return;
        }

        /*
         * without IR all CPUs can be addressed by IOAPIC/MSI
         * only in physical mode
         */
        x2apic_phys = 1;
    }
    x2apic_enable();
}

It is called from enable_IR_x2apic(void) of arch/x86/kernel/apic/apic.c file:

    ir_stat = irq_remapping_prepare();
    if (ir_stat < 0 && !x2apic_supported())
        return;
...

    /* If irq_remapping_prepare() succeeded, try to enable it */
    if (ir_stat >= 0)
        ir_stat = irq_remapping_enable();
    /* ir_stat contains the remap mode or an error code */
    try_to_enable_x2apic(ir_stat);

There were some patches in `try_to_enable_x2apic() function in 2015 https://lore.kernel.org/patchwork/patch/533651/ and https://lore.kernel.org/patchwork/patch/533496/; and there was large mailing list theread in 2013 when the code path was introduced https://groups.google.com/forum/#!msg/fa.linux.kernel/uD28wv9Inhg/KHgAvvkA0d4J

x86/apic: Only disable CPU x2apic mode when necessary

When interrupt remapping hardware is not in X2APIC, CPU X2APIC mode will be disabled if:

  • 1) Maximum CPU APIC ID is bigger than 255
  • 2) hypervisior doesn't support x2apic mode.

You should check you hypervisor (if used) and irq remapping method. There can be "Enabled IRQ remapping in" pr_info message from intel_enable_irq_remapping - https://elixir.bootlin.com/linux/latest/source/drivers/iommu/intel_irq_remapping.c#L827

Not the answer you're looking for? Browse other questions tagged or ask your own question.