57

I know that Numpy can use different backends like OpenBLAS or MKL. I have also read that MKL is heavily optimized for Intel, so usually people suggest to use OpenBLAS on AMD, right?

I use the following test code:

import numpy as np

def testfunc(x):
    np.random.seed(x)
    X = np.random.randn(2000, 4000)
    np.linalg.eigh(X @ X.T)

%timeit testfunc(0)

I have tested this code using different CPUs:

  • On Intel Xeon E5-1650 v3, this code performs in 0.7s using 6 out of 12 cores.
  • On AMD Ryzen 5 2600, this code performs in 1.45s using all 12 cores.
  • On AMD Ryzen Threadripper 3970X, this code performs in 1.55s using all 64 cores.

I am using the same Conda environment on all three systems. According to np.show_config(), the Intel system uses the MKL backend for Numpy (libraries = ['mkl_rt', 'pthread']), whereas the AMD systems use OpenBLAS (libraries = ['openblas', 'openblas']). The CPU core usage was determined by observing top in a Linux shell:

  • For the Intel Xeon E5-1650 v3 CPU (6 physical cores), it shows 12 cores (6 idling).
  • For the AMD Ryzen 5 2600 CPU (6 physical cores), it shows 12 cores (none idling).
  • For the AMD Ryzen Threadripper 3970X CPU (32 physical cores), it shows 64 cores (none idling).

The above observations give rise to the following questions:

  1. Is that normal, that linear algebra on up-to-date AMD CPUs using OpenBLAS is that much slower than on a six-year-old Intel Xeon? (also addressed in Update 3)
  2. Judging by the observations of the CPU load, it looks like Numpy utilizes the multi-core environment in all three cases. How can it be that the Threadripper is even slower than the Ryzen 5, even though it has almost six times as many physical cores? (also see Update 3)
  3. Is there anything that can be done to speed up the computations on the Threadripper? (partially answered in Update 2)

Update 1: The OpenBLAS version is 0.3.6. I read somewhere, that upgrading to a newer version might help, however, with OpenBLAS updated to 0.3.10, the performance for testfunc is still 1.55s on AMD Ryzen Threadripper 3970X.


Update 2: Using the MKL backend for Numpy in conjunction with the environment variable MKL_DEBUG_CPU_TYPE=5 (as described here) reduces the run time for testfunc on AMD Ryzen Threadripper 3970X to only 0.52s, which is actually more or less satisfying. FTR, setting this variable via ~/.profile did not work for me on Ubuntu 20.04. Also, setting the variable from within Jupyter did not work. So instead I put it into ~/.bashrc which works now. Anyways, performing 35% faster than an old Intel Xeon, is this all we get, or can we get more out of it?


Update 3: I play around with the number of threads used by MKL/OpenBLAS:

run time by the number of threads and library

The run times are reported in seconds. The best result of each column is underlined. I used OpenBLAS 0.3.6 for this test. The conclusions from this test:

  • The single-core performance of the Threadripper using OpenBLAS is a bit better than the single-core performance of the Xeon (11% faster), however, its single-core performance is even better when using MKL (34% faster).
  • The multi-core performance of the Threadripper using OpenBLAS is ridiculously worse than the multi-core performance of the Xeon. What is going on here?
  • The Threadripper performs overall better than the Xeon, when MKL is used (26% to 38% faster than Xeon). The overall best performance is achieved by the Threadripper using 16 threads and MKL (36% faster than Xeon).

Update 4: Just for clarification. No, I do not think that (a) this or (b) that answers this question. (a) suggests that "OpenBLAS does nearly as well as MKL", which is a strong contradiction to the numbers I observed. According to my numbers, OpenBLAS performs ridiculously worse than MKL. The question is why. (a) and (b) both suggest using MKL_DEBUG_CPU_TYPE=5 in conjunction with MKL to achieve maximum performance. This might be right, but it does neither explain why OpenBLAS is that dead slow. Neither it explains, why even with MKL and MKL_DEBUG_CPU_TYPE=5 the 32-core Threadripper is only 36% faster than the six-year-old 6-core Xeon.

23
  • maybe relevant pugetsystems.com/labs/hpc/… also Google openblas vs MKL
    – qwr
    Commented Jul 7, 2020 at 21:01
  • I'd suspect inter-core latency might be a bigger issue across CCX clusters of 4 cores on Threadripper? 3970X is a Zen 2 part, so it should have 2x 256-bit SIMD FMA throughput (per core), same as Intel Haswell. Perhaps a library tuned for AMD is only using 128-bit SIMD because that was sometimes better for Zen1. (Your Ryzen 5 2600 is a Zen1, 1x 128-bit FMA uop per clock, so it's crazy that it's slower than a Zen2). Different BLAS libraries might be a big factor. Commented Jul 7, 2020 at 22:34
  • 1
    I'd advise to run comparisons with different number of threads (OPENBLAS_NUM_THREADS, MKL_NUM_THREADS). Server processors have slower per-core speed, and multicore speedups in BLAS libraries are usually very appalling.
    – amiasato
    Commented Jul 8, 2020 at 0:45
  • 3
    Generating random numbers takes a lot of time (1/4 of total time on my system). It would be better to only get the timings of np.linalg.eigh(X @ X.T). Also set the MKL_NUM_THREADS to the number of physical threads. This BLAS algortihms usually scale negative with virtual cores.
    – max9111
    Commented Jul 8, 2020 at 8:12
  • 1
    Intel documents the single-core max turbo, and you can just manually look at clock speeds while the benchmark is running. (grep MHz /proc/cpuinfo or whatever). Ideally run your program under perf on Linux: perf stat my_benchmark to record HW performance counters which includes the cycles event, and will calculate the average clock speed the CPU actually ran at over the benchmark interval. (By dividing cycles by the task-clock kernel event.) Commented Jul 8, 2020 at 15:28

3 Answers 3

4

As of 2021, Intel unfortunately removed the MKL_DEBUG_CPU_TYPE to prevent people on AMD use the workaround presented in the accepted answer. This means that the workaround no longer works, and AMD users have to either switch to OpenBLAS or keep using MKL.

To use the workaround, follow this method:

  1. Create a conda environment with conda's and NumPy's MKL=2019.
  2. Activate the environment
  3. Set MKL_DEBUG_CPU_TYPE = 5

The commands for the above steps:

  1. conda create -n my_env -c anaconda python numpy mkl=2019.* blas=*=*mkl
  2. conda activate my_env
  3. conda env config vars set MKL_DEBUG_CPU_TYPE=5

And thats it!

10
  • 1
    You do currently have enough rep to comment, thanks to your useful contributions getting upvotes :). This is actually a relevant answer for future readers facing the problem of slow MKL Numpy on AMD CPUs, though, so it's fine. In some cases it might be better to suggest an edit to an existing answer, pointing out that it doesn't work with the latest MKL, but here a separate answer makes as much sense as editing 3 different answers. Especially if you make this into an answer that does directly address the question here. Commented Aug 26, 2021 at 17:26
  • I think you can still use an older MKL version, right? At least, 2020.0 still works for me.
    – kostrykin
    Commented Aug 27, 2021 at 14:52
  • 1
    I use mkl=2020.0 along with blas=*=mkl in my environment .yml files, however, I am not 100% sure that it works, since I have noticed some strange slow downs in a recently created environment.
    – kostrykin
    Commented Aug 27, 2021 at 18:27
  • There is no "accepted answer" on this question. It's usually not a good idea to copy/paste the identical answers onto different questions, since future editors will need to find them both / all. This should probably still be a link to your answer on another question for the full step-by-step guide, maybe just say here to use 2019 MKL with the MKL_DEBUG_CPU_TYPE=5 environment setting, see that for full details. Commented Sep 16, 2021 at 7:37
  • And you can make the rest of this answer be specific to this question by describing what Intel's "cripple-AMD" function actually does. Commented Sep 16, 2021 at 7:39
2

Wouldn't it make sense to try using an optimized BLIS library from AMD?

Maybe I am missing (misunderstanding) something, but I would assume you could use BLIS instead of OpenBLAS. The only potential problem could be that AMD BLIS is optimized for AMD EPYC (but you're using Ryzen). I'm VERY curious about the results, since I'm in the process of buying a server for work, and am considering AMD EPYC and Intel Xeon.

Here are the respective AMD BLIS libraries: https://developer.amd.com/amd-aocl/

2
  • 2
    Even though installation of BLIS via conda looks easy, it seems non-straight forward to me how to make Numpy actually use BLIS as the backend. However, according to this, MKL outperforms BLIS on Ryzen ("with some quick/dirty results on my Ryzen 3700X [...] You can see performance basically double on MKL when MKL_DEBUG_CPU_TYPE=5 is used").
    – kostrykin
    Commented Aug 13, 2020 at 15:13
  • How to compile and install numpy with BLIS linked to AMD's AOCL BLIS # download files from developer.amd.com/amd-aocl # unpack to e.g. /home/AOCL/2.2 # create ~/.numpy-site.cfg [blis] libraries = blis library_dirs = /home/AOCL/2.2/lib include_dirs = /home/AOCL/2.2/include runtime_library_dirs = /home/AOCL/2.2/lib # git clone github.com/numpy/numpy.git # cd numpy # pip install .
    – tryptofame
    Commented Sep 10, 2020 at 12:30
1

I think this should help:

"The best result in the chart is for the TR 3960x using MKL with the environment var MKL_DEBUG_CPU_TYPE=5. AND it is significantly better than the low optimization code path from MKL alone. AND,OpenBLAS does nearly as well as MKL with MKL_DEBUG_CPU_TYPE=5 set." https://www.pugetsystems.com/labs/hpc/How-To-Use-MKL-with-AMD-Ryzen-and-Threadripper-CPU-s-Effectively-for-Python-Numpy-And-Other-Applications-1637/

How to set up: 'Make the setting permanent by entering MKL_DEBUG_CPU_TYPE=5 into the System Environment Variables. This has several advantages, one of them being that it applies to all instances of Matlab and not just the one opened using the .bat file' https://www.reddit.com/r/matlab/comments/dxn38s/howto_force_matlab_to_use_a_fast_codepath_on_amd/?sort=new

3

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.