Questions tagged [numba]
Numba is an open source NumPy-aware optimizing compiler for Python.
2,153
questions
1
vote
0
answers
28
views
Parallel sorting a dictionary and returning the first k items
One approach to returning the first k items of a sorted dictionary is shown in the code snippet below:
dict(sorted(dictionary.items(), key=lambda item: item[1], reverse=True)[:k])
Therefore, is there ...
0
votes
0
answers
19
views
Shap import python
I am trying to import shap, but getting the following error-
SystemError: initialization of _internal failed without raising an exception
I tried solving this by uninstalling and installing again ...
-1
votes
1
answer
24
views
scipy-numba erfc complex number using error
I am trying to write a function which part of it make a complex number and then calculate the erfc of that number to be used to calculate wofz function in numba scipy. The reason for this is that ...
0
votes
0
answers
15
views
Signature issues in Numba
I am very new to Numba, here is my code with issues:
@nb.njit('float64[:,:](int64, int64, float64, float64[:,:], float64[:,:], float64)', parallel=True)
def monte_carlo_simulations(n, nt, a, x1, x2, ...
0
votes
0
answers
18
views
Spleeter and BeatNet incompatible numpy/numba libraries, any solutions?
I want to use two Python libraries, but they seem to be incompatible due to conflicting Numba requirements. I have found a workaround, but it's quite convoluted. I'm currently working in Google's ...
2
votes
1
answer
93
views
Why do these two different ways to sum a 2d array have such different performance?
Consider the following two ways of summing all the values in a 2d numpy array.
import numpy as np
from numba import njit
a = np.random.rand(2, 5000)
@njit(fastmath=True, cache=True)
def ...
0
votes
2
answers
50
views
Automatic parallelization with @jit
I want to make use of the numba Automatic parallelization with @jit, to help me speed runing my function as i am using very larg inputs, i have no experience with parralisation before, i tried several ...
0
votes
0
answers
20
views
Numba's parallelized logistic regression example
I'm having a difficulty understanding how the loop in this logistic regression example from Numba's documentation (https://numba.readthedocs.io/en/stable/user/parallel.html#numba-parallel) could be ...
-2
votes
0
answers
23
views
Can numba @jit be used with any arbitary function?
Can @jit decorate a function that takes some python object as input, processes it with an external python function (maybe non-numpy), returns back a numpy array. The idea is to launch the program from ...
0
votes
0
answers
29
views
Numba throws UNKNOWN_CUDA_ERROR when using cuda.copy_to_host() [closed]
So, I am working with a
cuda.jit()
function with numba on debian system:
@cuda.jit()
def cuda_trimmer(result_embed, seq_bin_embed, lig_bin_embed):
i = cuda.grid(1)
seq = seq_bin_embed[i]
...
-1
votes
0
answers
18
views
Create a cashflow for each item in a large pandas dataframe
I have a large dataframe that has columns for a start date, an end date and a daily cost. I need to create a cashflow for the items if the value in the level column is 8. I have successfully done it ...
0
votes
1
answer
41
views
What's the fastest way to append to a list with Numba?
I'm creating some code that does a lot of list appends. It has to be performant so I'm using Numba to @jit compile it.
I've checked the Numba documentation on lists, but it doesn't give much ...
0
votes
1
answer
34
views
Fast way of getting all positions of the elements of a sub-list
I'm trying to obtain all positions of a sub-list of elements taken from a big list.
In Python, using numpy, say I have
from datetime import datetime as dt
import numpy as np
from numba import jit, ...
1
vote
1
answer
73
views
Maximizing the speed of Cython array operations when using typed memory views and BLAS
I'm trying to maximize the speed of my Cython 3.0 code that involves updating an array with a loop of several array operations (including matrix-vector multiplication, vector-vector addition, and ...
0
votes
0
answers
18
views
How does Numpy index a 2D array with a 2D array and an integer? [duplicate]
I'm following this tutorial and trying to implement the following two lines in Numba:
old_value = q_table[state, action]
q_table[state, action] = new_value
where q_table has shape (10, 4), state has ...
2
votes
1
answer
49
views
why is numba dict lookup so much slower than cpython?
I ran the following simple experiment:
import numba
import numpy as np
@njit
def foo():
d = dict()
d[(1.0, 2)] = np.random.rand(500)
return d
e = dict()
e[(1.0, 2)] = np....
5
votes
1
answer
74
views
Catch overflow error in numba integer multiplication
I am using numba and I would like to know if an overflow has occurred when I multiply two integers. Say the integers are positive for simplicity.
I have written the following function to try and ...
1
vote
0
answers
27
views
Numba nested function returns "Cannot modify readonly array" error
i have this function:
@njit
def partial_1dproblem(var,var2,der_array): #First derivative
y = var
x = var2
dydx = der_array
if y.shape != dydx.shape:
raise ValueError('...
1
vote
0
answers
28
views
Numba TypingError when indexing 2D array with 2D array
I'm trying to index a 2D array with another 2D array which works fine in Numpy, but I want to use it in a function decorated with Numba's njit decorator, which gives me this error:
TypingError: No ...
0
votes
0
answers
45
views
Stack overflow "-1073741571 (0xC00000FD)" when activating parallelization in Numba, no recursive functions
I am working on Python 3.10, Pycharm IDE.
I have this quite long code with loops that I accelerated using numba. The code runs fine if I don't add the flag (parallel=True) to the decorator @njit, but ...
-3
votes
0
answers
82
views
numba: Use of unsupported NumPy function 'numpy.ndarray' or unsupported use of the function
Details:
I have CUDA toolkit installed, I am also using Python 3.10. along with these packages: \
py -3.10 -m pip install numba
py -3.10 -m pip install numpy
py -3.10 -m pip install cuda-python
I am ...
2
votes
1
answer
33
views
Numba: difference between using a factory function vs `cache=True`
Looking for using (and improving execution speed of) a jitted (numba) function having one or several other jitted functions as parameters, I could see in numbas's FAQ the following:
dispatching with ...
-1
votes
0
answers
20
views
Speeding up pandas correlation through Numba
I am hoping to speed up the pandas correlation function, I try this:
@numba.vectorize
def df_corr(returns: pd.DataFrame):
df = returns.corr(method='pearson')
return df
However, when the ...
3
votes
1
answer
43
views
Why do small changes have dramatic effects on the runtime of my numba parallel function?
I'm trying to understand why my parallelized numba function is acting the way it does. In particular, why it is so sensitive to how arrays are being used.
I have the following function:
@njit(parallel=...
0
votes
0
answers
7
views
numba prints do not show (until I interrupt the ipynb kernel), buffering
How do I set something like pythonunbuffered=1 but for numba? It works fine in terminal, lines get printed sequentially as expected, but in my ipynb it prints many lines at once always.
@numba.njit
...
0
votes
0
answers
30
views
calling cuda.jit in a loop
I want to know what might be the best way to call a jit.cuda function in a loop. Currently, I want to compute an array of location locations=[[1,2], [2,3], [6,8]...] based on a given matrix A. And ...
0
votes
1
answer
28
views
Numba how to use dict in a class
as specified below, the dict should have keys 2-tuples of int and values ints.
from numba.experimental import jitclass
import numba
@jitclass({'shape': numba.types.Tuple((numba.int32, numba.int32)), '...
1
vote
1
answer
47
views
Is @cuda.jit compilation enqueued?
I choose matrix multiplication so that a decent amount of time is spent on the gpu, which will make it easier to fill up the launch queue.
from numba import cuda, float64
import numpy as np
import ...
0
votes
1
answer
50
views
Is cuda.to_device asynchronous?
Does cuda.to_device use the same stream as kernel launches?
It seems that memcpy is synchronous (with respect to the host).
from numba import cuda
import numpy as np
A = np.ones((10000, 10000))
%...
0
votes
0
answers
30
views
numba passing a int or float to a function?
I am learning cuda computing by using numba. If I have a function like
@cuda.jit
def cuda_computing(a, b, c)
c=a+b
Becase the a and b are just integers or floats. I want to put them into device ...
2
votes
1
answer
82
views
Numba - is it possible to do absolute sum without nan faster?
I have a slightly modified example from numba official doc as follows:
from numba import njit
import numpy as np
@njit
def do_sum(A, lb, ub):
n = len(A)
acc = 0.0
for i in range(n):
...
1
vote
0
answers
36
views
Numba parallel is slower than sequential. How to improve the speed?
I am trying to speed up my code using numba parallel=True, but I find that it is slower when the iterations are very fast. Here is a simplified example of my code:
spec = [
('value', nb.float64),
]...
8
votes
0
answers
159
views
Why is PyTorch C++ extension much slower than its equivalent numba version?
I have been experimenting with various options to speed up some for-loop-heavy-logic in PyTorch. The two obvious options to do so are either using numba or writing a custom C++ extension.
As an ...
0
votes
1
answer
34
views
Numba: how to get indexes for all rows that contain at least one nan values?
Suppose I have a numpy 2d array (m by n), I want to get indexes of all its rows contain at least one nan values.
It is relatively straightforward to do it in pure numpy as follows:
import numpy as np
...
0
votes
0
answers
28
views
Join arrays in numba
I am having trouble neatly joining numpy arrays that are in a list or from a generator in numba jited functions. The simplest example is as follows:
import numpy as np
from numba import njit
@njit
...
46
votes
2
answers
4k
views
Why is np.dot so much faster than np.sum?
Why is np.dot so much faster than np.sum? Following this answer we know that np.sum is slow and has faster alternatives.
For example:
In [20]: A = np.random.rand(1000)
In [21]: B = np.random.rand(...
-1
votes
1
answer
74
views
How to make numpy clip run faster?
I have a custom machine learning objective function which is a kind of a linear bounded function and mainly use numpy.clip. During training, the objective function will be run a whole of times. The ...
0
votes
0
answers
37
views
How can I write a faster Lempel-Ziv complexity calculator function, for a time series, in Python?
I'm trying to improve upon the execution time of the below function for large input arrays (of a time series with 5-10 million data points). After trying different variations for hours (and not quite ...
-1
votes
1
answer
48
views
Why is accessing elements of an array slower in the GPU than the CPU with Numba?
Since we can't call print inside @cuda.jit and trying to print cuda.to_device(A) results in <numba.cuda.cudadrv.devicearray.DeviceNDArray at 0x7f2c5c0605e0>, I didn't think we could print ...
0
votes
1
answer
53
views
Calling functions on arrays moved to GPU with Numba
I didn't think we could print anything from the GPU since calling print inside a @cuda.jit function doesn't work, but then I tried calling A.shape to see what would happen.
import numpy as np
from ...
1
vote
0
answers
27
views
Calling Cython Functions from Numba njited function where numpy ndarrays are involved
I am trying to do the following...
# my_cython.pyx
cpdef api double cyf(double x, double[:] xarr, double[:] yarr) nogil:
# Do stuff
return result
# my_cython.pxd
cpdef api double cyf(double x,...
0
votes
1
answer
57
views
How to calculate the correlation coefficient on a rolling window of a vector using numba?
People were kind enough to explain :
How to calculate the correlation coefficient on a rolling window of a vector using numpy?
with this answer where I picked up:
f_PH_numpy is my approach, which ...
-1
votes
1
answer
59
views
why is Numba parallel is slower than normal python loop?
Following is normal python loop (I copied example from official doc - https://numba.readthedocs.io/en/stable/user/parallel.html)
def two_d_array_reduction_prod(n):
shp = (13, 17)
result1 = 2 * ...
0
votes
0
answers
24
views
Pyodide with numba compiled .so library
I have a python library which has a numba AOT compile function into a .so file. I am trying to bundle this into a pyodide wheel. When I examine the wheel file, the .so file (or webasm version) is ...
0
votes
1
answer
29
views
Numba cannot resolve function (np.digitize)
I get an error by numba, it is complaining it can't resolve a function.
The minimal code to reproduce my errors is
import numba
import numpy as np
@numba.jit("float64(float64)", nopython=...
0
votes
0
answers
21
views
Numba JIT on static method works on one computer but throws an error on another
I have a static method that I want to speed up with Numba:
@nb.njit
def numba_loop_choice(population, weights, k):
wc = np.cumsum(weights)
m = wc[-1]
sample = np.empty(k, population.dtype)
...
2
votes
0
answers
34
views
Memory overwriting other memory issue in Numba?
Take the following script:
import numba
@numba.njit
def foo():
lst1 = [[1,2,3,4,5], [6,7,8,9,10]]
for i in range(300):
lst2 = []
lst2.append(1)
del lst1[1]
del lst1[0]
lst1....
0
votes
0
answers
198
views
How to convert python CPU script to runs on GPU
How to edit a python script that runs on CPU to runs on GPU which support cuda?
here is the code
import numpy as np
import pandas as pd
from datetime import datetime as dt
import scipy.signal
import ...
0
votes
1
answer
34
views
how to return list for eager compilation?
The numba allows eager compilation by telling the function signature. But, I can not find some information about the list type. My test code is:
import numba as nb
import numpy as np
# @nb.jit(nb....
1
vote
0
answers
33
views
How to solve the numba type error led in nopython mode pipeline (step: nopython frontend) Invalid use of type(CPUDispatcher(<function
So I am trying to write a code with numba that is calculating eigenvalues of a complex matrix and gives eigenvalues which are complex but with infinitesimally small imaginary part which is expected. ...