Questions tagged [numba]

Numba is an open source NumPy-aware optimizing compiler for Python.

Filter by
Sorted by
Tagged with
1 vote
0 answers
28 views

Parallel sorting a dictionary and returning the first k items

One approach to returning the first k items of a sorted dictionary is shown in the code snippet below: dict(sorted(dictionary.items(), key=lambda item: item[1], reverse=True)[:k]) Therefore, is there ...
0 votes
0 answers
19 views

Shap import python

I am trying to import shap, but getting the following error- SystemError: initialization of _internal failed without raising an exception I tried solving this by uninstalling and installing again ...
-1 votes
1 answer
24 views

scipy-numba erfc complex number using error

I am trying to write a function which part of it make a complex number and then calculate the erfc of that number to be used to calculate wofz function in numba scipy. The reason for this is that ...
  • 9
0 votes
0 answers
15 views

Signature issues in Numba

I am very new to Numba, here is my code with issues: @nb.njit('float64[:,:](int64, int64, float64, float64[:,:], float64[:,:], float64)', parallel=True) def monte_carlo_simulations(n, nt, a, x1, x2, ...
0 votes
0 answers
18 views

Spleeter and BeatNet incompatible numpy/numba libraries, any solutions?

I want to use two Python libraries, but they seem to be incompatible due to conflicting Numba requirements. I have found a workaround, but it's quite convoluted. I'm currently working in Google's ...
2 votes
1 answer
93 views

Why do these two different ways to sum a 2d array have such different performance?

Consider the following two ways of summing all the values in a 2d numpy array. import numpy as np from numba import njit a = np.random.rand(2, 5000) @njit(fastmath=True, cache=True) def ...
  • 19.1k
0 votes
2 answers
50 views

Automatic parallelization with @jit

I want to make use of the numba Automatic parallelization with @jit, to help me speed runing my function as i am using very larg inputs, i have no experience with parralisation before, i tried several ...
0 votes
0 answers
20 views

Numba's parallelized logistic regression example

I'm having a difficulty understanding how the loop in this logistic regression example from Numba's documentation (https://numba.readthedocs.io/en/stable/user/parallel.html#numba-parallel) could be ...
  • 55
-2 votes
0 answers
23 views

Can numba @jit be used with any arbitary function?

Can @jit decorate a function that takes some python object as input, processes it with an external python function (maybe non-numpy), returns back a numpy array. The idea is to launch the program from ...
0 votes
0 answers
29 views

Numba throws UNKNOWN_CUDA_ERROR when using cuda.copy_to_host() [closed]

So, I am working with a cuda.jit() function with numba on debian system: @cuda.jit() def cuda_trimmer(result_embed, seq_bin_embed, lig_bin_embed): i = cuda.grid(1) seq = seq_bin_embed[i] ...
-1 votes
0 answers
18 views

Create a cashflow for each item in a large pandas dataframe

I have a large dataframe that has columns for a start date, an end date and a daily cost. I need to create a cashflow for the items if the value in the level column is 8. I have successfully done it ...
0 votes
1 answer
41 views

What's the fastest way to append to a list with Numba?

I'm creating some code that does a lot of list appends. It has to be performant so I'm using Numba to @jit compile it. I've checked the Numba documentation on lists, but it doesn't give much ...
  • 775
0 votes
1 answer
34 views

Fast way of getting all positions of the elements of a sub-list

I'm trying to obtain all positions of a sub-list of elements taken from a big list. In Python, using numpy, say I have from datetime import datetime as dt import numpy as np from numba import jit, ...
1 vote
1 answer
73 views

Maximizing the speed of Cython array operations when using typed memory views and BLAS

I'm trying to maximize the speed of my Cython 3.0 code that involves updating an array with a loop of several array operations (including matrix-vector multiplication, vector-vector addition, and ...
  • 25
0 votes
0 answers
18 views

How does Numpy index a 2D array with a 2D array and an integer? [duplicate]

I'm following this tutorial and trying to implement the following two lines in Numba: old_value = q_table[state, action] q_table[state, action] = new_value where q_table has shape (10, 4), state has ...
  • 845
2 votes
1 answer
49 views

why is numba dict lookup so much slower than cpython?

I ran the following simple experiment: import numba import numpy as np @njit def foo(): d = dict() d[(1.0, 2)] = np.random.rand(500) return d e = dict() e[(1.0, 2)] = np....
  • 19.1k
5 votes
1 answer
74 views

Catch overflow error in numba integer multiplication

I am using numba and I would like to know if an overflow has occurred when I multiply two integers. Say the integers are positive for simplicity. I have written the following function to try and ...
  • 53
1 vote
0 answers
27 views

Numba nested function returns "Cannot modify readonly array" error

i have this function: @njit def partial_1dproblem(var,var2,der_array): #First derivative y = var x = var2 dydx = der_array if y.shape != dydx.shape: raise ValueError('...
  • 121
1 vote
0 answers
28 views

Numba TypingError when indexing 2D array with 2D array

I'm trying to index a 2D array with another 2D array which works fine in Numpy, but I want to use it in a function decorated with Numba's njit decorator, which gives me this error: TypingError: No ...
  • 845
0 votes
0 answers
45 views

Stack overflow "-1073741571 (0xC00000FD)" when activating parallelization in Numba, no recursive functions

I am working on Python 3.10, Pycharm IDE. I have this quite long code with loops that I accelerated using numba. The code runs fine if I don't add the flag (parallel=True) to the decorator @njit, but ...
-3 votes
0 answers
82 views

numba: Use of unsupported NumPy function 'numpy.ndarray' or unsupported use of the function

Details: I have CUDA toolkit installed, I am also using Python 3.10. along with these packages: \ py -3.10 -m pip install numba py -3.10 -m pip install numpy py -3.10 -m pip install cuda-python I am ...
  • 35
2 votes
1 answer
33 views

Numba: difference between using a factory function vs `cache=True`

Looking for using (and improving execution speed of) a jitted (numba) function having one or several other jitted functions as parameters, I could see in numbas's FAQ the following: dispatching with ...
  • 875
-1 votes
0 answers
20 views

Speeding up pandas correlation through Numba

I am hoping to speed up the pandas correlation function, I try this: @numba.vectorize def df_corr(returns: pd.DataFrame): df = returns.corr(method='pearson') return df However, when the ...
  • 7,350
3 votes
1 answer
43 views

Why do small changes have dramatic effects on the runtime of my numba parallel function?

I'm trying to understand why my parallelized numba function is acting the way it does. In particular, why it is so sensitive to how arrays are being used. I have the following function: @njit(parallel=...
  • 366
0 votes
0 answers
7 views

numba prints do not show (until I interrupt the ipynb kernel), buffering

How do I set something like pythonunbuffered=1 but for numba? It works fine in terminal, lines get printed sequentially as expected, but in my ipynb it prints many lines at once always. @numba.njit ...
  • 1,695
0 votes
0 answers
30 views

calling cuda.jit in a loop

I want to know what might be the best way to call a jit.cuda function in a loop. Currently, I want to compute an array of location locations=[[1,2], [2,3], [6,8]...] based on a given matrix A. And ...
  • 409
0 votes
1 answer
28 views

Numba how to use dict in a class

as specified below, the dict should have keys 2-tuples of int and values ints. from numba.experimental import jitclass import numba @jitclass({'shape': numba.types.Tuple((numba.int32, numba.int32)), '...
  • 1,695
1 vote
1 answer
47 views

Is @cuda.jit compilation enqueued?

I choose matrix multiplication so that a decent amount of time is spent on the gpu, which will make it easier to fill up the launch queue. from numba import cuda, float64 import numpy as np import ...
  • 145
0 votes
1 answer
50 views

Is cuda.to_device asynchronous?

Does cuda.to_device use the same stream as kernel launches? It seems that memcpy is synchronous (with respect to the host). from numba import cuda import numpy as np A = np.ones((10000, 10000)) %...
  • 145
0 votes
0 answers
30 views

numba passing a int or float to a function?

I am learning cuda computing by using numba. If I have a function like @cuda.jit def cuda_computing(a, b, c) c=a+b Becase the a and b are just integers or floats. I want to put them into device ...
  • 409
2 votes
1 answer
82 views

Numba - is it possible to do absolute sum without nan faster?

I have a slightly modified example from numba official doc as follows: from numba import njit import numpy as np @njit def do_sum(A, lb, ub): n = len(A) acc = 0.0 for i in range(n): ...
  • 99
1 vote
0 answers
36 views

Numba parallel is slower than sequential. How to improve the speed?

I am trying to speed up my code using numba parallel=True, but I find that it is slower when the iterations are very fast. Here is a simplified example of my code: spec = [ ('value', nb.float64), ]...
  • 11
8 votes
0 answers
159 views

Why is PyTorch C++ extension much slower than its equivalent numba version?

I have been experimenting with various options to speed up some for-loop-heavy-logic in PyTorch. The two obvious options to do so are either using numba or writing a custom C++ extension. As an ...
  • 22.3k
0 votes
1 answer
34 views

Numba: how to get indexes for all rows that contain at least one nan values?

Suppose I have a numpy 2d array (m by n), I want to get indexes of all its rows contain at least one nan values. It is relatively straightforward to do it in pure numpy as follows: import numpy as np ...
  • 99
0 votes
0 answers
28 views

Join arrays in numba

I am having trouble neatly joining numpy arrays that are in a list or from a generator in numba jited functions. The simplest example is as follows: import numpy as np from numba import njit @njit ...
  • 53
46 votes
2 answers
4k views

Why is np.dot so much faster than np.sum?

Why is np.dot so much faster than np.sum? Following this answer we know that np.sum is slow and has faster alternatives. For example: In [20]: A = np.random.rand(1000) In [21]: B = np.random.rand(...
  • 19.1k
-1 votes
1 answer
74 views

How to make numpy clip run faster?

I have a custom machine learning objective function which is a kind of a linear bounded function and mainly use numpy.clip. During training, the objective function will be run a whole of times. The ...
  • 99
0 votes
0 answers
37 views

How can I write a faster Lempel-Ziv complexity calculator function, for a time series, in Python?

I'm trying to improve upon the execution time of the below function for large input arrays (of a time series with 5-10 million data points). After trying different variations for hours (and not quite ...
-1 votes
1 answer
48 views

Why is accessing elements of an array slower in the GPU than the CPU with Numba?

Since we can't call print inside @cuda.jit and trying to print cuda.to_device(A) results in <numba.cuda.cudadrv.devicearray.DeviceNDArray at 0x7f2c5c0605e0>, I didn't think we could print ...
  • 145
0 votes
1 answer
53 views

Calling functions on arrays moved to GPU with Numba

I didn't think we could print anything from the GPU since calling print inside a @cuda.jit function doesn't work, but then I tried calling A.shape to see what would happen. import numpy as np from ...
  • 145
1 vote
0 answers
27 views

Calling Cython Functions from Numba njited function where numpy ndarrays are involved

I am trying to do the following... # my_cython.pyx cpdef api double cyf(double x, double[:] xarr, double[:] yarr) nogil: # Do stuff return result # my_cython.pxd cpdef api double cyf(double x,...
  • 287
0 votes
1 answer
57 views

How to calculate the correlation coefficient on a rolling window of a vector using numba?

People were kind enough to explain : How to calculate the correlation coefficient on a rolling window of a vector using numpy? with this answer where I picked up: f_PH_numpy is my approach, which ...
  • 138
-1 votes
1 answer
59 views

why is Numba parallel is slower than normal python loop?

Following is normal python loop (I copied example from official doc - https://numba.readthedocs.io/en/stable/user/parallel.html) def two_d_array_reduction_prod(n): shp = (13, 17) result1 = 2 * ...
0 votes
0 answers
24 views

Pyodide with numba compiled .so library

I have a python library which has a numba AOT compile function into a .so file. I am trying to bundle this into a pyodide wheel. When I examine the wheel file, the .so file (or webasm version) is ...
0 votes
1 answer
29 views

Numba cannot resolve function (np.digitize)

I get an error by numba, it is complaining it can't resolve a function. The minimal code to reproduce my errors is import numba import numpy as np @numba.jit("float64(float64)", nopython=...
0 votes
0 answers
21 views

Numba JIT on static method works on one computer but throws an error on another

I have a static method that I want to speed up with Numba: @nb.njit def numba_loop_choice(population, weights, k): wc = np.cumsum(weights) m = wc[-1] sample = np.empty(k, population.dtype) ...
  • 983
2 votes
0 answers
34 views

Memory overwriting other memory issue in Numba?

Take the following script: import numba @numba.njit def foo(): lst1 = [[1,2,3,4,5], [6,7,8,9,10]] for i in range(300): lst2 = [] lst2.append(1) del lst1[1] del lst1[0] lst1....
  • 491
0 votes
0 answers
198 views

How to convert python CPU script to runs on GPU

How to edit a python script that runs on CPU to runs on GPU which support cuda? here is the code import numpy as np import pandas as pd from datetime import datetime as dt import scipy.signal import ...
  • 23
0 votes
1 answer
34 views

how to return list for eager compilation?

The numba allows eager compilation by telling the function signature. But, I can not find some information about the list type. My test code is: import numba as nb import numpy as np # @nb.jit(nb....
1 vote
0 answers
33 views

How to solve the numba type error led in nopython mode pipeline (step: nopython frontend) Invalid use of type(CPUDispatcher(<function

So I am trying to write a code with numba that is calculating eigenvalues of a complex matrix and gives eigenvalues which are complex but with infinitesimally small imaginary part which is expected. ...
  • 188

1
2 3 4 5
44