5

If I understand correctly I can increase the performance of a numba function by adding a signature. Example:

@njit(int32(int32, int32))
def f(x, y):
    # A somewhat trivial example
    return x + y

Now I have function which takes two sets. What is the correct signature?

@njit(int32(set(int32), set(int32)))
def f(set_1, set_2):
    # A somewhat trivial example
    return x

I thought the signature (int32(set(int32), set(int32))) could be correct but nothing is happening. print(numba.typeof(set_1)) returns reflected set(int32)

1 Answer 1

6

If I understand correctly I can increase the performance of a numba function by adding a signature.

That's wrong - or only partially correct. With signature numba just compiles the function ahead-of-time instead of when called for the first time with these arguments. After the first call both should be equally fast. In some cases the function might be slightly faster without signature (especially with arrays where numba can use the array-alignment of the input).

Now I have function which takes two sets. What is the correct signature?

The correct signature for a Python set containing integers is:

numba.types.Set(numba.int64, reflected=True)

So the signature for a function taking two sets (and returning one) would be:

import numba as nb

reflected_int_set = nb.types.Set(nb.int64, reflected=True)

@nb.njit(reflected_int_set(reflected_int_set, reflected_int_set))
def f(set_1, set_2):
    return set_1

>>> f({1,2,3}, {3,4,5})
{1, 2, 3}

But since it (very likely) doesn't improve performance I wouldn't bother with the signature at all.

Also a word of caution: numba will convert the Python set to a numba set internally so passing a Python set to a numba function or returning a set from a numba function to a Python context will copy the complete set. In most cases that overhead is much more significant than potential speed-ups that numba provides.

In my experience sets and lists with numba only make sense if they are strictly confined to numba functions. So if you use them as arguments or return them (to non numba functions/contexts) you have to measure the performance and check if you really get speed-ups.

2
  • Does it have to be int64 or cannot also be int32?
    – Christian
    Commented Jun 27, 2023 at 17:15
  • @Christian I'm actually not sure. I haven't used numba for years. I assume it's possible. But why don't you try it yourself?
    – MSeifert
    Commented Aug 4, 2023 at 16:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.