202

Has anyone implemented type hinting for the specific numpy.ndarray class?

Right now, I'm using typing.Any, but it would be nice to have something more specific.

For instance if the NumPy people added a type alias for their array_like object class. Better yet, implement support at the dtype level, so that other objects would be supported, as well as ufunc.

8
  • 1
    pypi.python.org/pypi/plac can make use of Py3 annotations - to populate an argparse parser. For Py2, it uses decorators to create a similar annocation database.
    – hpaulj
    Feb 27, 2016 at 20:30
  • 1
    typing is new to Py 3.5. Many numpy users still work with Py2. I have 3.5 on my system, but I don't have numpy installed for it. numpy developers are not going to add features for the cutting edge of Python (with the exception of the @ operator)
    – hpaulj
    Feb 27, 2016 at 21:26
  • 1
    numpy is maintained on a github repository. Look at the issues and pull requests; sign up and submit your own issue. There may be another forum for discussing development issues, but most I look at the github issues.
    – hpaulj
    Feb 28, 2016 at 20:17
  • 7
    For anyone looking into the issue - it looks like there's a relevant solution here: stackoverflow.com/questions/52839427/… Jun 20, 2019 at 12:24
  • 2
    > There is now... @Jasha this ticket was opened by me, the OP, 4.5 years ago.
    – Inon
    Sep 3, 2020 at 19:48

5 Answers 5

80

Numpy 1.21 includes a numpy.typing module with an NDArray generic type.


From the Numpy 1.21 docs:
numpy.typing.NDArray = numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]]

A generic version of np.ndarray[Any, np.dtype[+ScalarType]].

Can be used during runtime for typing arrays with a given dtype and unspecified shape.

Examples:

>>> import numpy as np
>>> import numpy.typing as npt

>>> print(npt.NDArray)
numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]]

>>> print(npt.NDArray[np.float64])
numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]]

>>> NDArrayInt = npt.NDArray[np.int_]
>>> a: NDArrayInt = np.arange(10)

>>> def func(a: npt.ArrayLike) -> npt.NDArray[Any]:
...     return np.array(a)

As of 2022-09-05, support for shapes is still a work in progress per numpy/numpy#16544.

6
  • 3
    I am just wondering what if I use ndarray rather than NDArray in type hinting ? Is there any fundamental difference ?
    – Jiadong
    Sep 5, 2022 at 1:54
  • 2
    Looking at the definition of NDarray, it seems that (1) there is a difference at runtime (as NDArray is a generic alias while ndarray is a class), and (2) at type-checking time (e.g. when using mypy, pyright, etc) there should by no difference between NDarray[Foo] and np.ndarray[Any, np.dtype[Foo]].
    – Jasha
    Sep 5, 2022 at 20:27
  • Is there a way to define the number of dimensions (Vector, Matrix, 3 dimensions, etc...)?
    – Royi
    Nov 11, 2022 at 8:43
  • No, Numpy does not support that currently. There are long term efforts to support such shape hints in the python ecosystem, e.g. PEP 646 was recently introduced in python3.11. I suspect that Numpy will likely move towards eventual support of shape hints / number-of-dimension hints via PEP 646, but it will likely take a long time to implement and roll out. In the meantime, there are 3rd party libraries such as nptyping that provide type hints for Numpy with support for shape hints.
    – Jasha
    Nov 11, 2022 at 9:12
  • 1
    What's the first argument to the np.ndarray[...] type hint and why is it Any in all the examples?
    – Joooeey
    Feb 3 at 11:03
71

Update

Check recent numpy versions for a new typing module

https://numpy.org/doc/stable/reference/typing.html#module-numpy.typing

dated answer

It looks like typing module was developed at:

https://github.com/python/typing

The main numpy repository is at

https://github.com/numpy/numpy

Python bugs and commits can be tracked at

http://bugs.python.org/

The usual way of adding a feature is to fork the main repository, develop the feature till it is bomb proof, and then submit a pull request. Obviously at various points in the process you want feedback from other developers. If you can't do the development yourself, then you have to convince someone else that it is a worthwhile project.

cython has a form of annotations, which it uses to generate efficient C code.


You referenced the array-like paragraph in numpy documentation. Note its typing information:

A simple way to find out if the object can be converted to a numpy array using array() is simply to try it interactively and see if it works! (The Python Way).

In other words the numpy developers refuse to be pinned down. They don't, or can't, describe in words what kinds of objects can or cannot be converted to np.ndarray.

In [586]: np.array({'test':1})   # a dictionary
Out[586]: array({'test': 1}, dtype=object)

In [587]: np.array(['one','two'])  # a list
Out[587]: 
array(['one', 'two'], 
      dtype='<U3')

In [589]: np.array({'one','two'})  # a set
Out[589]: array({'one', 'two'}, dtype=object)

For your own functions, an annotation like

def foo(x: np.ndarray) -> np.ndarray:

works. Of course if your function ends up calling some numpy function that passes its argument through asanyarray (as many do), such an annotation would be incomplete, since your input could be a list, or np.matrix, etc.


When evaluating this question and answer, pay attention to the date. 484 was a relatively new PEP back then, and code to make use of it for standard Python still in development. But it looks like the links provided are still valid.

6
  • 1
    What software, editor or interpreter are you using that makes use of annotations? As best I know, in plain Python 3, a function gets a __annotations__ dictionary, but the interpreter does nothing with it.
    – hpaulj
    Mar 2, 2016 at 0:26
  • 1
    Do you want typing annotations added to existing numpy functions (including np.array), or just types that would make it easier to add annotations to your own functions?
    – hpaulj
    Mar 2, 2016 at 0:47
  • 3
    I've marked this answer as the accepted one, but just for completeness, I was going for the latter (type hinting in my own code, which uses Numpy). I'm all for Duck Typing, but when you can provide static type information, I don't see why you wouldn't, if only for static code analysis (PyCharm does warn about incompatible types). Thanks, @hpaulj!
    – Inon
    Mar 2, 2016 at 19:03
  • 2
    What about the shape? I can add hints like def blah() -> np.ndarray(785): But I can't can't add a second dimension like -> np.ndarray(785, 10). Having a shape hint is very helpful and brings clarity to multiple functions in my code that produce arrays of varying dimensionality.
    – Steve3p0
    Jul 15, 2020 at 2:35
  • 1
    @Steve3p0 support for shapes is work-in-progress in python (peps.python.org/pep-0646) as well as numpy (github.com/numpy/numpy/issues/16544)
    – APaul
    Oct 3, 2022 at 0:08
29

At my company we've been using:

from typing import TypeVar, Generic, Tuple, Union, Optional
import numpy as np

Shape = TypeVar("Shape")
DType = TypeVar("DType")

class Array(np.ndarray, Generic[Shape, DType]):
    """  
    Use this to type-annotate numpy arrays, e.g. 
        image: Array['H,W,3', np.uint8]
        xy_points: Array['N,2', float]
        nd_mask: Array['...', bool]
    """
    pass

def compute_l2_norm(arr: Array['N,2', float]) -> Array['N', float]:
    return (arr**2).sum(axis=1)**.5

print(compute_l2_norm(arr = np.array([(1, 2), (3, 1.5), (0, 5.5)])))

We actually have a MyPy checker around this that checks that the shapes work out (which we should release at some point). Only thing is it doesn't make PyCharm happy (ie you still get the nasty warning lines):

enter image description here

6
  • 1
    any updates on the MyPy checker? would love to integrate it to my env Jun 6, 2021 at 12:08
  • 3
    This is good stuff, thanks for sharing. It seems, however, that the nptyping package (github.com/ramonhagenaars/nptyping) considerably generalizes this.
    – amka66
    Nov 19, 2021 at 18:10
  • I still find myself using this version over nptyping for documentation purposes, because I find aArray['2,2',int] easier to type than NDArray[Shape["2, 2"], Int], and you can give meaning via what you name dimensions, e.g. BGRImageArray = Array['H,W,3', 'uint8'] makes it clear that the first dimension is height. That said, if you actually intend to use mypy for type checking, definitely go for nptyping.
    – Peter
    Feb 6 at 19:13
  • This sort of reinvents the wheel (albeit not a very good one), with numpy.typing being a thing.
    – usernumber
    Mar 1 at 14:29
  • @amka66 Puzzlingly, nptyping doesn't currently seem to allow Mypy to check shape mismatches, so in that sense it is worse than the solution in this answer if you don't need support for all that extra stuff like recarrays...
    – smheidrich
    Apr 29 at 10:05
9

nptyping adds lots of flexibility for specifying numpy type hints.

1
  • nptyping is a life changer.. it really solved my problems when typing numpy arrays. It works great with unittests when one needs to verify the instance type, shape, etc. I highly recommend using it! Nov 22 at 19:04
-2

What i did was to just define it as

Dict[Tuple[int, int], TYPE]

So for example if you want an array of floats you can do:

a = numpy.empty(shape=[2, 2], dtype=float) # type: Dict[Tuple[int, int], float]

This is of course not exact from a documentation perspective, but for analyzing correct usage and getting proper completion with pyCharm it works great!

3
  • 30
    this is worse than using np.ndarray as a type Sep 30, 2019 at 15:28
  • 1
    @JulesG.M., may I know what's the difference of using np.array and NDArray as a type? if you have a quick answer.
    – Jiadong
    Sep 5, 2022 at 1:57
  • this is an old comment, from before NDArray came to exist @Jiadong. NDArray is better now because it has tools to also indicate the dtype of the array Sep 6, 2022 at 17:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.