8

In Pycharm, the following code produces a warning:

from typing import List

list1: List[int] = [1, 2, 3]
list2: List[str] = ["1", "2", "3"]
list3: List[object] = list1 + list2
#                             ↳ Expected type List[int] (matched generic type List[_T]),
#                               got List[str] instead.

Why? Should I not be concatenating two lists of mixed, hinted types?

8
  • 3
    It's common to use lists for the same type of objects. It's not an error to mix the contents, but I consider it bad style.
    – Matthias
    Jun 24, 2019 at 14:18
  • @Matthias Why bad style? Say I want a list of objects (or some supertype) and need to build it piecewise (lists of subtypes) — is there a preferred way?
    – Cai
    Jun 24, 2019 at 14:27
  • What happens if you make list1 and list2 annotated to also be of type List[object]? That makes your program type-check under at least mypy -- perhaps it'll also satisfy Pycharm. Jun 24, 2019 at 15:49
  • @Michael0x2a That does indeed get rid of the warning, but it doesn't answer the "why"... For example, if list1 and list2 aren't themselves type-hinted, but are instead assigned the output of functions with hinted return types (e.g. def gli() -> List[int]: return [1, 2, 3] then list1 = gli()), I still get the warning. What if these functions are library code? Why would concatenating their outputs be bad?
    – Cai
    Jun 25, 2019 at 12:22
  • I think that's just how the type signatures for list are defined -- e.g. see the signature for list.__add__ in typeshed, the canonical collection of type hints for the stdlib. The type signature could perhaps be broadened (e.g. overloaded to also accept List[S] and return List[Union[T, S]]) but I suspect that kind of thing isn't too useful in practice and makes life harder for people who want strictly homogeneous lists or want to subclass them. Jun 25, 2019 at 14:02

3 Answers 3

7

As requested in the comments, here are some reasons why type checkers don't allow this.

The first reason is somewhat prosaic: the type signature of list.__add__ simply doesn't allow for anything other then a list containing the same type to be passed in:

_T = TypeVar('_T')

# ...snip...

class list(MutableSequence[_T], Generic[_T]):

    # ...snip...

    def __add__(self, x: List[_T]) -> List[_T]: ...

And Pycharm, which supports PEP 484, uses (in part) data from Typeshed.

It's possible that we could broaden this type signature in some way (e.g. overload it to also accept a List[_S] and return List[Union[_T, _S]] in that case), but I don't think anybody's bothered to investigate the feasibility of this approach: this sort of thing isn't too useful in practice, makes life harder for people who want strictly homogeneous lists or want to subclass them, and would potentially disrupt a lot of existing code that relies on the current type signature.

This type signature is also probably a reflection of the broader choice made during the initial design of PEP 484 to assume that lists are always homogenous -- always contains values of the same type.

The designers of PEP 484 strictly speaking didn't need to make this choice: they could have required type checkers to special-case interactions with it, like we currently do for tuples. But it's overall simpler not to do this, I think. (And also arguably better style, but whatever.)


The second reason has to do with a fundamental limitation of the PEP 484 type system: there's no way to declare that some function or method does not modify state.

Basically, the behavior you want is safe only if lst1.__add__(lst2) is guaranteed to not mutate either operands. But there's no way of actually guaranteeing this -- what if lst1 is some weird list subclass that copies items from lst2 to itself? Then temporarily relaxing lst1's type from SomeListSubtype[int] to SomeListSubtype[object] would be unsafe: lst1 would no longer contain only ints after adding/injecting the strings from lst2.

Of course, actually writing such a subclass is also bad practice, but type checkers don't have the luxury of assuming users will follow best practices if they're not enforced: type checkers, compilers, and similar tools are fundamentally conservative beasts.


And finally, it's worth noting that none of these problems are intrinsically insurmountable. There are several things type checker implementers could do, such as:

  1. Tinkering with the type signature of list (and making sure it doesn't break any existing code)
  2. Introduce some sort of way of declaring that a method is pure -- does no mutation. Basically, generalize the ideas behind PEP 591 to also apply to functions. (But this would require writing a PEP, modifying typeshed to use the new typing construct, doing a lot of careful design and implementation work...)
  3. Maybe special-case this interaction when we know for certain the two variables are not subclasses of lists. (But realistically, the number of times we'd know this for certain is pretty limited.)

...and so forth.

But all of these things take time and energy to do: it's a matter of prioritization. The issue tracker for PyCharm (and mypy, etc) are pretty long, and there's no shortages of other bugs/feature requests to work through.

0
3

Reasons why are explained explicitly in other comments, so I would like just to highlight potential work-arounds for those who can't skip the concatenation step in their code and don't want to see this annoying warning.

In the above case, the operation that produced no warning:

list1 += list2

While sometimes I had to do:

[*list1, *list2]

0

Like Pycharm says it's just a warning you are allowed to concatenate different objects or lists but it is considered to be a bad practice.

1
  • 4
    I understand the warning is probably there for a reason. But why is this considered bad practice?
    – Cai
    Jun 25, 2019 at 12:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.