53

I am trying to understand the lower level implementations of python 3. There is one module named _posixsubprocess used by the subprocess module. I tried to find the location of this module in my system and found that it's a stub file.

Could someone guide me as I have no idea about what are the stub files and how are they implemented at the lower level?

7
  • Maybe you were looking for this? Modules/_posixsubprocess.c
    – exhuma
    Nov 27, 2019 at 9:23
  • So it is essentially a Cython module that can directly be integrated into python interpreter being a C compiled code, correct me if I am wrong? Nov 27, 2019 at 11:05
  • "directly integrated" is maybe not the wording I would choose. When compiled it will become a .so/.pyd file which can be imported if it's findable on the path. You can "easily" create your own modules in C. I doubt that it's cython though. It's more likely that it uses the standard way of writing C extensions
    – exhuma
    Nov 27, 2019 at 12:24
  • btw: You mention _posixsubprocess, which is - in CPython - a .c file, But you also mention that you see it as a .pyi file. Where exactly did you find that .pyi file? Do you have a link to the Python source repository?
    – exhuma
    Nov 27, 2019 at 12:28
  • 1
    I was confusing the standard way of writing C extensions for python with python. It's cleared now. Also, I found this link which will be quite helpful to understand implementation level details realpython.com/cpython-source-code-guide/… Nov 27, 2019 at 19:34

1 Answer 1

84

_posixsubprocess

The file you are referencing is a Python module written in C. It's not a "stub" file. The real implementation can be found in the stdlib at Modules/_posixsubprocess.c. You can see how writing a C/C++ extension is written by having a look at Building C and C++ Extensions. This should help you understanding the code in _posixsubprocess.c.

In order to add type-hints to that file (which is an "Extension Module" as it is written in C), the type hints are added to a "stub" file with the extension .pyi.

That file can be found in the typeshed which is a collection of stub files. The typeshed also contains stubs for third-party modules which is a historical remnant. That is no longer needed since PEP-561 has been adopted.

Concerning stub/pyi files

Stub files contain type-hinting information of normal Python modules. The full official documentation can be found in the section about stub-files in PEP-484.

For example, if you have a Python module mymodule.py like this:

def myfunction(name):
   return "Hello " + name

Then you can add type-hints via a stub-file mymodule.pyi. Note that here the ellipsis (...) is part of the syntax, so the code-block below really shows the complete file contents:

def myfunction(name: str) -> str: ...

They look very similar to C header files in that they contain only the function signatures, but their use is purely optional.

You can also add type hints directly in the .py module like the following:

def myfunction(name: str) -> str:
   return "Hello " + name

But there are some cases where you want to keep them separate in stubs:

  • You want to keep your code Python 2 compatible and don't like the # type: ... comment syntax
  • You use function annotations for something else but still want to use type-hints
  • You are adding type-hints into an existing code-base and want to keep code-churn in existing files minimal

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.