In python, generators are NOW used in a number of various ways. The original purpose of generators was to suspend execution and then yield
a value back to the caller. The caller can then call next later to resume the generator. Hence generators were data producers.
Now the above version of generators only allowed returning data through the yield
statement. Now for a function to be a coroutine, it should also accept values from the caller. Hence PEP 342 was introduced in python 2.5 to enhance generators so that they can act as full fledged coroutines. This allowed callers to send values to the generators.
Now the new issue was that when generators were refactored and you wanted to delegate parts of its operation to subgenerators, you need to explicitly invoke the subgenerator as an iterator, propagate the data send by caller and handle exception. To simplify the operation of subgenerators, a new operation yield from
was defined in PEP 380 as part of python 3.3. The yield from
is syntactically is much more than the plain yield syntax. In a perfect world, a new keyword probably would have used.
Now the issue was that generators were used in two different contexts. As an iterator and as a coroutine. It would have been better if a generator can be explicitly defined as a coroutine. Hence PEP 492 introduced async
and await
keywords in Python 3.5. Hence any generator which was used as a coroutine was indicated by the async
keyword. The coroutine in Python 3.5 can use await
keyword instead of the yield from
. Note that from python 3.5 onwards coroutines are a different type!!
Now assume you have a generator function with def
and yield
. You can convert an existing generator type to a coroutine type using the types.coroutine
decorator. These are consumers who can accept values through send()
and delegate the same to subgenerators using yield from
.
In python 3.5, you can use async
to indicate that the function is a coroutine type. Such a function can contain plain yield
and await
. They cannot contain yield from
(since await
replaces the feature). When a coroutine contains plain yield
, they are the lowest in a chain of generator calls and hence called an asynchronous data producer.
Any coroutine without plain yield
will be a data consumer since it must call another coroutine through await
to get asynchronous data.