1

From a comment by Jim Fasarakis Hilliard:

Generators: def functions that contain one or more yield expressions.

Generators are used as data producers (they yield values).

I can understand that.

Generator-based coroutine: A generator (def + yield) that is wrapped by types.coroutine. You need to wrap it in types.coroutine if you need it to be considered a coroutine object.

Generator-based coroutines are used as consumers (you .send values to them or to a sub-generator they yield from).

What does "consumers (you .send values to them or to a sub-generator they yield from)" mean?

Asynchronous Generator: async def functions that contain a one or more yield expressions. These can also contain await expressions.

Asynchronous generators are asynchronous data producers.

What does "asynchronous data producers" mean?

Coroutine: async def without zero or more awaits and no yields.

coroutines are asynchronous data consumers.

What does "asynchronous data consumers" mean?

Thanks.

1 Answer 1

4

In python, generators are NOW used in a number of various ways. The original purpose of generators was to suspend execution and then yield a value back to the caller. The caller can then call next later to resume the generator. Hence generators were data producers.

Now the above version of generators only allowed returning data through the yield statement. Now for a function to be a coroutine, it should also accept values from the caller. Hence PEP 342 was introduced in python 2.5 to enhance generators so that they can act as full fledged coroutines. This allowed callers to send values to the generators.

Now the new issue was that when generators were refactored and you wanted to delegate parts of its operation to subgenerators, you need to explicitly invoke the subgenerator as an iterator, propagate the data send by caller and handle exception. To simplify the operation of subgenerators, a new operation yield from was defined in PEP 380 as part of python 3.3. The yield from is syntactically is much more than the plain yield syntax. In a perfect world, a new keyword probably would have used.

Now the issue was that generators were used in two different contexts. As an iterator and as a coroutine. It would have been better if a generator can be explicitly defined as a coroutine. Hence PEP 492 introduced async and await keywords in Python 3.5. Hence any generator which was used as a coroutine was indicated by the async keyword. The coroutine in Python 3.5 can use await keyword instead of the yield from. Note that from python 3.5 onwards coroutines are a different type!!

Now assume you have a generator function with def and yield. You can convert an existing generator type to a coroutine type using the types.coroutine decorator. These are consumers who can accept values through send() and delegate the same to subgenerators using yield from.

In python 3.5, you can use async to indicate that the function is a coroutine type. Such a function can contain plain yield and await. They cannot contain yield from (since await replaces the feature). When a coroutine contains plain yield, they are the lowest in a chain of generator calls and hence called an asynchronous data producer.

Any coroutine without plain yield will be a data consumer since it must call another coroutine through await to get asynchronous data.

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.