-2

Per PEP-492 I am trying to implement an asynchronous iterator, such that I can do e.g.

async for foo in bar:
    ...

Here is a trivial example, similar to the one in the docs, with a very basic test of instantiation and async iteration:

import pytest

class TestImplementation:
    def __aiter__(self):
        return self
    async def __anext__(self):
        raise StopAsyncIteration


@pytest.mark.asyncio  # note use of pytest-asyncio marker
async def test_async_for():
    async for _ in TestImplementation():
        pass

However, when I execute my test suite, I see:

=================================== FAILURES ===================================
________________________________ test_async_for ________________________________

    @pytest.mark.asyncio
    async def test_async_for():
>       async for _ in TestImplementation():
E       TypeError: 'async for' received an invalid object from __aiter__: TestImplementation

...: TypeError
===================== 1 failed, ... passed in 2.89 seconds ======================

Why does my TestImplementation appear to be invalid? As far as I can tell it meets the protocol:

  1. An object must implement an __aiter__ method ... returning an asynchronous iterator object.
  2. An asynchronous iterator object must implement an __anext__ method ... returning an awaitable.
  3. To stop iteration __anext__ must raise a StopAsyncIteration exception.

This is failing with the latest released versions of Python (3.5.1), py.test (2.9.2) and pytest-asyncio (0.4.1).

9
  • Works for me with pytest 2.9.2. What version are you using? Jun 25, 2016 at 18:07
  • @MikeMüller pytest-2.8.5 and asyncio-0.3.0 plugin, will see if an upgrade helps things...
    – jonrsharpe
    Jun 25, 2016 at 18:08
  • @MikeMüller nope, same outcome with pytest-2.9.2 and asyncio-0.4.1 plugin
    – jonrsharpe
    Jun 25, 2016 at 18:10
  • I'm using Python 3.5.1. Jun 25, 2016 at 18:11
  • 1
    @PadraicCunningham the above will start to work in 3.5.2 (tomorrow!) and 3.6.x, it's only the first two patch releases of 3.5.x that fail...
    – jonrsharpe
    Jun 25, 2016 at 18:40

2 Answers 2

47

If you read a little further down the documentation it mentions that (emphasis mine):

PEP 492 was accepted in CPython 3.5.0 with __aiter__ defined as a method, that was expected to return an awaitable resolving to an asynchronous iterator.

In 3.5.2 (as PEP 492 was accepted on a provisional basis) the __aiter__ protocol was updated to return asynchronous iterators directly.

Therefore for versions prior to 3.5.2 (released 2016/6/27) the documentation is slightly out of step with how to write a working asynchronous iterator. The fixed version for 3.5.0 and 3.5.1 looks like:

class TestImplementation:
    async def __aiter__(self):
  # ^ note
        return self
    async def __anext__(self):
        raise StopAsyncIteration

This was introduced on closing bug #27243 and is a little clearer in the data model documentation, which also suggests a way of writing backwards compatible code.

0
5

Asynchronous iterators have been implemented in Python 3.6 - see PEP-525

Then you don't need your TestImplementation at all in order to use async for. You can just use yield (example taken from PEP-525):

async def ticker(delay, to):
    """Yield numbers from 0 to `to` every `delay` seconds."""
    for i in range(to):
        yield i
        await asyncio.sleep(delay)

You can then use async for as you would expect:

async for i in ticker(1, 10):                                                                     
    print(f'Tick #{i}')
1
  • 5
    This is a generator which is not the same thing as an iterator. Not all iterators can be refactored into generators, nor should they be. The point of the OP's question was to create an iterator, not test an async for loop. Sep 16, 2020 at 14:26

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.