221

Is there a simple way of testing if the generator has no items, like peek, hasNext, isEmpty, something along those lines?

6
  • Correct me if I'm wrong, but if you could make a truly generic solution to any generator, it would be the equivalent of setting breakpoints on the yield statements and having the ability to "step backward". Would that mean cloning the stack frame on yields and restoring them on StopIteration?
    – user44484
    Mar 19, 2009 at 13:07
  • Well, I guess restore them StopIteration or not, but at least StopIteration would tell you it was empty. Yeah I need sleep...
    – user44484
    Mar 19, 2009 at 13:19
  • 4
    I think I know why he wants this. If you're doing web development with templates, and passing the return value into a template like Cheetah or something, empty list [] is conveniently Falsey so you can do an if check on it and do special behavior for something or nothing. Generators are true even if they yield no elements.
    – jpsimons
    Mar 8, 2011 at 5:02
  • 1
    Here's my use case... I'm using glob.iglob("filepattern") on a user-supplied wildcard pattern, and I want to warn the user if the pattern does not match any files. Sure I can work around this in various ways, but it's useful to be able to cleanly test whether the iterator came up empty or not.
    – LarsH
    Jan 24, 2013 at 16:48
  • May be use this solution: stackoverflow.com/a/11467686/463758
    – balki
    Nov 17, 2015 at 20:15

25 Answers 25

143

Suggestion:

def peek(iterable):
    try:
        first = next(iterable)
    except StopIteration:
        return None
    return first, itertools.chain([first], iterable)

Usage:

res = peek(mysequence)
if res is None:
    # sequence is empty.  Do stuff.
else:
    first, mysequence = res
    # Do something with first, maybe?
    # Then iterate over the sequence:
    for element in mysequence:
        # etc.
7
  • 3
    I don't quite get the point of returning the first element twice in return first, itertools.chain([first], rest).
    – njzk2
    Aug 15, 2014 at 19:52
  • 11
    @njzk2 I was going for a "peek" operation (hence the function name). wiki "peek is an operation which returns the value of the top of the collection without removing the value from the data"
    – John Fouhy
    Jun 10, 2015 at 22:04
  • 2
    This won't work if the generator is designed to yield None. def gen(): for pony in range(4): yield None if pony == 2 else pony
    – Paul
    Nov 3, 2016 at 21:53
  • 9
    @Paul Look at the return values closely. If the generator is done -- i.e., not returning None, but raising StopIteration -- the result of the function is None. Otherwise, it's a tuple, which is not None.
    – Nic
    Jan 20, 2017 at 23:57
  • 2
    Won't a large number of peek calls create a never ending chain of itertools.chain objects containing references to other itertools.chain objects? Nov 22, 2018 at 18:38
75

The simple answer to your question: no, there is no simple way. There are a whole lot of work-arounds.

There really shouldn't be a simple way, because of what generators are: a way to output a sequence of values without holding the sequence in memory. So there's no backward traversal.

You could write a has_next function or maybe even slap it on to a generator as a method with a fancy decorator if you wanted to.

11
  • 2
    fair enough, that makes sense. i knew there was no way of finding the length of a generator, but thought i might have missed a way of finding if it is initially going to generate anything at all.
    – Dan
    Mar 19, 2009 at 17:31
  • 1
    Oh, and for reference, I tried implementing my own "fancy decorator" suggestion. HARD. Apparently copy.deepcopy doesn't work on generators. Mar 19, 2009 at 19:50
  • 91
    I'm not sure I can agree with "there shouldn't be a simple way". There are plenty of abstractions in computer science that are designed to output a sequence of values without holding the sequence in memory, but that allow the programmer to ask whether there is another value without removing it from the "queue" if there is. There is such thing as single peek-ahead without requiring "backward traversal". That's not to say an iterator design must supply such a feature, but it sure is useful. Maybe you're objecting on the basis that the first value might change after the peek?
    – LarsH
    Jan 24, 2013 at 16:50
  • 12
    I'm objecting on the grounds that a typical implementation doesn't even calculate a value until it is needed. One could force the interface to do this, but that might be sub-optimal for lightweight implementations. Jan 30, 2013 at 20:54
  • 10
    @S.Lott you don't need to generate the entire sequence to know if the sequence is empty or not. One element's worth of storage is sufficient - see my answer. Jun 13, 2014 at 6:34
48

A simple way is to use the optional parameter for next() which is used if the generator is exhausted (or empty). For example:

_exhausted  = object()

if next(some_generator, _exhausted) is _exhausted:
    print('generator is empty')
9
  • 1
    Why objects and all that stuff? Simply: if next(itreable,-1) == -1 then the gen is empty!
    – Apostolos
    Oct 1, 2020 at 20:37
  • 5
    @Apostolos Because next(iter([-1, -2, -3]), -1) == -1 is True. In other words, any iterable with first element equal to -1 will appear as empty using your condition.
    – Jeyekomon
    Oct 5, 2020 at 10:13
  • 2
    @Apostolos In the simple case, yes, that is the solution. But it fails if you plan to create a general tool for any iterable, without constraints.
    – Jeyekomon
    Oct 6, 2020 at 15:40
  • 2
    @Apostolos The object() is the extraordinary value that won't be contained in the generator.
    – Mikko Koho
    Oct 22, 2020 at 14:27
  • 10
    N.B.; this is still a "peek" function and will take off one element from the generator.
    – phlaxyr
    Nov 3, 2020 at 19:24
27

Quick-dirty solution:

next(my_generator(), None) is not None

Or replace None by whatever value you know it's not in your generator.

Edit: Yes, this will skip 1 item in the generator. Sometimes, however, I check whether a generator is empty only for validation purposes, then don't really use it. Otherwise, I do something like:

def foo(self):
    if next(self.my_generator(), None) is None:
        raise Exception("Not initiated")

    for x in self.my_generator():
        ...

That is, this works if your generator comes from a function, as in my_generator().

5
  • 5
    Why this is not the best answer? In case the generator returns None?
    – Sait
    Jun 29, 2016 at 17:48
  • 11
    Probably because this forces you to actually consume the generator instead of just testing if it’s empty.
    – bfontaine
    Aug 9, 2016 at 22:03
  • 6
    It's bad because the moment you call next(generator, None) you will skip 1 item if it is available
    – Nathan Do
    Nov 30, 2016 at 8:06
  • 2
    Correct, you are going to miss 1st element of your gen and also you are going to consume your gen rather testing if its empty.
    – A.J.
    Mar 5, 2020 at 7:35
  • This is not a general purpose solution as it will only work for those generators for which we know beforehand a value the generator will never return, e.g., None.
    – Mikko Koho
    Oct 29, 2021 at 18:14
18

The best approach, IMHO, would be to avoid a special test. Most times, use of a generator is the test:

thing_generated = False

# Nothing is lost here. if nothing is generated, 
# the for block is not executed. Often, that's the only check
# you need to do. This can be done in the course of doing
# the work you wanted to do anyway on the generated output.
for thing in my_generator():
    thing_generated = True
    do_work(thing)

If that's not good enough, you can still perform an explicit test. At this point, thing will contain the last value generated. If nothing was generated, it will be undefined - unless you've already defined the variable. You could check the value of thing, but that's a bit unreliable. Instead, just set a flag within the block and check it afterward:

if not thing_generated:
    print "Avast, ye scurvy dog!"
4
  • 4
    This solution will try to consume the whole generator thus making it unusable for infinite generators. Jul 2, 2013 at 12:53
  • 1
    @ViktorStískala: I don't see your point. It would be foolish to test if an infinite generator produced any results.
    – vezult
    Nov 7, 2013 at 21:24
  • I wanted to point out that your solution could contain break in the for loop, because you are not processing the other results and it's useless for them to generated. range(10000000) is finite generator (Python 3), but you don't need to go through all the items to find out if it generates something. Nov 12, 2013 at 14:53
  • 3
    @ViktorStískala: Understood. However, my point is this: Generally, you actually want to operate on the generator output. In my example, if nothing is generated, you now know it. Otherwise, you operate on the generated output as intended - "The use of the generator is the test". No need for special tests, or pointlessly consuming the generator output. I've edited my answer to clarify this.
    – vezult
    Nov 12, 2013 at 15:39
9

Just fell on this thread and realized that a very simple and easy to read answer was missing:

def is_empty(generator):
    for item in generator:
        return False
    return True

If we are not suppose to consume any item then we need to re-inject the first item into the generator:

def is_empty_no_side_effects(generator):
    try:
        item = next(generator)
        def my_generator():
            yield item
            yield from generator
        return my_generator(), False
    except StopIteration:
        return (_ for _ in []), True

Example:

>>> g=(i for i in [])
>>> g,empty=is_empty_no_side_effects(g)
>>> empty
True
>>> g=(i for i in range(10))
>>> g,empty=is_empty_no_side_effects(g)
>>> empty
False
>>> list(g)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
8

I hate to offer a second solution, especially one that I would not use myself, but, if you absolutely had to do this and to not consume the generator, as in other answers:

def do_something_with_item(item):
    print item

empty_marker = object()

try:
     first_item = my_generator.next()     
except StopIteration:
     print 'The generator was empty'
     first_item = empty_marker

if first_item is not empty_marker:
    do_something_with_item(first_item)
    for item in my_generator:
        do_something_with_item(item)

Now I really don't like this solution, because I believe that this is not how generators are to be used.

8

Prompted by Mark Ransom, here's a class that you can use to wrap any iterator so that you can peek ahead, push values back onto the stream and check for empty. It's a simple idea with a simple implementation that I've found very handy in the past.

class Pushable:

    def __init__(self, iter):
        self.source = iter
        self.stored = []

    def __iter__(self):
        return self

    def __bool__(self):
        if self.stored:
            return True
        try:
            self.stored.append(next(self.source))
        except StopIteration:
            return False
        return True

    def push(self, value):
        self.stored.append(value)

    def peek(self):
        if self.stored:
            return self.stored[-1]
        value = next(self.source)
        self.stored.append(value)
        return value

    def __next__(self):
        if self.stored:
            return self.stored.pop()
        return next(self.source)
1
  • UPDATE: I thought it was worth turning this into a PyPI package, since I have used it quite a few times in the past. Please find a slightly more elaborate version here - pypi.org/project/pushable
    – sfkleach
    Mar 21 at 11:27
6

All you need to do to see if a generator is empty is to try to get the next result. Of course if you're not ready to use that result then you have to store it to return it again later.

Here's a wrapper class that can be added to an existing iterator to add an __nonzero__ test, so you can see if the generator is empty with a simple if. It can probably also be turned into a decorator.

class GenWrapper:
    def __init__(self, iter):
        self.source = iter
        self.stored = False

    def __iter__(self):
        return self

    def __nonzero__(self):
        if self.stored:
            return True
        try:
            self.value = next(self.source)
            self.stored = True
        except StopIteration:
            return False
        return True

    def __next__(self):  # use "next" (without underscores) for Python 2.x
        if self.stored:
            self.stored = False
            return self.value
        return next(self.source)

Here's how you'd use it:

with open(filename, 'r') as f:
    f = GenWrapper(f)
    if f:
        print 'Not empty'
    else:
        print 'Empty'

Note that you can check for emptiness at any time, not just at the start of the iteration.

3
  • This is headed in the right direction. It should be modified to allow peeking ahead as far as you wish, storing as many results as needed. Ideally it would allow for pushing of arbitrary items onto the head of the stream. A pushable-iterator is a very useful abstraction that I often use.
    – sfkleach
    Jun 12, 2016 at 19:04
  • 1
    @sfkleach I don't see a need to complicate this for multiple peek-ahead, it's quite useful as is and answers the question. Even though this is an old question it's still getting the occasional look, so if you want to leave your own answer someone might find it useful. Jun 22, 2018 at 20:50
  • Mark's quite right that his solution answers the question, which is the key point. I should have phrased it better. What I meant was that pushable-iterators with unbounded pushback are an idiom I have found extremely useful & the implementation is arguably even simpler. As suggested I will post the variant code.
    – sfkleach
    Jan 7, 2020 at 12:13
4

I realize that this post is 5 years old at this point, but I found it while looking for an idiomatic way of doing this, and did not see my solution posted. So for posterity:

import itertools

def get_generator():
    """
    Returns (bool, generator) where bool is true iff the generator is not empty.
    """
    gen = (i for i in [0, 1, 2, 3, 4])
    a, b = itertools.tee(gen)
    try:
        a.next()
    except StopIteration:
        return (False, b)
    return (True, b)

Of course, as I'm sure many commentators will point out, this is hacky and only works at all in certain limited situations (where the generators are side-effect free, for example). YMMV.

2
  • 4
    This will only call the gen generator once for each item, so side-effects are not too bad a problem. But it will store a copy of everything that has been pulled from the generator via b, but not via a, so the memory implications are similar to just running list(gen) and checking that. May 8, 2017 at 22:40
  • It has two issues. 1. This itertool may require significant auxiliary storage (depending on how much temporary data needs to be stored). In general, if one iterator uses most or all of the data before another iterator starts, it is faster to use list() instead of tee(). 2. tee iterators are not threadsafe. A RuntimeError may be raised when using simultaneously iterators returned by the same tee() call, even if the original iterable is threadsafe.
    – A.J.
    Mar 5, 2020 at 7:43
3

Sorry for the obvious approach, but the best way would be to do:

for item in my_generator:
     print item

Now you have detected that the generator is empty while you are using it. Of course, item will never be displayed if the generator is empty.

This may not exactly fit in with your code, but this is what the idiom of the generator is for: iterating, so perhaps you might change your approach slightly, or not use generators at all.

4
  • Or... questioner could provide some hint as to why one would try to detect an empty generator?
    – S.Lott
    Mar 19, 2009 at 10:17
  • did you mean "nothing will be displayed since generator is empty"? Mar 19, 2009 at 10:17
  • S.Lott. I agree. I can't see why. But I think even if there was a reason, the problem might be better turned to use each item instead.
    – Ali Afshar
    Mar 19, 2009 at 10:25
  • 4
    This does not tell the program if the generator was empty. Nov 1, 2011 at 19:33
3
>>> gen = (i for i in [])
>>> next(gen)
Traceback (most recent call last):
  File "<pyshell#43>", line 1, in <module>
    next(gen)
StopIteration

At the end of generator StopIteration is raised, since in your case end is reached immediately, exception is raised. But normally you shouldn't check for existence of next value.

another thing you can do is:

>>> gen = (i for i in [])
>>> if not list(gen):
    print('empty generator')
3
  • 2
    Which does actually consume the whole generator. Sadly, it's not clear from the question if this is desirable or undesirable behavior.
    – S.Lott
    Mar 19, 2009 at 10:15
  • as any other way of "touching" generator, I suppose. Mar 19, 2009 at 10:16
  • 1
    I realise this is old, but using 'list()' can't be the best way, if the list generated is not empty but in fact large then this is needlessly wasteful Feb 7, 2017 at 18:16
3

I found only this solution as working for empty iterations as well.

def is_generator_empty(generator):
    a, b = itertools.tee(generator)
    try:
        next(a)
    except StopIteration:
        return True, b
    return False, b

is_empty, generator = is_generator_empty(generator)

Or if you do not want to use exception for this try to use

def is_generator_empty(generator):
    a, b = itertools.tee(generator)
    for item in a:
        return False, b
    return True, b

is_empty, generator = is_generator_empty(generator)

In the marked solution you are not able to use it for empty generators like

def get_empty_generator():
    while False:
        yield None 

generator = get_empty_generator()
1

If you need to know before you use the generator, then no, there is no simple way. If you can wait until after you have used the generator, there is a simple way:

was_empty = True

for some_item in some_generator:
    was_empty = False
    do_something_with(some_item)

if was_empty:
    handle_already_empty_generator_case()
1

Simply wrap the generator with itertools.chain, put something that will represent the end of the iterable as the second iterable, then simply check for that.

Ex:

import itertools

g = some_iterable
eog = object()
wrap_g = itertools.chain(g, [eog])

Now all that's left is to check for that value we appended to the end of the iterable, when you read it then that will signify the end

for value in wrap_g:
    if value == eog: # DING DING! We just found the last element of the iterable
        pass # Do something
2
  • Use eog = object() instead of assuming that float('-inf') will never occur in the iterable.
    – bfontaine
    Aug 9, 2016 at 22:04
  • @bfontaine Good idea
    – smac89
    Aug 9, 2016 at 22:22
1

In my case I needed to know if a host of generators was populated before I passed it on to a function, which merged the items, i.e., zip(...). The solution is similar, but different enough, from the accepted answer:

Definition:

def has_items(iterable):
    try:
        return True, itertools.chain([next(iterable)], iterable)
    except StopIteration:
        return False, []

Usage:

def filter_empty(iterables):
    for iterable in iterables:
        itr_has_items, iterable = has_items(iterable)
        if itr_has_items:
            yield iterable


def merge_iterables(iterables):
    populated_iterables = filter_empty(iterables)
    for items in zip(*populated_iterables):
        # Use items for each "slice"

My particular problem has the property that the iterables are either empty or has exactly the same number of entries.

1

Use the peek function in cytoolz.

from cytoolz import peek
from typing import Tuple, Iterable

def is_empty_iterator(g: Iterable) -> Tuple[Iterable, bool]:
    try:
        _, g = peek(g)
        return g, False
    except StopIteration:
        return g, True

The iterator returned by this function will be equivalent to the original one passed in as an argument.

1

Just to try to help with my "2 cents", I am going to describe my experience:

I have a generator that I need slicing it using itertools.islice into small generators. Then to check if my sub generators are empty or not, I just convert/consume them to a small list and I check if the list is empty or not.

For example:

from itertools import islice

def generator(max_yield=10):
    a = 0

    while True:
        a += 1

        if a > max_yield:
            raise StopIteration()

        yield a

tg = generator()

label = 1

while True:
    itg = list(islice(tg, 3))

    if not itg:  # <-- I check if the list is empty or not
        break

    for i in itg:
        print(f'#{label} - {i}')

    label += 1

Output:

#1 - 1
#1 - 2
#1 - 3
#2 - 4
#2 - 5
#2 - 6
#3 - 7
#3 - 8
#3 - 9
#4 - 10

Maybe this is not the best approach, mainly because it consumes the generator, however it works to me.

1

Inspecting the generator before iterating over it conforms to the LBYL coding style. Another approach (EAFP) would be to iterate over it and then check whether it was empty or not.

is_empty = True

for item in generator:
    is_empty = False
    do_something(item)

if is_empty:
    print('Generator is empty')

This approach also handles well infinite generators.

0

Here's a simple decorator which wraps the generator, so it returns None if empty. This can be useful if your code needs to know whether the generator will produce anything before looping through it.

def generator_or_none(func):
    """Wrap a generator function, returning None if it's empty. """

    def inner(*args, **kwargs):
        # peek at the first item; return None if it doesn't exist
        try:
            next(func(*args, **kwargs))
        except StopIteration:
            return None

        # return original generator otherwise first item will be missing
        return func(*args, **kwargs)

    return inner

Usage:

import random

@generator_or_none
def random_length_generator():
    for i in range(random.randint(0, 10)):
        yield i

gen = random_length_generator()
if gen is None:
    print('Generator is empty')

One example where this is useful is in templating code - i.e. jinja2

{% if content_generator %}
  <section>
    <h4>Section title</h4>
    {% for item in content_generator %}
      {{ item }}
    {% endfor %
  </section>
{% endif %}
1
  • This calls the generator function twice, so will incur the start-up cost of the generator twice. That could be substantial if, for example, the generator function is a database query.
    – Ian Goldby
    Feb 21, 2019 at 11:21
0

peekable from more-itertools allows checking whether it's exhausted by checking its truth value. Demo with one empty and one non-empty iterator:

from more_itertools import peekable

for source in '', 'foobar':
    it = iter(source)
    
    if it := peekable(it):
        print('values:', *it)
    else:
        print('empty')

Output:

empty
values: f o o b a r
-1

This is an old and answered question, but as no one has shown it before, here it goes:

for _ in generator:
    break
else:
    print('Empty')

You can read more here

2
  • 1
    But how is that useful when you do want to actually do some work with generator items? Just inserting this snippet before main code looks very dirty WA Sep 18, 2020 at 9:59
  • This obviously doesn't work if generator producing more than one item. Sep 21, 2020 at 20:17
-1

There's a very simple solution: if next(generator,-1) == -1 then the generator is empty!

7
  • 4
    This will consume the generator. Jan 15, 2021 at 13:45
  • 1
    To recap: the question is about checking before consuming anything.
    – Wolf
    Jan 15, 2021 at 15:10
  • What consuming are you talking about? This is done once at start! My solution is certainly not wrong!
    – Apostolos
    Jan 16, 2021 at 18:57
  • Although this doesn't answer the exact question as stated, I'm going to upvote it because it does deal with a common case where finding out if a generator would return anything. Quite often I find myself wanting to write something like matches = filter(lambda x: ..., my_list); return next(matches) if any_results(matches) else None. I only just learned that this can be written as matches = filter(lambda x: ..., my_list); return next(matches, None)
    – Tom
    Apr 9, 2021 at 15:14
  • 1
    "My solution is certainly not wrong" is a pretty bold statement. The note about "consuming" means that by applying next to your generator, you're extracting its first value and use it to test its emptiness. The value is not stored in any variable and thus immediately discarded. When you later use the generator, the first value will be missing. Please learn more about generators.
    – Jeyekomon
    Oct 17, 2022 at 8:59
-3

I solved it by using the sum function. See below for an example I used with glob.iglob (which returns a generator).

def isEmpty():
    files = glob.iglob(search)
    if sum(1 for _ in files):
        return True
    return False

*This will probably not work for HUGE generators but should perform nicely for smaller lists

-3

bool(generator) will return the correct result

3
  • easy and elegant! Feb 9, 2022 at 10:55
  • I am not sure that is correct though: >>> def foo(): ... if False: ... yield 99 ... >>> bool(foo()) True >>> next(foo()) Traceback (most recent call last): File "<stdin>", line 1, in <module> StopIteration Tested on Python 3.9.10.
    – sfkleach
    Feb 21, 2022 at 16:34
  • 2
    This is incorrect. An empty generator still gets True when converted to Boolean.
    – xmcp
    Jun 14, 2022 at 18:41

Not the answer you're looking for? Browse other questions tagged or ask your own question.