30

I know of base classes Enum and IntEnum. Both are very helpful but I miss features for flag operations. I don't expect that these two classes implement my wished feature.

Let's construct an example:

class NetlistKind(IntEnum):
  Unknown = 0
  LatticeNetlist = 1
  QuartusNetlist = 2
  XSTNetlist = 4
  CoreGenNetlist = 8
  All = 15

As you can see, I'm already using IntEnum to get arithmetic features for this enum. It would be nice to have something like @unique to ensure all values are a power of two. I can do this by forking enum.unique for my needs. (I'm aware that All is an exception from that rule.)

How is such an enum used?

filter = NetlistKind.LatticeNetlist | NetlistKind.QuartusNetlist

Thanks to the underlaying int bit operations are possible and filter has an internal value of 3.

If would be nice to have a "is flag X set in filter Y" function or even better an operator. I add a magic function for x in y:

@unique
class NetlistKind(IntEnum):
  Unknown = 0
  LatticeNetlist = 1
  QuartusNetlist = 2
  XSTNetlist = 4
  CoreGenNetlist = 8
  All = 15

  def __contains__(self, item):
    return  (self.value & item.value) == item.value

Usage example:

....
  def GetNetlists(self, filter=NetlistKind.All):
    for entity in self._entities:
      for nl in entity.GetNetlists():
        if (nl.kind in filter):
          yield nl

  def GetXilinxNetlists(self):
    return self.GetNetlists(NetlistKind.XSTNetlist | NetlistKind.CoreGenNetlist)

So the questions are:

  • Are there better ways to implement bit fields?
  • Are thete better ways to implement such an 1-D filter? I don't want to use lamdas for such a simple filter condition?
  • Is such solution already included in the Python standard library?
  • How to add this enum extension to the next Python version? :)

Open features:

  • return a list of all active flags in __str__
  • ...?
5
  • 1
    I've recently equipped a flags library of mine with unit tests and published it on pypi. I'm going to finish it's README.rst and add some extra features in the next few days. Its interface is heavily influenced by the standard enum module of python3. Have a look if you are interested: pypi.python.org/pypi/py-flags I've seen discussions about whether flags is a pythonic approach or not. My future updates to the README.rst will have a section that will discuss the pros and cons of using several bools as function args or storing bools in an object or dict VS using sets VS using flags. Apr 25, 2016 at 12:24
  • 1
    Please post your comment as an answer, so I can upvote it! It looks very good and mature. Just one question: Why do I need to give a FQN to the enum? Example: TextStyle('TextStyle.bold'). I think bold is enough, because the namespace is already restricted to TextStyle, because you pass it into its constructor.
    – Paebbels
    Apr 25, 2016 at 13:29
  • 1
    Link only answers aren't welcome on SO I'm afraid... The str() of the enum can be used in other context not only in case of serialization, this is why __str__ returns fqdn. I think str() should be interpretable even without the flags class in context. Actually for the purpose of custom serialization I've provided a to_simple_str() besides the standard __str__. In this case to_simple_str() would emit simply 'bold' and TextStyle('bold') would also work. Actually the pickle serializer support of flags saves only the flags class name and the output of to_simple_str(). Apr 25, 2016 at 13:39
  • 1
    I was actually thinking a lot whether to return fqdn or not in str(). I had the idea of returning a non-fqdn from str() and returning fqdn only as part of repr() and maybe with a utility function but finally I decided to go with fqdn to behave the same way as the str() of the standard enum module. Apr 25, 2016 at 14:12
  • 1
    Regarding link only answers: Add an example of your py-flags library, which replaces my initial example, by using your classes/syntax/.... Then add a link to PyPI (docs, download). I'm doing this in the same way for my library and I had never trouble with SO regarding link only posts or spam. Is totally valid if you explain how your solution solves my issue and then link to a common download plattform. The rules are relaxed if its open source and not a commercial product :)
    – Paebbels
    Apr 25, 2016 at 14:33

3 Answers 3

55

Python 3.6 has added Flag and IntFlag which support the usual bit-wise operations. As a bonus, the resulting values from the bit-wise operations are still members of the original flag class, and are singletons [1].

The aenum library also has this addition and is usable back to Python 2.7.

[1] A bug exists in 3.6.0: if the psuedo-flag members are being created in threads then there may end up being duplicates; this is fixed in 3.6.1 (and never existed in aenum).

4
  • Thanks Ethan. The py-flags module provided by @pasztorpisti is very powerful. Maybe Python 3.6 should look into his module (metaclass based) and incorporate some features. => github.com/pasztorpisti/py-flags
    – Paebbels
    Mar 8, 2017 at 1:43
  • 1
    @Paebbels: Looks like it's pretty similar to the stdlib version. Mar 8, 2017 at 4:18
  • 3
    @Paebbels When it comes to the basic functionality they are almost equivalent so python3.6+ users should consider the std lib version. py-flags contains some extra features that can be considered as either a good or a bad thing and developers can actually add these features to the std lib version if needed using inheritance. py-flags deliberately avoided deriving the Flags from Enum as a design decision. I think the correct relation between enums and flags (if needed) should look like the relation between the enum and "set of enum" types of pascal: wiki.freepascal.org/Set Mar 10, 2017 at 23:15
  • The aenum library is great, just what I needed when I found out that only 3.6 has the new Flag stuff. I can't quite tell if its just the 3.6 code taken and backported or what, but it works well.
    – mbrig
    Sep 21, 2017 at 1:16
23

I've recently published an opensource package py-flags that aims this problem. That library has exactly this functionality and its design is heavily influenced by the python3 enum module.

There are debates about whether it is pythonic enough to implement such a flags class because its functionality has huge overlaps with other methods provided by the language (collection of bool variables, sets, objects with bool attributes or dicts with bool items, ...). For this reason I feel a flags class to be too narrow purpose and/or redundant to make its way to the standard library but in some cases it is much better than the previously listed solutions so having a "pip install"-able library can come in handy.

Your example would look like the following using the py-flags module:

from flags import Flags

class NetlistKind(Flags):
    Unknown = 0
    LatticeNetlist = 1
    QuartusNetlist = 2
    XSTNetlist = 4
    CoreGenNetlist = 8
    All = 15

The above things could be tweaked a bit further because a flags class declared with the library automatically provides two "virtual" flags: NetlistKind.no_flags and NetlistKind.all_flags. These make the already declared NetlistKind.Unknown and NetlistKind.All redundant so we could leave them out from the declaration but the problem is that no_flags and all_flags don't match your naming convention. To aid this we declare a flags base class in your project instead of flags.Flags and you will have to use that in the rest of your project:

from flags import Flags

class BaseFlags(Flags):
    __no_flags_name__ = 'Unknown'
    __all_flags_name__ = 'All'

Based on the previously declared base class that can be subclassed by any of your flags in your project we could change your flag declaration to:

class NetlistKind(BaseFlags):
    LatticeNetlist = 1
    QuartusNetlist = 2
    XSTNetlist = 4
    CoreGenNetlist = 8

This way NetlistKind.Unknown is automatically declared with a value of zero. NetlistKind.All is also there and it is automatically the combination of all of your declared flags. It is possible to iterate enum members with/without these virtual flags. You can also declare aliases (flags that have the same value as another previously declared flag).

As an alternative declaration using the "function-call style" (also provided by the standard enum module):

NetlistKind = BaseFlags('NetlistKind', ['LatticeNetlist', 'QuartusNetlist',
                                        'XSTNetlist', 'CoreGenNetlist'])

If a flags class declares some members then it is considered to be final. Trying to subclass it will result in error. It is semantically undesired to allow subclassing a flag class for the purpose of adding new members or change functionality.

Besides this the flags class provides the operators your listed (bool operators, in, iteration, etc...) in a type-safe way. I'm going to finish the README.rst along with a little plumbing on the package interface in the next few days but the basic functionality is already there and tested with quite good coverage.

9
  • Argh, I should have started the bounty before I accepted your answer. Now the link is gone :(. So virtually +100 for the overall work and the hint on a user defined base class to override the *_flags names. It works like a charm.
    – Paebbels
    Apr 25, 2016 at 19:24
  • @Paebbels Thanks for being generous but I'm usually not here for the points. I visit stackoverflow when I'm too tired to do other things. :-D I'm working on the lib and it's README.rst file. A more detailed doc with better interface description and a bit of design philosophy will be available soon in the next few days. The current documentation is super basic as you've seen but even in case of larger docs I usually start with a similar TL;DR section... Poll the pypi or github page of the lib if you are interested in the upcoming longer version. Apr 25, 2016 at 19:50
  • 1
    @Paebbels For applications people usually just use a requirements.txt with 1 dependency per line and pip install -r requirements.txt. Each line of requirements.txt contains a parameter list for pip install e.g.: whatever_lib, mock>=1.2.0 or -r inherited_base_requirements.txt. Apr 25, 2016 at 20:34
  • 1
    I think I found a little issue: None is a keyword and no identifier in Python, thus is cannot be used as a enum member. NetlistKind.None reports a syntax error. I already have a requirements.lst for Travis-CI in a deep sub-directory ... I'll move it into the root dir. Thanks.
    – Paebbels
    Apr 25, 2016 at 20:39
  • 1
    @Paebbels Add your dependencies with a pinned version to requirements.txt e.g.: flags==1.0.1 and not simply flags. This way if a library gets updated (or in worst case several libraries) in the pypi then your app won't be affected. It is better to manually upgrade and increase the version numbers when you explicitly want it. Apr 26, 2016 at 0:15
3

I feel like having an example of a native Python flag and bitwise operations would be nice here.

from enum import Flag, auto


class FoodCategory(Flag):
    FISH = auto()
    SHELLFISH = auto()
    TREE_NUTS = auto()
    PEANUTS = auto()
    WHEAT = auto()
    RICE = auto()
    MILK = auto()
    BUTTER = auto()
    ALLERGENS = TREE_NUTS | PEANUTS | WHEAT | MILK


if __name__ == "__main__":
    DAIRY = FoodCategory.MILK | FoodCategory.BUTTER
    print(f"{FoodCategory.MILK in DAIRY = }")
    print(f"{FoodCategory.SHELLFISH in DAIRY = }")
    print(f"{FoodCategory.WHEAT in FoodCategory.ALLERGENS = }")
    print(f"{FoodCategory.SHELLFISH in FoodCategory.ALLERGENS = }")

Output:

FoodCategory.MILK in DAIRY = True
FoodCategory.SHELLFISH in DAIRY = False
FoodCategory.WHEAT in FoodCategory.ALLERGENS = True
FoodCategory.SHELLFISH in FoodCategory.ALLERGENS = False
1
  • This is nice, I wasn't aware that Flag implements __contains__ in this way. Feels more idiomatic than using & to check if some flags are set. Do you happen to know if there's also an alternative way of unsetting individual flags? (Alternative to flags &= ~FoodCategory.MILK to disable milk, for example)
    – Seb
    Jan 4 at 14:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.