114

According to the documentation of the keyword module, two new members have been added in Python 3.9:

  • issoftkeyword
  • softkwlist

However their documentation doesn't reveal anything about their purpose. This change is not even mentioned in the What's New article, where typically all API changes are documented. Digging a little further into the source code eventually leads to this pull request where it is mentioned that "this is essentially an internal tool" and that "soft keywords are still unused". So what's the purpose of Python's soft keywords?

1
  • Soft keywords are mentioned in PEP 634
    – khelwood
    Jan 19, 2021 at 22:22

3 Answers 3

76

Short: Soft keywords can still be used as variable or argument names.

PEP 622 sheds some light (Emphasis mine):

The difference between hard and soft keywords is that hard keywords are always reserved words, even in positions where they make no sense (e.g. x = class + 1), while soft keywords only get a special meaning in context.

[...] The match and case keywords are proposed to be soft keywords, so that they are recognized as keywords at the beginning of a match statement or case block respectively, but are allowed to be used in other places as variable or argument names.

10
  • 4
    Or even PEP 634 which superseded 622
    – khelwood
    Jan 19, 2021 at 22:23
  • 6
    PEP 634 contains the same example (match and case) but does not provide a general explanation of what a soft keyword is. PEP 622 does.
    – couka
    Jan 19, 2021 at 22:34
  • 2
    In Python 2, True and False weren't keywords at all: they were just identifiers in the built-in scope. Neither as nor None have even been valid identifier names. (At least, I think as has been a keyword from the beginning, as part of the import statement. If it was ever a valid identifier, it was sometime prior to Python 1.5.)
    – chepner
    Jan 20, 2021 at 16:08
  • 5
    @chepner See this issue from 2003, which was actually by myself — assigning to None was definitely a SyntaxWarning in 2003, and assigning to as was too up to and including Python 2.5.
    – gerrit
    Jan 20, 2021 at 16:19
  • 3
    also called contextual keywords in C++ and C#, and Context-sensitive keywords in C++/CLI
    – phuclv
    Jan 21, 2021 at 2:10
39

I think this is best explained with a demo. async and await were soft keywords in Python 3.5 and 3.6, and thus they could be used as identifiers:

>>> async = "spam"
>>> async def foo():
...     pass
...
>>> await = "bar"
>>> async, await
('spam', 'bar')

But in Python 3.7 they became proper keywords and can only be used in specific contexts where they make sense:

>>> async = "123"
  File "<stdin>", line 1
    async = "123"
          ^
SyntaxError: invalid syntax
>>> async def foo():
...     pass
...
>>> await = "bar"
  File "<stdin>", line 1
    await = "bar"
          ^
SyntaxError: invalid syntax
>>> async, await
  File "<stdin>", line 1
    async, await
         ^
SyntaxError: invalid syntax

The idea in first introducing them as soft keywords was mainly to not break any existing code that uses them as identifiers. The same reasoning is with the upcoming match keyword which would completely break for example re.match and millions of projects.

2
  • 1
    Indeed I forgot about async and await being soft keywords back then (though they were not included in the keyword module). I got the impression that soft keywords are made possible by the new PEG parser, so do you know how they realized async and await back in Python 3.6?
    – a_guest
    Jan 21, 2021 at 14:00
  • 2
    They were implemented with a tokenizer (lexer) hack. The tokenizer does the lookahead and maintains context about if you're inside a function, instead of the parser, and returns a special token for async and await. That solution didn't work for async comprehensions defined outside of async functions (invalid in 3.6, but valid in 3.7), and wasn't a scalable solution for future soft keywords.
    – bgw
    Mar 2, 2021 at 5:01
23

Soft keywords are keywords that are context sensitive. For example, it'd allow you to use class as a variable name as long as it can't be interpreted as defining a class. It'd allow use to replace cls with class for example.

Today that's not possible, since class is a keyword:

>>> def a(class):
  File "<stdin>", line 1
    def a(class):
          ^

Given the context it's clear that the user didn't intend to define a new class, but wanted an identifier named class.

2
  • 10
    More importantly, it works in the other direction, too: it let's you add a keyword to the language without invalidating its use as a variable name in existing code. One reason why the assignment operator := exists is because no suitable re-use of an existing keyword (as) could be agreed on. My personal preferance would have been let x = 3 in x * 2 as opposed to (x:=3) + 2, but an extremely high bar is set for adding keywords because of the risk of breaking existing code.
    – chepner
    Jan 20, 2021 at 14:50
  • (To be clear, I'm not sure such a let expression was explicitly considered, but the new-keyword aspect would have been at least as big an issue as adding yet another meaning to the in keyword.)
    – chepner
    Jan 20, 2021 at 14:51

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.