r-strings by example (raw strings)
Just to provide a slightly more "example-oriented" pedagogy, with an eye out for the edge cases:
| Syntax |
Meaning |
Note |
'a\nb' |
a, \n, b |
"Regular" string with a newline |
r'a\nb' |
a, \, n, b |
r string: \ does not create magic characters anymore |
r'a\\b' |
a, \, \, b |
\ doesn't even escape itself, we get two \ |
r'a\'b' |
a, \, ', b |
The only thing that \ "escapes" in r-strings is ' itself. But the \ and ' still appear in the string. |
r"a\"b" |
a, \, ", b |
Double quotes is analogous. |
r'a'b' |
Syntax error |
Unbalanced single quotes, we'd need the \ like above |
r"a'b" |
a, ', b |
We can get a single quote without \ by using " instead |
r'''a'"b''' |
a, ', ", b |
We can get both ' and " in by using triple quotes |
'a\'\'\'"""b' |
a, ', ', ', ", ", ", b |
It is impossible to have both triple ' and triple " in a raw string without some backslash escaping: https://stackoverflow.com/questions/4630465/how-to-include-a-double-quote-and-or-single-quote-character-in-a-raw-python-stri| |
The easiest way to play with this yourself is to convert the input string literal to a list of characters with the list() function as mentioned at How do I split a string into a list of characters? e.g.:
>>> list(r'a\nb')
['a', '\\', 'n', 'b']
Application of r-strings: it removes the need to escape \, common in regexes
E.g. if you want to match ISO dates yyyy-mm-dd, without r, the cleanest way would be to write:
re.compile('\\d{4}-\\d{2}-\\d{2}')
because the \ has to be present in the final string seen by regexp. It would also actually work in certain Python versions if you did just:
re.compile('\d{4}-\d{2}-\d{2}')
because \d is not a valid escape sequence and gets interpreted as \ + d. But that is confusing, as it is hard to remember what is a valid escape or not (\a, \b, \f, \n, \r, \t, \v
are valid, what in the name is a "Vertical Tab"???), and Python 3.12 already gives a warning if you do that:
<stdin>:1: SyntaxWarning: invalid escape sequence '\d'
see also: How to fix "SyntaxWarning: invalid escape sequence" in Python?
So with r-string we can write the simpler:
re.compile(r'\d{4}-\d{2}-\d{2}')
which is much more readable and sane.
The downside of r strings is that you then can't have magic characters like newline in your string. But these are not very common in regular expressions.
u strings are the default in Python 3 (Unicode strings)
In Python 3, 'abc' is the same as u'abc', and the u syntax exists just to help with code backward compatibility and is never needed.
And to get a Python 2 'abc' (byte string), you have to do b'abc' in Python 3.
See also: What's the u prefix in a Python string?
| Syntax |
Python version |
Meaning |
'abc' |
2 |
Byte string |
u'abc' |
2 |
Unicode string |
b'abc' |
2 |
Didn't exist |
'abc' |
3 |
Unicode string |
u'abc' |
3 |
Unicode string |
b'abc' |
3 |
Byte string |
Unicode string vs byte string
The byte string can only have "ASCII characters" (more precisely, single byte values 0-255), while the Unicode string can have any Unicode character.
For example, in Python 3, if we play around with é, an 'e' with an acute accent present e.g. in French and Portuguese, and encoded as two bytes in UTF-8 0xC3 + 0xA9 we get:
>>> list('aéi')
['a', 'é', 'i']
>>> list(b'aéi')
File "<stdin>", line 1
list(b'aéi')
^^^^^^
SyntaxError: bytes can only contain ASCII literal characters
>>> list(map(lambda x: hex(x), list(bytes('aéi', 'utf8'))))
['0x61', '0xc3', '0xa9', '0x69']
so we see that:
'aéi' contains three Unicode characters. Doing e.g. 'aéi'[1] gives é as intuitively expected
bytes('aéi', 'utf8') contains four bytes, because the é is made up of two bytes