36

I need to convert an arbitrary string to a string that is a valid variable name in Python.

Here's a very basic example:

s1 = 'name/with/slashes'
s2 = 'name '

def clean(s):
    s = s.replace('/', '')
    s = s.strip()

    return s

# the _ is there so I can see the end of the string
print clean(s1) + '_'

That is a very naive approach. I need to check if the string contains invalid variable name characters and replace them with ''

What would be a pythonic way to do this?

5
  • 1
    What problem are you trying to solve using this method? There may be a better way.
    – Daenyth
    Jul 21, 2010 at 20:06
  • 2
    print repr("a string ") shows the string in quotes - neater than appending _ to it. Jul 21, 2010 at 20:38
  • 2
    I'm traversing a scene graph from cinema 4d and need to re-create it in blender. To keep things easy for me to understand I want to use the actual names of the cinema4d objects as variable names for Blender Python, so I need to adjust those first Jul 21, 2010 at 20:40
  • 1
    BTW, the general conversion is called "slugifying" and searching stacoverflow or the internet for "python slugify" will find many solutions.
    – samwyse
    Jul 25 at 15:30
  • Thanks @samwyse ! if I only new this 13 years ago :))) Never too late to learn something new though. Jul 25 at 21:03

4 Answers 4

72

Well, I'd like to best Triptych's solution with ... a one-liner!

>>> def clean(varStr): return re.sub('\W|^(?=\d)','_', varStr)
...

>>> clean('32v2 g #Gmw845h$W b53wi ')
'_32v2_g__Gmw845h_W_b53wi_'

This substitution replaces any non-variable appropriate character with underscore and inserts underscore in front if the string starts with a digit. IMO, 'name/with/slashes' looks better as variable name name_with_slashes than as namewithslashes.

5
  • 10
    If you want to collapse multiple invalid chars into a single _, use the following: re.sub(r'\W+|^(?=\d)','_', varStr) (add a + sign after the non-word-char \W)
    – grandchild
    Jan 29, 2020 at 11:52
  • Please do not use lambda to define a "named" function. It brings multiple problems. lambda is intended for other uses. If you insist on writing the code in a single line - in this case you can do it easily: def clean(varStr): return re.sub('\W|^(?=\d)','_', varStr) --- Otherwise I agree that it is usually better to replace the invalid characters than to remove them. Apr 26, 2021 at 15:26
  • 1
    @pabouk i suppose i meant to show it's a single expression - but ok sure, def it is
    – Nas Banov
    May 1, 2021 at 1:29
  • @grandchild no, i wouldn't want to collapse N var-name-invalid chars into 1 _ - i wanted that to be N _, as a clue how many were "erased"
    – Nas Banov
    Jul 31, 2022 at 20:44
  • @NasBanov oh, sure, the "you" in my answer wasn't meant to mean you specifically, but rather "anyone". I wanted this when I found this and I used your solution with my change and wanted to share it.
    – grandchild
    Aug 3, 2022 at 11:15
39

According to Python, an identifier is a letter or underscore, followed by an unlimited string of letters, numbers, and underscores:

import re

def clean(s):

   # Remove invalid characters
   s = re.sub('[^0-9a-zA-Z_]', '', s)

   # Remove leading characters until we find a letter or underscore
   s = re.sub('^[^a-zA-Z_]+', '', s)

   return s

Use like this:

>>> clean(' 32v2 g #Gmw845h$W b53wi ')
'v2gGmw845hWb53wi'
4
  • 2
    +1 for recognizing that valid characters are different for the first letter vs subsequent letters as well as only allowing valid letters vs stripping invalid letters. Jul 21, 2010 at 20:40
  • 1
    @JukkaSuomela has a great point. Since all the reserved keywords are only letters, you can guarantee there won't be a conflict by adding an underscore (if you want to be real good, you can first check if the name is a keyword and only adding an underscore if that's the case). Jul 21, 2010 at 20:54
  • 13
    You can use the "keyword" module to make sure the name does not conflict with any Python keywords, "keyword.iskeyword(s)"
    – flashk
    Jul 22, 2010 at 4:20
  • You can replace the first substitution string with r'\W'.
    – Blizz
    Jun 1 at 10:30
5

You can use the built in func:str.isidentifier() in combination with filter(). This requires no imports such as re and works by iterating over each character and returning it if its an identifier. Then you just do a ''.join to convert the array to a string again.

s1 = 'name/with/slashes'
s2 = 'name '

def clean(s):
    s = ''.join(filter(str.isidentifier, s))
    return s

print f'{clean(s1)}_' #the _ is there so I can see the end of the string

EDIT:

If, like Hans Bouwmeester in the replies, want numeric values to be included as well, you can create a lambda which uses both the isIdentifier and the isdecimal functions to check the characters. Obviously this can be expanded as far as you want to take it. Code:

s1 = 'name/with/slashes'
s2 = 'name i2, i3    '
s3 = 'epng2 0-2g [ q4o 2-=2 t1  l32!@#$%*(vqv[r 0-34 2]] '

def clean(s):
    s = ''.join(filter( 
        lambda c: str.isidentifier(c) or str.isdecimal(c), s))
    return s
#the _ is there so I can see the end of the string
print(f'{ clean(s1) }_')
print(f'{ clean(s2) }_')
print(f'{ clean(s3) }_')

Gives :

namewithslashes_
namei2i3_
epng202gq4o22t1l32vqvr0342_
6
  • Doesn't work. ''.join(filter(str.isidentifier, 'i1')) yields i where i1 itself is a valid identifier. Losing the '1' unacceptable if i2, i3, ... are present in the set also. Mar 4, 2022 at 15:19
  • 1
    @HansBouwmeester That depends on if one actually wants numbers in the name, I personally avoid it at all costs because there are few cases where numbers actually make it clearer (one good example of using numbers in method names is Mathf.Clamp01) usually its because someone is lazily enumerating functions which should actually get better names. With that said, I added in a line of code which also allows numbers in my answer if someone truly wants to do so.
    – ZXYNINE
    Apr 14, 2022 at 1:26
  • 1
    Fair enough but the question was to convert an arbitrary string to a valid variable name (i.e. identifier). Some applications will have no or little control over the incoming string contents. (I've +1 'ed the response anyway as I was not aware of the str.isidentifier method in Python). Apr 14, 2022 at 16:15
  • In general, I think it's best if the implementation leaves any string that's already a valid identifier to begin with untouched. Apr 14, 2022 at 16:23
  • @HansBouwmeester To quote you, "Fair enough". If this is the case, then an additional check using >>> from keyword import iskeyword Would be good to have to ensure the variable name does not clash with python keywords, if it does after the fixing you could add an underscore or something.
    – ZXYNINE
    Apr 14, 2022 at 17:00
3

You should build a regex that's a whitelist of permissible characters and replace everything that is not in that character class.

2
  • 8
    Or better yet, build a whitelist of permissible strings and link them to actual pre-defined objects. Every time questions like this come up, I get nervous and think someone is almost certainly about to write a massive security hole. Jul 21, 2010 at 20:04
  • :) not me...I'm just parsing a scene graph from cinema 4d and need to re create it in blender. Animators usually don't worry or need to worry about spaces/slashes/etc in object names Jul 21, 2010 at 20:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.