849

It seems like there should be a simpler way than:

import string
s = "string. With. Punctuation?" # Sample string 
out = s.translate(string.maketrans("",""), string.punctuation)

Is there?

13
  • 4
    Seems pretty straightforward to me. Why do you want to change it? If you want it easier just wrap what you just wrote in a function. Commented Nov 5, 2008 at 17:38
  • 3
    Well, it just seemed kind of hackish to be be using kind of a side effect of str.translate to be doing the work. I was thinking there might be something more like str.strip(chars) that worked on the entire string instead of just the boundaries that I had missed.
    – Redwood
    Commented Nov 5, 2008 at 18:00
  • 2
    Depends on the data too. Using this on data where there are server names with underscores as part of the name (pretty common some places) could be bad. Just be sure that you know the data and what it conatains or you could end up with a subset of the clbuttic problem.
    – EBGreen
    Commented Nov 5, 2008 at 18:10
  • 66
    Depends also on what you call punctuation. "The temperature in the O'Reilly & Arbuthnot-Smythe server's main rack is 40.5 degrees." contains exactly ONE punctuation character, the second "." Commented Mar 8, 2010 at 21:49
  • 46
    I'm surprised no one mentioned that string.punctuation doesn't include non-English punctuation at all. I'm thinking about 。,!?:ד”〟, and so on.
    – Clément
    Commented Jan 3, 2013 at 15:40

32 Answers 32

1278

From an efficiency perspective, you're not going to beat

s.translate(None, string.punctuation)

For higher versions of Python use the following code:

s.translate(str.maketrans('', '', string.punctuation))

It's performing raw string operations in C with a lookup table - there's not much that will beat that but writing your own C code.

If speed isn't a worry, another option though is:

exclude = set(string.punctuation)
s = ''.join(ch for ch in s if ch not in exclude)

This is faster than s.replace with each char, but won't perform as well as non-pure python approaches such as regexes or string.translate, as you can see from the below timings. For this type of problem, doing it at as low a level as possible pays off.

Timing code:

import re, string, timeit

s = "string. With. Punctuation"
exclude = set(string.punctuation)
table = string.maketrans("","")
regex = re.compile('[%s]' % re.escape(string.punctuation))

def test_set(s):
    return ''.join(ch for ch in s if ch not in exclude)

def test_re(s):  # From Vinko's solution, with fix.
    return regex.sub('', s)

def test_trans(s):
    return s.translate(table, string.punctuation)

def test_repl(s):  # From S.Lott's solution
    for c in string.punctuation:
        s=s.replace(c,"")
    return s

print "sets      :",timeit.Timer('f(s)', 'from __main__ import s,test_set as f').timeit(1000000)
print "regex     :",timeit.Timer('f(s)', 'from __main__ import s,test_re as f').timeit(1000000)
print "translate :",timeit.Timer('f(s)', 'from __main__ import s,test_trans as f').timeit(1000000)
print "replace   :",timeit.Timer('f(s)', 'from __main__ import s,test_repl as f').timeit(1000000)

This gives the following results:

sets      : 19.8566138744
regex     : 6.86155414581
translate : 2.12455511093
replace   : 28.4436721802
18
  • 35
    Thanks for the timing info, I was thinking about doing something like that myself, but yours is better written than anything I would have done and now I can use it as a template for any future timing code I want to write:).
    – Redwood
    Commented Nov 5, 2008 at 19:57
  • 30
    Great answer. You can simplify it by removing the table. The docs say: "set the table argument to None for translations that only delete characters" (docs.python.org/library/stdtypes.html#str.translate) Commented Jul 1, 2011 at 21:24
  • 3
    worth noting too that translate() behaves differently for str and unicode objects, so you need to be sure you're always working with the same datatype, but the approach in this answer works equally well for both, which is handy.
    – Richard J
    Commented Jan 16, 2015 at 9:35
  • 41
    In Python3, table = string.maketrans("","") should be replaced with table = str.maketrans({key: None for key in string.punctuation})? Commented May 13, 2016 at 23:36
  • 28
    To update the discussion, as of Python 3.6, regex is now the most efficient method! It is almost 2x faster than translate. Also, sets and replace are no longer so bad! They are both improved by over a factor of 4 :) Commented Jul 24, 2017 at 22:35
204

Regular expressions are simple enough, if you know them.

import re
s = "string. With. Punctuation?"
s = re.sub(r'[^\w\s]','',s)
4
  • 4
    @Outlier Explanation: replaces not (^) word characters or spaces with the empty string. Be careful though, the \w matches underscore too usually for example.
    – Matthias
    Commented Feb 3, 2016 at 15:28
  • 6
    @SIslam I think it will work with unicode with the unicode flag set, i.e. s = re.sub(r'[^\w\s]','',s, re.UNICODE). Testing it with python 3 on linux it works even without the flag using tamil letters, தமிழ்.
    – Matthias
    Commented Feb 3, 2016 at 15:31
  • 1
    @Matthias I tried the code with Python 3.6.5 on Mac, the Tamil letters output looks a bit different, input தமிழ் becomes தமழ. I have no knowledge about Tamil, not sure if that's expected.
    – shiouming
    Commented May 28, 2019 at 2:45
  • 1
    @Matthias It gets confused with word boundaries while working with UNICODE Bengali text and gives wrong words no matter UNICODE flag is used or not.
    – hafiz031
    Commented Sep 9, 2020 at 13:04
92

For the convenience of usage, I sum up the note of striping punctuation from a string in both Python 2 and Python 3. Please refer to other answers for the detailed description.


Python 2

import string

s = "string. With. Punctuation?"
table = string.maketrans("","")
new_s = s.translate(table, string.punctuation)      # Output: string without punctuation

Python 3

import string

s = "string. With. Punctuation?"
table = str.maketrans(dict.fromkeys(string.punctuation))  # OR {key: None for key in string.punctuation}
new_s = s.translate(table)                          # Output: string without punctuation
1
  • It's interesting that this solution (in particular the OR {key: None for...} option) allows to control what you want to insert in place of punctuation, which may be spaces (for this use key: " " instead of key: None).
    – Pablo
    Commented Oct 15, 2021 at 9:22
52
myString.translate(None, string.punctuation)
7
  • 4
    ah, I tried this but it doesn't work in all cases. myString.translate(string.maketrans("",""), string.punctuation) works fine.
    – Aidan Kane
    Commented Aug 12, 2010 at 12:30
  • 12
    Note that for str in Python 3, and unicode in Python 2, the deletechars argument is not supported.
    – agf
    Commented Apr 14, 2012 at 0:36
  • 4
    myString.translate(string.maketrans("",""), string.punctuation) will NOT work with unicode strings (found out the hard way) Commented Jul 25, 2014 at 19:25
  • 59
    TypeError: translate() takes exactly one argument (2 given) :( Commented Apr 23, 2015 at 18:35
  • 3
    @BrianTingle: look at the Python 3 code in my comment (it passes one argument). Follow the link, to see Python 2 code that works with unicode and its Python 3 adaptation
    – jfs
    Commented Jul 15, 2015 at 17:21
35

string.punctuation is ASCII only! A more correct (but also much slower) way is to use the unicodedata module:

# -*- coding: utf-8 -*-
from unicodedata import category
s = u'String — with -  «punctation »...'
s = ''.join(ch for ch in s if category(ch)[0] != 'P')
print 'stripped', s

You can generalize and strip other types of characters as well:

''.join(ch for ch in s if category(ch)[0] not in 'SP')

It will also strip characters like ~*+§$ which may or may not be "punctuation" depending on one's point of view.

2
  • 4
    You could: regex.sub(ur"\p{P}+", "", text)
    – jfs
    Commented Aug 11, 2012 at 12:07
  • 1
    Unfortunately, things like ~ are not part of the punctuation category. You need to also test for the Symbols category as well. Commented Oct 6, 2019 at 4:08
34

Not necessarily simpler, but a different way, if you are more familiar with the re family.

import re, string
s = "string. With. Punctuation?" # Sample string 
out = re.sub('[%s]' % re.escape(string.punctuation), '', s)
3
  • 1
    Works because string.punctuation has the sequence ,-. in proper, ascending, no-gaps, ASCII order. While Python has this right, when you try to use a subset of string.punctuation, it can be a show-stopper because of the surprise "-".
    – S.Lott
    Commented Nov 5, 2008 at 17:49
  • 2
    Actually, its still wrong. The sequence "\]" gets treated as an escape (coincidentally not closing the ] so bypassing another failure), but leaves \ unescaped. You should use re.escape(string.punctuation) to prevent this.
    – Brian
    Commented Nov 5, 2008 at 18:15
  • 1
    Yes, I omitted it because it worked for the example to keep things simple, but you are right that it should be incorporated. Commented Nov 5, 2008 at 23:21
32

I usually use something like this:

>>> s = "string. With. Punctuation?" # Sample string
>>> import string
>>> for c in string.punctuation:
...     s= s.replace(c,"")
...
>>> s
'string With Punctuation'
2
  • 2
    An uglified one-liner: reduce(lambda s,c: s.replace(c, ''), string.punctuation, s).
    – jfs
    Commented Aug 11, 2012 at 12:03
  • 1
    great, however doesn't remove some puctuation like longer hyphen Commented Jan 17, 2015 at 15:57
16

For Python 3 str or Python 2 unicode values, str.translate() only takes a dictionary; codepoints (integers) are looked up in that mapping and anything mapped to None is removed.

To remove (some?) punctuation then, use:

import string

remove_punct_map = dict.fromkeys(map(ord, string.punctuation))
s.translate(remove_punct_map)

The dict.fromkeys() class method makes it trivial to create the mapping, setting all values to None based on the sequence of keys.

To remove all punctuation, not just ASCII punctuation, your table needs to be a little bigger; see J.F. Sebastian's answer (Python 3 version):

import unicodedata
import sys

remove_punct_map = dict.fromkeys(i for i in range(sys.maxunicode)
                                 if unicodedata.category(chr(i)).startswith('P'))
4
  • To support Unicode, string.punctuation is not enough. See my answer
    – jfs
    Commented Sep 30, 2014 at 20:57
  • @J.F.Sebastian: indeed, my answer was just using the same characters as the top-voted one. Added a Python 3 version of your table. Commented Sep 30, 2014 at 21:08
  • the top-voted answer works only for ascii strings. Your answer claims explicitly the Unicode support.
    – jfs
    Commented Oct 1, 2014 at 8:53
  • 1
    @J.F.Sebastian: it works for Unicode strings. It strips ASCII punctuation. I never claimed it strips all punctuation. :-) The point was to provide the correct technique for unicode objects vs. Python 2 str objects. Commented Oct 1, 2014 at 9:02
15

string.punctuation misses loads of punctuation marks that are commonly used in the real world. How about a solution that works for non-ASCII punctuation?

import regex
s = u"string. With. Some・Really Weird、Non?ASCII。 「(Punctuation)」?"
remove = regex.compile(ur'[\p{C}|\p{M}|\p{P}|\p{S}|\p{Z}]+', regex.UNICODE)
remove.sub(u" ", s).strip()

Personally, I believe this is the best way to remove punctuation from a string in Python because:

  • It removes all Unicode punctuation
  • It's easily modifiable, e.g. you can remove the \{S} if you want to remove punctuation, but keep symbols like $.
  • You can get really specific about what you want to keep and what you want to remove, for example \{Pd} will only remove dashes.
  • This regex also normalizes whitespace. It maps tabs, carriage returns, and other oddities to nice, single spaces.

This uses Unicode character properties, which you can read more about on Wikipedia.

2
  • 1
    This line actually does not work: remove = regex.compile(ur'[\p{C}|\p{M}|\p{P}|\p{S}|\p{Z}]+', regex.UNICODE)
    – John Stud
    Commented Jun 29, 2020 at 0:09
  • @JohnStud Breaks in later versions of Python 3 because now all strings default to support unicode already. Can remove the 'u' from lines 2, 3, and 4 and it works. Commented Dec 15, 2021 at 22:57
13

I haven't seen this answer yet. Just use a regex; it removes all characters besides word characters (\w) and number characters (\d), followed by a whitespace character (\s):

import re
s = "string. With. Punctuation?" # Sample string 
out = re.sub(ur'[^\w\d\s]+', '', s)
3
  • 3
    \d is redundant since it is a subset of \w.
    – blhsing
    Commented Jan 10, 2019 at 18:37
  • Number characters are considered a subset of Word characters? I thought a Word character was any character that could construct a real word, e.g. a-zA-Z?
    – Blairg23
    Commented Jan 10, 2019 at 20:55
  • Yes, a "word" in regex includes alphabets, numbers and underscore. Please see the description for \w in the documentation: docs.python.org/3/library/re.html
    – blhsing
    Commented Jan 10, 2019 at 21:10
10

This might not be the best solution however this is how I did it.

import string
f = lambda x: ''.join([i for i in x if i not in string.punctuation])
1
  • That's a great solution but I'm not sure that string.punctuation covers all possible Unicode punctuation marks.
    – ingyhere
    Commented Jul 28, 2023 at 17:58
10

Here's a one-liner for Python 3.5:

import string
"l*ots! o(f. p@u)n[c}t]u[a'ti\"on#$^?/".translate(str.maketrans({a:None for a in string.punctuation}))
7
import re
s = "string. With. Punctuation?" # Sample string 
out = re.sub(r'[^a-zA-Z0-9\s]', '', s)
1
  • Seems like that would only work for ASCII characters.
    – avirr
    Commented Oct 14, 2019 at 19:15
7

Here is a function I wrote. It's not very efficient, but it is simple and you can add or remove any punctuation that you desire:

def stripPunc(wordList):
    """Strips punctuation from list of words"""
    puncList = [".",";",":","!","?","/","\\",",","#","@","$","&",")","(","\""]
    for punc in puncList:
        for word in wordList:
            wordList=[word.replace(punc,'') for word in wordList]
    return wordList
1
  • commas between punctuation not necessary. punctlist can just be a string
    – Kim
    Commented Apr 27, 2021 at 12:29
7

A one-liner might be helpful in not very strict cases:

''.join([c for c in s if c.isalnum() or c.isspace()])
6
>>> s = "string. With. Punctuation?"
>>> s = re.sub(r'[^\w\s]','',s)
>>> re.split(r'\s*', s)


['string', 'With', 'Punctuation']
1
  • 2
    Please edit with more information. Code-only and "try this" answers are discouraged, because they contain no searchable content, and don't explain why someone should "try this".
    – Paritosh
    Commented Aug 24, 2016 at 7:29
6

Just as an update, I rewrote the @Brian example in Python 3 and made changes to it to move regex compile step inside of the function. My thought here was to time every single step needed to make the function work. Perhaps you are using distributed computing and can't have regex object shared between your workers and need to have re.compile step at each worker. Also, I was curious to time two different implementations of maketrans for Python 3

table = str.maketrans({key: None for key in string.punctuation})

vs

table = str.maketrans('', '', string.punctuation)

Plus I added another method to use set, where I take advantage of intersection function to reduce number of iterations.

This is the complete code:

import re, string, timeit

s = "string. With. Punctuation"


def test_set(s):
    exclude = set(string.punctuation)
    return ''.join(ch for ch in s if ch not in exclude)


def test_set2(s):
    _punctuation = set(string.punctuation)
    for punct in set(s).intersection(_punctuation):
        s = s.replace(punct, ' ')
    return ' '.join(s.split())


def test_re(s):  # From Vinko's solution, with fix.
    regex = re.compile('[%s]' % re.escape(string.punctuation))
    return regex.sub('', s)


def test_trans(s):
    table = str.maketrans({key: None for key in string.punctuation})
    return s.translate(table)


def test_trans2(s):
    table = str.maketrans('', '', string.punctuation)
    return(s.translate(table))


def test_repl(s):  # From S.Lott's solution
    for c in string.punctuation:
        s=s.replace(c,"")
    return s


print("sets      :",timeit.Timer('f(s)', 'from __main__ import s,test_set as f').timeit(1000000))
print("sets2      :",timeit.Timer('f(s)', 'from __main__ import s,test_set2 as f').timeit(1000000))
print("regex     :",timeit.Timer('f(s)', 'from __main__ import s,test_re as f').timeit(1000000))
print("translate :",timeit.Timer('f(s)', 'from __main__ import s,test_trans as f').timeit(1000000))
print("translate2 :",timeit.Timer('f(s)', 'from __main__ import s,test_trans2 as f').timeit(1000000))
print("replace   :",timeit.Timer('f(s)', 'from __main__ import s,test_repl as f').timeit(1000000))

This is my results:

sets      : 3.1830138750374317
sets2      : 2.189873124472797
regex     : 7.142953420989215
translate : 4.243278483860195
translate2 : 2.427158243022859
replace   : 4.579746678471565
6

I was looking for a really simple solution. here's what I got:

import re 

s = "string. With. Punctuation?" 
s = re.sub(r'[\W\s]', ' ', s)

print(s)
'string  With  Punctuation '
5

Here's a solution without regex.

import string

input_text = "!where??and!!or$$then:)"
punctuation_replacer = string.maketrans(string.punctuation, ' '*len(string.punctuation))    
print ' '.join(input_text.translate(punctuation_replacer).split()).strip()

Output>> where and or then
  • Replaces the punctuations with spaces
  • Replace multiple spaces in between words with a single space
  • Remove the trailing spaces, if any with strip()
5

Why none of you use this?

 ''.join(filter(str.isalnum, s)) 

Too slow?

2
  • 4
    Note that this will also remove spaces.
    – Georgy
    Commented Jul 29, 2019 at 12:36
  • Still useful if you do a str.split() first.
    – Spartan
    Commented Apr 28, 2022 at 7:05
4

Here's one other easy way to do it using RegEx

import re

punct = re.compile(r'(\w+)')

sentence = 'This ! is : a # sample $ sentence.' # Text with punctuation
tokenized = [m.group() for m in punct.finditer(sentence)]
sentence = ' '.join(tokenized)
print(sentence) 
'This is a sample sentence'

4
# FIRST METHOD
# Storing all punctuations in a variable    
punctuation='!?,.:;"\')(_-'
newstring ='' # Creating empty string
word = raw_input("Enter string: ")
for i in word:
     if(i not in punctuation):
                  newstring += i
print ("The string without punctuation is", newstring)

# SECOND METHOD
word = raw_input("Enter string: ")
punctuation = '!?,.:;"\')(_-'
newstring = word.translate(None, punctuation)
print ("The string without punctuation is",newstring)


# Output for both methods
Enter string: hello! welcome -to_python(programming.language)??,
The string without punctuation is: hello welcome topythonprogramminglanguage
3
with open('one.txt','r')as myFile:

    str1=myFile.read()

    print(str1)


    punctuation = ['(', ')', '?', ':', ';', ',', '.', '!', '/', '"', "'"] 

for i in punctuation:

        str1 = str1.replace(i," ") 
        myList=[]
        myList.extend(str1.split(" "))
print (str1) 
for i in myList:

    print(i,end='\n')
    print ("____________")
3

Try that one :)

regex.sub(r'\p{P}','', s)
2

The question does not have a lot of specifics, so the approach I took is to come up with a solution with the simplest interpretation of the problem: just remove the punctuation.

Note that solutions presented don't account for contracted words (e.g., you're) or hyphenated words (e.g., anal-retentive)...which is debated as to whether they should or shouldn't be treated as punctuations...nor to account for non-English character set or anything like that...because those specifics were not mentioned in the question. Someone argued that space is punctuation, which is technically correct...but to me it makes zero sense in the context of the question at hand.

# using lambda
''.join(filter(lambda c: c not in string.punctuation, s))

# using list comprehension
''.join('' if c in string.punctuation else c for c in s)
1

Apparently I can't supply edits to the selected answer, so here's an update which works for Python 3. The translate approach is still the most efficient option when doing non-trivial transformations.

Credit for the original heavy lifting to @Brian above. And thanks to @ddejohn for his excellent suggestion for improvement to the original test.

#!/usr/bin/env python3

"""Determination of most efficient way to remove punctuation in Python 3.

Results in Python 3.8.10 on my system using the default arguments:

set       : 51.897
regex     : 17.901
translate :  2.059
replace   : 13.209
"""

import argparse
import re
import string
import timeit

parser = argparse.ArgumentParser()
parser.add_argument("--filename", "-f", default=argparse.__file__)
parser.add_argument("--iterations", "-i", type=int, default=10000)
opts = parser.parse_args()
with open(opts.filename) as fp:
    s = fp.read()
exclude = set(string.punctuation)
table = str.maketrans("", "", string.punctuation)
regex = re.compile(f"[{re.escape(string.punctuation)}]")

def test_set(s):
    return "".join(ch for ch in s if ch not in exclude)

def test_regex(s):  # From Vinko's solution, with fix.
    return regex.sub("", s)

def test_translate(s):
    return s.translate(table)

def test_replace(s):  # From S.Lott's solution
    for c in string.punctuation:
        s = s.replace(c, "")
    return s

opts = dict(globals=globals(), number=opts.iterations)
solutions = "set", "regex", "translate", "replace"
for solution in solutions:
    elapsed = timeit.timeit(f"test_{solution}(s)", **opts)
    print(f"{solution:<10}: {elapsed:6.3f}")
2
  • 1
    These tests are really not all that meaningful. For the sake of completeness, add tests for larger input strings, say like a few KBs large text file. The str.translate pulls ahead pretty quickly as s grows.
    – ddejohn
    Commented Oct 17, 2021 at 19:38
  • 1
    @ddejohn Excellent point. My original purpose was to respond to the users who were frustrated that they couldn't figure out how to get @Brian's original script to run under Python 3.x, but I can see that your suggestion improves the value of his original test significantly. And in fact, even replace beats out regex when the test is scaled up this way, and by a healthy margin.
    – Bob Kline
    Commented Oct 18, 2021 at 13:57
1

When you deal with the Unicode strings, I suggest using PyPi regex module because it supports both Unicode property classes (like \p{X} / \P{X}) and POSIX character classes (like [:name:]).

Just install the package by typing pip install regex (or pip3 install regex) in your terminal and hit ENTER.

In case you need to remove punctuation and symbols of any kind (that is, anything other than letters, digits and whitespace) you can use

regex.sub(r'[\p{P}\p{S}]', '', text)  # to remove one by one
regex.sub(r'[\p{P}\p{S}]+', '', text) # to remove all consecutive punctuation/symbols with one go
regex.sub(r'[[:punct:]]+', '', text)  # Same with a POSIX character class

See a Python demo online:

import regex

text = 'भारत India <><>^$.,,! 002'
new_text = regex.sub(r'[\p{P}\p{S}\s]+', ' ', text).lower().strip()
# OR
# new_text = regex.sub(r'[[:punct:]\s]+', ' ', text).lower().strip()

print(new_text)
# => भारत india 002

Here, I added a whitespace \s pattern to the character class

1

For serious natural language processing (NLP), you should let a library like SpaCy handle punctuation through tokenization, which you can then manually tweak to your needs.

For example, how do you want to handle hyphens in words? Exceptional cases like abbreviations? Begin and end quotes? URLs? IN NLP it's often useful to separate out a contraction like "let's" into "let" and "'s" for further processing.

SpaCy example tokenization

0

Considering unicode. Code checked in python3.

from unicodedata import category
text = 'hi, how are you?'
text_without_punc = ''.join(ch for ch in text if not category(ch).startswith('P'))
0

You can also do this:

import string
' '.join(word.strip(string.punctuation) for word in 'text'.split())

Not the answer you're looking for? Browse other questions tagged or ask your own question.