Remove specific characters from a string in Python

I'm trying to remove specific characters from a string using Python. This is the code I'm using right now. Unfortunately it appears to do nothing to the string.

for char in line: if char in " ?.!/;:": line.replace(char,'')

How do I do this properly?

It's been over 5 years, but how about using the filter function and a Lambda Expression: filter(lambda ch: ch not in " ?.!/;:", line). Pretty concise and efficient too, I think. Of course, it returns a new string that you'll have to assign a name to.
– John Red
Feb 6 '16 at 10:35

filter

filter(lambda ch: ch not in " ?.!/;:", line)

@JohnRed: Actually it returns an iterator that returns a list of characters but if you'd put this in an answer a few of us would be pleased to up-vote it.
– Bill Bell
Sep 4 '17 at 20:43

Easy peasy with re.sub in Python 3.5
– Serge Stroobandt
Sep 25 '17 at 21:25

re.sub

@BillBell: see stackoverflow.com/a/46930314/1587329 and stackoverflow.com/a/41413861/1587329
– serv-inc
Oct 25 '17 at 10:46

@BillBell: PS: it's an iterator in Python3 and a string, tuple, or list in Python2
– serv-inc
Oct 25 '17 at 10:55

26 Answers
26

Strings in Python are immutable (can't be changed). Because of this, the effect of line.replace(...) is just to create a new string, rather than changing the old one. You need to rebind (assign) it to line in order to have that variable take the new value, with those characters removed.

line.replace(...)

line

Also, the way you are doing it is going to be kind of slow, relatively. It's also likely to be a bit confusing to experienced pythonators, who will see a doubly-nested structure and think for a moment that something more complicated is going on.

Starting in Python 2.6 and newer Python 2.x versions *, you can instead use str.translate, (but read on for Python 3 differences):

str.translate

line = line.translate(None, '!@#$')

or regular expression replacement with re.sub

re.sub

import re line = re.sub('[!@#$]', '', line)

The characters enclosed in brackets constitute a character class. Any characters in line which are in that class are replaced with the second parameter to sub: an empty string.

line

sub

In Python 3, strings are Unicode. You'll have to translate a little differently. kevpie mentions this in a comment on one of the answers, and it's noted in the documentation for str.translate.

str.translate

When calling the translate method of a Unicode string, you cannot pass the second parameter that we used above. You also can't pass None as the first parameter, or even a translation table from string.maketrans. Instead, you pass a dictionary as the only parameter. This dictionary maps the ordinal values of characters (i.e. the result of calling ord on them) to the ordinal values of the characters which should replace them, or—usefully to us—None to indicate that they should be deleted.

translate

None

string.maketrans

ord

None

So to do the above dance with a Unicode string you would call something like

translation_table = dict.fromkeys(map(ord, '!@#$'), None) unicode_line = unicode_line.translate(translation_table)

Here dict.fromkeys and map are used to succinctly generate a dictionary containing

dict.fromkeys

map

ord('!'): None, ord('@'): None, ...

Even simpler, as another answer puts it, create the dictionary in place:

unicode_line = unicode_line.translate(ord(c): None for c in '!@#$')

* for compatibility with earlier Pythons, you can create a "null" translation table to pass in place of None:

None

import string line = line.translate(string.maketrans('', ''), '!@#$')

Here string.maketrans is used to create a translation table, which is just a string containing the characters with ordinal values 0 to 255.

string.maketrans

In Python3, line.translate takes only one argument and the first solution will not work
– marczoid
Nov 20 '12 at 9:19

line.translate

@marczoid: Thanks, added a note to that effect.
– intuited
Nov 20 '12 at 17:11

In python3, str.translate() does not take the 2nd argument. So, your answer will become line.translate(ord(i):None for i in '!@#$')
– naveen
Jan 12 '14 at 12:17

line.translate(ord(i):None for i in '!@#$')

Same as any other character. Python lets you use pairs of either single or double quotes. So you just write "'" for the character set.
– intuited
Nov 26 '15 at 20:14

"'"

@naveen's comment above worked for me. Pythony 2.7.13. In my case I wanted to strip " and ' characters: notes = notes.translate(ord(i):None for i in '"'')
– RyanG
Mar 14 '17 at 14:18

notes = notes.translate(ord(i):None for i in '"'')

Am I missing the point here, or is it just the following:

>>> string = "ab1cd1ef" >>> string.replace("1","") 'abcdef' >>>

Put it in a loop:

>>> >>> a = "a!b@c#d$" >>> b = "!@#$" >>> for char in b: ... a = a.replace(char,"") ... >>> print a abcd >>>

This will make a copy of the string in each loop, which might not be desirable. Also it is not very good Python. In Python you would loop like this instead: for char in b: a=a.replace(char,"")
– elgehelge
Oct 18 '14 at 14:05

for char in b: a=a.replace(char,"")

using user defined variables which overlaps system classes is not a good idea. You would better use variable STRING instead of STR and C instead of CHAR.
– Ayrat
Aug 26 '16 at 14:06

>>> line = "abc#@!?efg12;:?" >>> ''.join( c for c in line if c not in '?:!/;' ) 'abc#@efg12'

Thank you very much !! I was trying to remove a Japanese Yen symbol from a string I'd parsed in via XML http response. This solution worked and avoided alot of Unicode hassle. # -- coding: utf-8 -- yenSymbol = ord(u'u00A5') cpc = ''.join( c for c in cpcWithYen if ord(c) != yenSymbol )
– arcseldon
Jan 22 '14 at 12:48

how can i remove `` character ?????? i can't do this with any of these ways
– Arash Hatami
Jan 31 '16 at 8:37

use another string delimitor such as ''' or "
– ASignor
Jun 1 '17 at 20:00

The asker almost had it. Like most things in Python, the answer is simpler than you think.

>>> line = "H E?.LL!/;O:: " >>> for char in ' ?.!/;:': ... line = line.replace(char,'') ... >>> print line HELLO

You don't have to do the nested if/for loop thing, but you DO need to check each character individually.

how can i remove `` character ?????? i can't do this with any of these ways
– Arash Hatami
Jan 31 '16 at 8:36

yes i know, probably too late, but should work if you escape it. Like this: line = line.replace('`', '') read on: learnpythonthehardway.org/book/ex10.html
– Aiyion.Prime
Aug 26 '16 at 12:45

For the inverse requirement of only allowing certain characters in a string, you can use regular expressions with a set complement operator [^ABCabc]. For example, to remove everything except ascii letters, digits, and the hyphen:

[^ABCabc]

>>> import string >>> import re >>> >>> phrase = ' There were "nine" (9) chick-peas in my pocket!!! ' >>> allow = string.letters + string.digits + '-' >>> re.sub('[^%s]' % allow, '', phrase) 'Therewerenine9chick-peasinmypocket'

From the python regular expression documentation:

Characters that are not within a range can be matched by complementing
the set. If the first character of the set is '^', all the characters
that are not in the set will be matched. For example, [^5] will match
any character except '5', and [^^] will match any character except
'^'. ^ has no special meaning if it’s not the first character in the
set.

'^'

[^5]

[^^]

'^'

^

how can i remove `` character ?????? i can't do this with any of these ways
– Arash Hatami
Jan 31 '16 at 8:36

line = line.translate(None, " ?.!/;:")

+1 When using unicode it requires setting up a translation to delete instead of a delete string. docs.python.org/library/stdtypes.html#str.translate
– kevpie
Oct 15 '10 at 4:07

This is a great suggestion (ref: docs.python.org/2/library/string.html#string.translate ) The unicode note is good as well.
– cgseller
Dec 16 '15 at 19:25

Easy peasy with `re.sub` in Python 3.5

re.sub

re.sub(' |?|.|!|/|;|:', '', line)

>>> import re >>> line = 'Q: Do I write ;/.??? No!!!' >>> re.sub(' |?|.|!|/|;|:', '', line) 'QDoIwriteNo'

In regular expressions (regex), | is a logical OR and escapes spaces and special characters that might be actual regex commands. sub stands for substitution.

|

sub

>>> s = 'a1b2c3' >>> ''.join(c for c in s if c not in '123') 'abc'

My answer does provide a solution to the original question, but I was also interested (an perhaps the OP as well) in feedback as to why my solution might not be ideal. Should I have created a new question and referenced this one for context?
– eatkin
Oct 19 '15 at 21:05

Strings are immutable in Python. The replace method returns a new string after the replacement. Try:

replace

for char in line: if char in " ?.!/;:": line = line.replace(char,'')

How can you iterate over line and modify it at the same time?
– eumiro
Oct 15 '10 at 12:40

@eumiro: The iteration proceeds over the original line.
– Greg Hewgill
Oct 15 '10 at 18:57

line

good to know! So if I iterate over an array, I iterate over an original array. Iteration over an iterator wouldn't be possible.
– eumiro
Oct 15 '10 at 19:09

I was surprised that no one had yet recommended using the builtin filter function.

import operator import string # only for the example you could use a custom string s = "1212edjaq"

Say we want to filter out everything that isn't a number. Using the filter builtin method "...is equivalent to the generator expression (item for item in iterable if function(item))" [Python 3 Builtins: Filter]

sList = list(s) intsList = list(string.digits) obj = filter(lambda x: operator.contains(intsList, x), sList)))

In Python 3 this returns

>> <filter object @ hex>

To get a printed string,

nums = "".join(list(obj)) print(nums) >> "1212"

I am not sure how filter ranks in terms of efficiency but it is a good thing to know how to use when doing list comprehensions and such.

UPDATE

Logically, since filter works you could also use list comprehension and from what I have read it is supposed to be more efficient because lambdas are the wall street hedge fund managers of the programming function world. Another plus is that it is a one-liner that doesnt require any imports. For example, using the same string 's' defined above,

num = "".join([i for i in s if i.isdigit()])

That's it. The return will be a string of all the characters that are digits in the original string.

If you have a specific list of acceptable/unacceptable characters you need only adjust the 'if' part of the list comprehension.

target_chars = "".join([i for i in s if i in some_list])

or alternatively,

target_chars = "".join([i for i in s if i not in some_list])

Using filter, you'd just need one line

filter

line = filter(lambda char: char not in " ?.!/;:", line)

This treats the string as an iterable and checks every character if the lambda returns True:

lambda

True

>>> help(filter) Help on built-in function filter in module __builtin__: filter(...) filter(function or None, sequence) -> list, tuple, or string Return those items of sequence for which function(item) is true. If function is None, return the items that are true. If sequence is a tuple or string, return the same type, else return a list.

>>> # Character stripping >>> a = '?abcd1234!!' >>> t.lstrip('?') 'abcd1234!!' >>> t.strip('?!') 'abcd1234'

Only removes characters from the beginning or end of the string
– divenex
Jan 10 at 16:52

Here's my Python 2/3 compatible version. Since the translate api has changed.

def remove(str_, chars): """Removes each char in `chars` from `str_`. Args: str_: String to remove characters from chars: String of to-be removed characters Returns: A copy of str_ with `chars` removed Example: remove("What?!?: darn;", " ?.!:;") => 'Whatdarn' """ try: # Python2.x return str_.translate(None, chars) except TypeError: # Python 3.x table = ord(char): None for char in chars return str_.translate(table)

I'd use dict.fromkeys(map(ord, '!@#$')) to create the map.
– Martijn Pieters♦
Apr 8 '17 at 16:03

dict.fromkeys(map(ord, '!@#$'))

map is generally less readable than a list/dict/set/generator comprehension. So much so that Guido wanted to remove it from the language. Using fromkeys is also a bit clever and requires a doc check.
– Bryce Guinta
Apr 8 '17 at 23:21

map

fromkeys

#!/usr/bin/python import re strs = "how^ much for the maple syrup? $20.99? That's ricidulous!!!" print strs nstr = re.sub(r'[?|$|.|!|a|b]',r' ',strs)#i have taken special character to remove but any #character can be added here print nstr nestr = re.sub(r'[^a-zA-Z0-9 ]',r'',nstr)#for removing special character print nestr

how can i remove `` character ?????? i can't do this with any of these ways
– Arash Hatami
Jan 31 '16 at 8:36

Do you mean speech marks? re has backslash to escape the code and consider ' as a string. docs.python.org/2/library/re.html
– JasTonAChair
May 5 '16 at 1:04

'

How about this:

def text_cleanup(text): new = "" for i in text: if i not in " ?.!/;:": new += i return new

Could you elaborate more your answer adding a little more description about the solution you provide?
– abarisone
Mar 24 '15 at 8:18

Below one.. with out using regular expression concept..

ipstring ="text with symbols!@#$^&*( ends here" opstring='' for i in ipstring: if i.isalnum()==1 or i==' ': opstring+=i pass print opstring

You can also use a function in order to substitute different kind of regular expression or other pattern with the use of a list. With that, you can mixed regular expression, character class, and really basic text pattern. It's really useful when you need to substitute a lot of elements like HTML ones.

*NB: works with Python 3.x

import re # Regular expression library def string_cleanup(x, notwanted): for item in notwanted: x = re.sub(item, '', x) return x line = "<title>My example: A text %very% $clean!!</title>" print("Uncleaned: ", line) # Get rid of html elements html_elements = ["<title>", "</title>", "", ""] line = string_cleanup(line, html_elements) print("1st clean: ", line) # Get rid of special characters special_chars = ["[!@#$]", "%"] line = string_cleanup(line, special_chars) print("2nd clean: ", line)

In the function string_cleanup, it takes your string x and your list notwanted as arguments. For each item in that list of elements or pattern, if a substitute is needed it will be done.

The output:

Uncleaned: <title>My example: A text %very% $clean!!</title> 1st clean: My example: A text %very% $clean!! 2nd clean: My example: A text very clean

My method I'd use probably wouldn't work as efficiently, but it is massively simple. I can remove multiple characters at different positions all at once, using slicing and formatting.
Here's an example:

words = "things" removed = "%s%s" % (words[:3], words[-1:])

This will result in 'removed' holding the word 'this'.

Formatting can be very helpful for printing variables midway through a print string. It can insert any data type using a % followed by the variable's data type; all data types can use %s, and floats (aka decimals) and integers can use %d.

Slicing can be used for intricate control over strings. When I put words[:3], it allows me to select all the characters in the string from the beginning (the colon is before the number, this will mean 'from the beginning to') to the 4th character (it includes the 4th character). The reason 3 equals till the 4th position is because Python starts at 0. Then, when I put word[-1:], it means the 2nd last character to the end (the colon is behind the number). Putting -1 will make Python count from the last character, rather than the first. Again, Python will start at 0. So, word[-1:] basically means 'from the second last character to the end of the string.

So, by cutting off the characters before the character I want to remove and the characters after and sandwiching them together, I can remove the unwanted character. Think of it like a sausage. In the middle it's dirty, so I want to get rid of it. I simply cut off the two ends I want then put them together without the unwanted part in the middle.

If I want to remove multiple consecutive characters, I simply shift the numbers around in the (slicing part). Or if I want to remove multiple characters from different positions, I can simply sandwich together multiple slices at once.

Examples:

words = "control" removed = "%s%s" % (words[:2], words[-2:])

removed equals 'cool'.

words = "impacts" removed = "%s%s%s" % (words[1], words[3:5], words[-1])

removed equals 'macs'.

In this case, [3:5] means character at position 3 through character at position 5 (excluding the character at the final position).

Remember, Python starts counting at 0, so you will need to as well.

e.g.,

os.rename(file_name, file_name.translate(ord(c): None for c in '0123456789'))

To remove all the number from the string

you can use set

charlist = list(set(string.digits+string.ascii_uppercase) - set('10IO')) return ''.join([random.SystemRandom().choice(charlist) for _ in range(passlen)])

When giving an answer it is preferable to give some explanation as to WHY your answer is the one.
– Stephen Rauch
Feb 17 '17 at 2:28

Try this one:

def rm_char(original_str, need2rm): ''' Remove charecters in "need2rm" from "original_str" ''' return original_str.translate(str.maketrans('','',need2rm))

This method works well in python 3.5.2

Recursive split:
s=string ; chars=chars to remove

def strip(s,chars): if len(s)==1: return "" if s in chars else s return strip(s[0:int(len(s)/2)],chars) + strip(s[int(len(s)/2):len(s)],chars)

example:

print(strip("Hello!","lo")) #He!

Even the below approach works

line = "a,b,c,d,e" alpha = list(line) while ',' in alpha: alpha.remove(',') finalString = ''.join(alpha) print(finalString)

output >> abcde

Here's some possible ways to achieve this task:

def attempt1(string): return "".join([v for v in string if v not in ("a", "e", "i", "o", "u")]) def attempt2(string): for v in ("a", "e", "i", "o", "u"): string = string.replace(v, "") return string def attempt3(string): import re for v in ("a", "e", "i", "o", "u"): string = re.sub(v, "", string) return string def attempt4(string): return string.replace("a", "").replace("e", "").replace("i", "").replace("o", "").replace("u", "") for attempt in [attempt1, attempt2, attempt3, attempt4]: print(attempt("murcielago"))

PS: Instead using " ?.!/;:" the examples use the vowels... and yeah, "murcielago" is the Spanish word to say bat... funny word as it contains all the vowels :)

PS2: If you're interested on performance you could measure these attempts with a simple code like:

import timeit K = 1000000 for i in range(1,5): t = timeit.Timer( f"attempti('murcielago')", setup=f"from __main__ import attempti" ).repeat(1, K) print(f"attempti",min(t))

In my box you'd get:

attempt1 2.2334518376057244 attempt2 1.8806643818474513 attempt3 7.214925774955572 attempt4 1.7271184513757465

So it seems attempt4 is the fastest one for this particular input.

You are creating a needless list in attempt1 and the tuple can be rewritten to "aeiou" for simplicity sakes (removing [ and ] will turn in into a generator without creating a list). You create tons of throwaway intermediary strings in attemt2, you use multiple applications of regex in attempt3 where you could use r'[aeiou]' in one pass. each one has flaws - its nice to see different ways to do things, but please fix them to be good attempts as well
– Patrick Artner
Jul 22 at 13:08

list

attempt1

"aeiou"

[

]

attemt2

attempt3

r'[aeiou]'

@PatrickArtner You're absolutely right... from the dozens ways I've got in mind to achieve this task I've picked up the slower ones (wanted to show the OP some easiest ones)... That said, after you guys closed the other thread I've lost motivation to put more effort on this already answered old thread, so... :) . Thanks for the points though.
– BPL
Jul 22 at 13:14

@PatrickArtner Ok... just for just sake added a new one, "attempt4"... haven't measured but I think that one should be the faster one
– BPL
Jul 22 at 13:18

@PatrickArtner Edited... attempt4 was the fastest from the little set of attempts. Anyway, I'm not wasting more time with this stuff :)
– BPL
Jul 22 at 13:39

This is the most pythonic way I think it could be.

clean_string = ''.join(dirty_string.strip(char_you_want_to_remove))

You have to reassign your str variable:

for char in line: if char in " ?.!/;:": line = line.replace(char,'')

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Search This Blog

YTjnti