0

I have a df with 18 columns and 15K rows.

df.info() gives for first column:

'''

0   Behandelcode                17451 non-null  object

''' Converting all values in columns 'Behandelcode' to integers fails because some strings have a letter at the end, example: '''

(405, '33971'),
 (406, '38154'),
 (407, '033620A'),
 (408, '33971'),

''' Every time a string has a letter at the end, length of string is 7.

I've been trying for too long now. So once again I need your help.

The question is: how can i iterate over column: df2['Behandelcode'], so that all values are kept in place, but (for example) the A in 0336620A on index row 407 gets deleted and only; 0336620 stays in place. And this for all the values ending with a letter.

I tried this, didn't work.... ( I did i, value because df['Behandelcode'] gives a series.

'''

for i, value in enumerate(df2['Behandelcode']):
y = len(value)
if y == 7:
    value[:-1]
else:
    value = value

''' Maybe there is a safer / more clean python method then working with len. For now, first things first and that is cleaning op this column so I can set it astype(int32). I would be very very thankfull if you can help me. greetings Jan

4 Answers 4

1

You could use the str.replace() to modify strings in bulk by using regular expressions:

df2['Behandelcode'].str.replace(r'(?P<match>\d{6}).*', lambda x: x.group('match'))

This expression will succesfully match only values with atleast 6 Digits continued by an indefinite number of characters, and will truncate it to only the first 6 Digits

3
  • Thanks! I can see this works, but how to get this inplace in the column? I mean: your code works, but it doesn't seem to change my df. thanks again!
    – Janneman
    Aug 9, 2020 at 8:59
  • I added some extra code, results to my question. thanks!
    – Janneman
    Aug 9, 2020 at 9:08
  • 1
    No Problem, just as a side note if there's any legit integers that are made up with 6 digits you could try adjusting the regular expression i.e. ... r'(?P<match>\d*)[A-Za-z]' ... will keep the first concatenation of numbers with any length and delete any trailing letters, including both Upper or Lower case i.e. '123456789AABbZZz' will be changed to '123456789'
    – SGar28
    Aug 9, 2020 at 10:26
0

Here is a way, based on looking at the last position in the string:

# the data frame
print(df)

    id     code
0  405    33971
1  406    38154
2  407  033620A
3  408    33971
4  409  035774A  # <-- new last element

# list of letters, to be stripped
letters = 'ABCD' # extend to all letters in alphabet...

# results
df['code'].apply(lambda x: x[:-1] if x[-1] in letters else x)

0     33971
1     38154
2    033620
3     33971
4    035774
Name: code, dtype: object

UPDATE: I added a new element to the data frame (409, '035774A') and re-ran the code. In my environment, the trailing 'A' was removed.

0
0

Thanks jsmart! Sadly it doesn't seem to work.

df2.iloc['Behandelcode'] for example stil gives: '035774A'.

Maybe a view of part of my df helps?

enter image description here

A morning edit to my post: thank you all in for trying to help me out.

Maybe this will help: This code gives a list that looks like the part of the list below the code. When I run len(zeven) it gives the value of 1. For me this is strange because alle the df['Behandelcode'] values with lengt 7 are in this list.

'''

for x in df2['Behandelcode']:
    zeven = []
    if len(x) == 7:
        print(x)
        zeven.append(x)
    else:
        x=x

''' This results in this (part of total list zeven):

035774A
035774A
035774A
033620D
035774A
033620D
035774A
033620A
035774A

I tried this code as well. It runs (made a copy of df2 > df3) without Error but doesn't effect my column....

'''

for x in df3['Behandelcode']:
    zeven = []
    if len(x) == 7:
        df3['Behandelcode'].apply(lambda x: x[:-1])
        zeven.append(x)
    else:
        x=x

'''

It feels like the solution should be simple, but still can't figure it out: how to get rid of all the letters at the end of these strings so afterwards I will be able to convert them to integers. Thanks again!

2
  • did you mean to use iloc in this statement df2.iloc['Behandelcode'] -- iloc[] expects integer arguments (it is position-based indexing). Also, please can you post example data frames in executable statements.
    – jsmart
    Aug 8, 2020 at 19:04
  • Hey. I added some extra information to my question. Thanks again!
    – Janneman
    Aug 9, 2020 at 8:06
0

Sgar28 thanks again. Question is: how to get this inplace in my df?

Before your lambda function (last rows of output):

''' df['Behandelcode']

'''

17446      31802
17447      31802
17448      31802
17449      31802
17450    031714A

apply your lambda: '''

df['Behandelcode'].str.replace(r'(?P<match>\d{6}).*', lambda x: x.group('match'))

''' gives:

17446     31802
17447     31802
17448     31802
17449     31802
17450    031714

But checking my df with: '''

df.loc[17450]['Behandelcode']

''' gives:

'031714A'

The solution is simple to my last question...... '''

df['Behandelcode'] = df['Behandelcode'].str.replace(r'(?P<match>\d{6}).*', lambda x: x.group('match'))

'''

Thanks again!!!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.