12

I have been at this all morning and have slowly pieced things together. But for the life of me I can not figure out how to use the .str.startswith() function in Pandas.

My XLSX spreadsheet is as follows

1 Name, Registration Date, Phone number
2 John Doe, 2015-11-20T19:54:45Z, 1.1112223333
3 Jane Doe, 2015-11-20T20:44:26Z, 65.1112223333
etc...

So I am importing it as a data frame, cleaning the header so that there are no spaces and such, then I want to delete any rows not starting with '1.' (or keep rows that start with '1.') and delete all others. So in this short example, delete the entire 'Jane Doe' entry since her phone number starts with '65.'

import pandas as pd
df = pd.read_excel('testingpanda.xlsx', sheetname = 'Export 1')
def colHeaderCleaner():
    cols = df.columns
    cols = cols.map(lambda x: x.replace(' ', '_') if isinstance(x, (str, unicode)) else x)
    df.columns = cols
    df.columns = [x.lower() for x in df.columns]

colHeaderCleaner()

#by default it sets the values in 'registrant_phone' as float64, so this is fixing that...
df['registrant_phone'] = df['registrant_phone'].astype('object')

The closest I have gotten, and by that I mean the only line I have been able to execute without annoying tracebacks and other errors is:

df['registrant_phone'] = df['registrant_phone'].str.startswith('1')

But all that does is convert all phone values to 'NaN', it maintains all of the rows and everything as shown below:

print df
[output] name, registration_date, phone_number
[output] John Doe, 2015-11-20T19:54:45Z, NaN
[output] Jane Doe, 2015-11-20T20:44:26Z, NaN

I have searched far too many places to even try to list, I have tried different versions of df.drop and just can't seem to figure anything out. Where do I go from here?

1
  • you want df = df.loc[df['registrant_phone'].str.startswith('1')]
    – EdChum
    Commented Feb 3, 2016 at 19:55

1 Answer 1

37

I am a bit confused by your question. In any case, if you have a DataFrame df with a column 'c', and you would like to remove the items starting with 1, then the safest way would be to use something like:

df = df[~df['c'].astype(str).str.startswith('1')]
2
  • That worked. Well...with a few modifications, it needed a 's' at the end to make it 'startswith' and it flipped and kept everything I didn't want, but I took out the ~ and it worked. What is the ~ saying? is that just another way of saying 'is not'? I guess my real problem was the 'object' part, from what I was reading 'object' was Pandas' version of str, but apparently not quite. Thanks for the help!
    – Mxracer888
    Commented Feb 3, 2016 at 20:24
  • 1
    You're very welcome. ~ is indeed negation in numpy-ish.
    – Ami Tavory
    Commented Feb 3, 2016 at 20:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.