12

I want to iterate through every row in a pandas DataFrame, and do something with the elements in each row.

Right now I have

for row in df.iterrows(): 
    if row['col'] > 1.5:
        doSomething

but it tells me that the 'tuple indices must be integers, not str' . How do I access the column that I want in a certain row?

4
  • 1
    I would question why do it this way? The whole point of using pandas is to try to perform operations on the whole series or dataframe. If all you wanted to do was perform some operation just on the rows that met that criteria then df.loc[df['col']>1.5, 'col'] = doSomething would achieve the same result and will be blisteringly fast as it will be vectorised
    – EdChum
    Oct 7, 2014 at 19:57
  • okay, great. where would i put an alternate doSomething if it wasn't greater than 1.5? Oct 7, 2014 at 20:06
  • you can either do a np.where or just 2 statements so either df['col'] = np.where(df['col'] > 1.5, doSomething, doSomethingElse) or add another statement for the opposite condition df.loc[df['col'] <=1.5, 'col'] = doSomethingElse
    – EdChum
    Oct 7, 2014 at 20:25
  • The point here is to avoid looping unless there is no way to avoid it, at the moment your code snippet doesn't specify what you really want to do so we can only guess but it's likely that you don't want to loop over the rows and you really should try to use a vectorised method where possible
    – EdChum
    Oct 7, 2014 at 20:26

3 Answers 3

8

You can use apply function with option axis=1. For example:

def my_function(row):
    if row['col'] > 1.5:
        doSomething()
    else:
        doSomethingElse()

my_df.apply(my_function, axis=1)

source

1
  • 2
    This is the best answer. Because it is the most efficient. Using for loop takes a lot more time. using vector functions instead of for loop takes less time to compute. Watch this video by Udacity how to optimize code using vector functions of pandas and NumPy. youtu.be/WF9n_19V08g
    – Ahwar
    Aug 10, 2020 at 5:17
7

Probably the simplest solution is to use the APPLYMAP or APPLY fucntions which applies the function to every data value in the entire data set.

You can execute this in a few ways:

df.applymap(someFunction)

or

df[["YourColumns"]].apply(someFunction)

The Links are below:

ApplyMap Docs

Apply Docs

4

iterrows yields (index, Series) pairs. Therefore, use:

for index, row in df.iterrows(): 
    if row['col'] > 1.5:
        doSomething

Note, however, that a DataFrame is a primarily column-based data structure, so you'll get better performance if you can structure your code around column-wise operations, instead of row-wise operations.

2
  • thanks! These pairs are immutable, correct? Is there a way I could rewrite the element in the column? Oct 7, 2014 at 19:57
  • 1
    EdChum's method, df.loc[df['col']>1.5, 'col'] = doSomething would be better, assuming doSomething is a number. If not, you'll have to explain in more detail what doSomething is.
    – unutbu
    Oct 7, 2014 at 20:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.