Here's an MWE of some code I'm using. I slowly whittle down an initial dataframe via slicing and some conditions until I have only the rows that I need. Each block of five rows actually represents a different object so that, as I whittle things down, if any one row in each block of five meets the criteria, I want to keep it -- this is what the loop over keep.index accomplishes. No matter what, when I'm done I can see that the final indices I want exist, but I get an error message saying "IndexError: positional indexers are out-of-bounds." What is happening here?
import pandas as pd
import numpy as np
temp = np.random.rand(100,5)
df = pd.DataFrame(temp, columns=['First', 'Second', 'Third', 'Fourth', 'Fifth'])
df_cut = df.iloc[10:]
keep = df_cut.loc[(df_cut['First'] < 0.5) & (df_cut['Second'] <= 0.6)]
new_indices_to_use = []
for item in keep.index:
remainder = (item % 5)
add = np.arange(0-remainder,5-remainder,1)
inds_to_use = item + add
new_indices_to_use.append(inds_to_use)
new_indices_to_use = [ind for sublist in new_indices_to_use for ind in sublist]
final_indices_to_use = []
for item in new_indices_to_use:
if item not in final_indices_to_use:
final_indices_to_use.append(item)
final = df_cut.iloc[final_indices_to_use]
final_indices_to_use
and verifying it is what you think it is? On your example I get[10, 11, ..., 98, 99]
andlen(df_cut)
gives90
..loc
instead of.iloc
in your last line --.iloc
is for positional access, but you want label-based access.df_cut
has a length of 90; indices above89
will give anIndexError
.