0

I am reading a sheet using pandas. After reading the sheet, I am getting an empty row between the values.

So, I need to find the index value of that row and delete all the rows below that, then make a new data-frame.

from xlrd import open_workbook

import pandas as pd

from pandas import ExcelWriter

pathbook = open_workbook("S:\\1. DIRECTORY MASTER\\FINANCIAL RESEARCH\\Data 
Initiative - PROJECTS\\Market Rollout\\"
                     "Modified Files\\2016\\2016A-3032 - CA.xlsx")
pathbook_sheet = pathbook.sheet_by_name("1-Rollout")

file = "S:\\1. DIRECTORY MASTER\\FINANCIAL RESEARCH\\Data Initiative - 
PROJECTS\\Market Rollout\\" \
   "Modified Files\\2016\\2016A-3032 - CA.xlsx"

for rowidx in range(pathbook_sheet.nrows):
    row = pathbook_sheet.row(rowidx)
    for colidx, cell in enumerate(row):
        if cell.value == "Canadian Market":
            print("Sheet Name:", pathbook_sheet.name)
            print("Row Number:", rowidx)
            CADvalue = int(rowidx)
            CADvalue += 1

print(CADvalue)
reading_book = pd.read_excel(file, sheet_name="1-Rollout", 
skiprows=CADvalue, index_col=0).iloc[:12]

write = ExcelWriter("Final" + ".xlsx")
reading_book.to_excel(write, 'Sheet1', index=False)
write.save()

The example output in the excel file i am getting

Sales 2016 2017 2018 2019 2020 2021 Units Sold 0 0 0 4 14 37 Unit Sale Price 1285 1285 1285 1285 1285 1285 Unit Profit 4000 4000 4000 4000 4000 4000
Rest of the World Market

So there is an empty row between the last 3 rows

5
  • Can you show data and expected output? May 15, 2018 at 14:10
  • What is empty? 0? NaN? ` `?
    – JE_Muc
    May 15, 2018 at 14:14
  • @ScottBoston it done in the edit
    – sid soni
    May 15, 2018 at 14:24
  • @sidsoni I still can't see what the empty row looks like. Is it just empty strings?
    – JE_Muc
    May 15, 2018 at 14:26
  • @Scotty1- It is just an empty row in the excel sheet with no values in it - like a blank line and when i print the data-frame it shows the "nan" in every column.
    – sid soni
    May 15, 2018 at 14:33

2 Answers 2

1
#First, find NaN entries in first column
blank_row_bool = reading_book.iloc[:,1].isna()
#Next, get index of first NaN entry
blank_row_index =  [i for i, x in enumerate(blank_row_bool) if x][0]
#Finally, restrict dataframe to rows before the first NaN entry
reading_book = reading_book.iloc[:(blank_row_index-1)]

Or, in a single line:

reading_book = reading_book.iloc[:([i for i, x in 
enumerate(reading_book.iloc[:,1].isna()) if x][0]-1)]
0

The solution depends on what empty means. If it is just an empty string, as in '', the code to find the index would be:

empty = ''
idx_first_empty_row = reading_book.index[reading_book.iloc[:, 0] == empty][0]

This works if the first column is empty. If for example "empty" means NaN, then replace the line with:

idx_first_empty_row = reading_book.index[np.isnan(reading_book.iloc[:, 0])]

This works if the dtype of the rows is any numeric numpy type, like np.float64.

If the dtype is not any numpy numeric type, try the following:

idx_first_empty_row = np.where(reading_book.iloc[:, 0].isnull().values == True)

You can also, depending on the types of data in your rows, try this:

idx_first_empty_row = reading_book.index[reading_book.iloc[:, 0].isnull().values]
17
  • if i do with empty = ' ' the i get an following error. File "C:/Users/s.soni/PycharmProjects/marketdatarollout/Test2.py", line 27, in <module> idx_first_empty_row = reading_book.index[reading_book.iloc[:, 0] == empty] File "C:\Users\s.soni\PycharmProjects\marketdatarollout\venv\lib\site-packages\pandas\core\ops.py", line 879, in wrapper res = na_op(values, other) File "C:\Users\s.soni\PycharmProjects\marketdatarollout\venv\lib\site-packages\pandas\core\ops.py", line 818, in na_op raise TypeError("invalid type comparison") TypeError: invalid type comparison
    – sid soni
    May 15, 2018 at 14:27
  • But if i try with empty = np.nan the i get the following result by printing print(idx_first_empty_row). The result is - Float64Index([], dtype='float64')
    – sid soni
    May 15, 2018 at 14:30
  • You should improve your formatting. But considering your problem: Could you please post the content of your empty line? The problem is exactly what I was asking before: What does the "empty" row look like? Is it filled with empty strings, zeros or NaN?
    – JE_Muc
    May 15, 2018 at 14:34
  • It is just an empty row in the excel sheet with no values in it - like a blank line and when i print the data-frame it shows the "nan" in every column.
    – sid soni
    May 15, 2018 at 14:35
  • Ahhhh, so it is NaN values... Ok, one second.
    – JE_Muc
    May 15, 2018 at 14:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.