0

I am working with a large excel file having 22 sheets, where each sheet has the same coulmn headings but do not have equal number of rows. I would like to obtain the mean values (excluding zeros) for columns AA to AX for all the 22 sheets. The columns have titles which I use in my code.

Rather than reading each sheet, I want to loop through the sheets and get as output the mean values. With help from answers to other posts, I have this:

import pandas as pd

xls = pd.ExcelFile('myexcelfile.xlsx')
xls.sheet_names
#print(xls.sheet_names)
out_df = pd.DataFrame()

for sheets in xls.sheet_names:
    df = pd.read_excel('myexcelfile.xlsx', sheet_names= None)
    df1= df[df[:]!=0]
    df2=df1.loc[:,'aa':'ax'].mean()
    out_df.append(df2)  ## This will append rows of one dataframe to another(just like your expected output)

print(out_df2)

## out_df will have data from all the sheets

The code works so far, but only one of the sheets. How do I get it to work for all 22 sheets?

2 Answers 2

0

You can use numpy to perform basic math on pandas.DataFrame or pandas.Series take a look at my code below

import pandas as pd, numpy as np

XL_PATH = r'C:\Users\YourName\PythonProject\Book1.xlsx'


xlFile = pd.ExcelFile(XL_PATH)
xlSheetNames = xlFile.sheet_names

dfList = []     # variable to store all DataFrame

for shName in xlSheetNames:
    df = pd.read_excel(XL_PATH, sheet_name=shName)  # read sheet X as DataFrame
    dfList.append(df)   # put DataFrame into a list


for df in dfList:
    print(df)
    dfAverage = np.average(df)  # use numpy to get DataFrame average
    print(dfAverage)
2
  • Hi! Thanks for your input. I did this, but still it works on only one sheet.
    – Neddy
    Nov 18, 2020 at 12:02
  • I use df = df.replace(0, np.NaN) to exclude the zeros while getting the mean values
    – Neddy
    Nov 18, 2020 at 12:03
0
#Try code below

import pandas as pd, numpy as np, os

XL_PATH = "YOUR EXCEL FULL PATH"
SH_NAMES = "WILL CONTAINS LIST OF EXCEL SHEET NAME"
DF_DICT = {} """WILL CONTAINS DICTIONARY OF DATAFRAME"""

def readExcel():
        if not os.path.isfile(XL_PATH): return FileNotFoundError
        SH_NAMES = pd.ExcelFile(XL_PATH).sheet_names
        
        # pandas.read_excel() have argument 'sheet_name'
        # when you put a list to 'sheet_name' argument
        # pandas will return dictionary of dataframe with sheet_name as keys
        DF_DICT = pd.read_excel(XL_PATH, sheet_name=SH_NAMES)
        return SH_NAMES, DF_DICT
        
#Now you have DF_DICT that contains all DataFrame for each sheet in excel
#Next step is to append all rows data from Sheet1 to SheetX
#This will only works if you have same column for all DataFrame

def appendAllSheets():
    dfAp = pd.DataFrame()
    for dict in DF_DICT:
        df = DF_DICT[dict]
        dfAp = pd.DataFrame.append(self=dfAp, other=df)
    return dfAp
    
#you can now call the function as below:
dfWithAllData = appendAllSheets()
#now you have one DataFrame with all rows combine from Sheet1 to SheetX
#you can fixed the data, for example to drop all rows which contain '0'
dfZero_Removed = dfWithAllData[[dfWithAllData['Column_Name'] != 0]]
dfNA_removed = dfWithAllData[not[pd.isna(dfWithAllData['Column_Name'])]]

#last step, to find average or other math operation
#just let numpy do the job
average_of_all_1 = np.average(dfZero_Removed)
average_of_all_2 = np.average(dfNA_Removed)
#show result
#this will show the average of all
#rows of data from Sheet1 to SheetX from your target Excel File
print(average_of_all_1, average_of_all_2)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.