1

I have a folder with a lot of files. I want to create a loop or function where I can read all the files and create corresponding data frames for each one of the files with the prefix format 'filename'+_df.

I have this code that reads everything and creates a large data frame from all the files in the folder, but I'm not sure how to modify it to create multiple data frames and use the filename as a prefix:

import os
import pandas as pd

path = os.path.join(os.getcwd(),'folder')

files = [os.path.join(path,i) for i in os.listdir(path) if os.path.isfile(os.path.join(path,i))]

df = pd.DataFrame()

for file in files:
    _df = pd.read_csv(file)
    df = df.append(_df)

If there's a better way to do this, that would be great as well.

2
  • you can store all the DataFrames into a list. Then concat all dataframes into one
    – Joe Ferndz
    Feb 16, 2021 at 6:15
  • I don't want to create a single data frame. I want to create a data frame per file Feb 16, 2021 at 6:17

1 Answer 1

1

You can create dictionary of DataFrames in dictionary comprehension:

dfs = {f'{file}_df': pd.read_csv(file) for file in files}

Then for each DataFrame select by key, e.g. file='myfile':

dfs['myfile_df']

Thank you @Abdul Niyas P M for idea:

from pathlib import Path
dfs = {f'{Path(file).stem}_df': pd.read_csv(file) for file in files}

Or:

import os
dfs = {f'{os.path.basename(file)}_df': pd.read_csv(file) for file in files}
1
  • 1
    In the OP the files seems like a list of abs paths. May be you can use Pathlib to get only the filename. dfs = {f'{Path(file).stem}_df': pd.read_csv(file) for file in files} Feb 16, 2021 at 6:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.