1

Edit: I believe this was all user error. I have been typing df.T by default, and it just occurred to me that this is very likely the TRANSPOSE output. By typing df, the data frame is output normally (headers as columns). Thank you for those who stepped up to try and help. In the end, it was just my misunderstanding of pandas language..

Original Post

I'm not sure if I am making a simple mistake but the columns in a .csv file are being imported as rows using pd.read_csv. The dataframe turns out to be 5 rows by 2000 columns. I am importing only 5 columns out of 14 so I set up a list to hold the names of the columns I want. They match exactly those in the .csv file. What am I doing wrong here?

import os
import numpy as np
import pandas as pd

fp = 'C:/Users/my/file/path'
os.chdir(fp)

cols_to_use = ['VCOMPNO_CURRENT', 'MEASUREMENT_DATETIME',
               'EQUIPMENT_NUMBER', 'AXLE', 'POSITION']

df = pd.read_csv('measurement_file.csv',
                 usecols=cols_to_use,
                 dtype={'EQUIPMENT_NUMBER': np.int,
                        'AXLE': np.int},
                 parse_dates=[2],
                 infer_datetime_format=True)

Output:

                                    0  ...            2603
VCOMPNO_CURRENT                T92656  ...          T5M247
MEASUREMENT_DATETIME  7/26/2018 13:04  ...  9/21/2019 3:21
EQUIPMENT_NUMBER                  208  ...             537
AXLE                                1  ...               6
POSITION                            L  ...               R

[5 rows x 2000 columns]

Thank you.

Edit: To note, if I import the entire .csv with the standard pd.read_csv('measurement_file.csv'), the columns are imported properly.

Edit 2: Sample csv:

VCOMPNO_CURRENT,MEASUREMENT_DATETIME,REPAIR_ORDER_NUMBER,EQUIPMENT_NUMBER,AXLE,POSITION,FLANGE_THICKNESS,FLANGE_HEIGHT,FLANGE_SLOPE,DIAMETER,RO_NUMBER_SRC,CL,VCOMPNO_AT_MEAS,VCOMPNO_SRC
T92656,10/19/2018 7:11,5653054,208,1,L,26.59,27.34,6.52,691.3,OPTIMESS_DATA,2MTA ,T71614 ,RO_EQUIP     
T92656,10/19/2018 7:11,5653054,208,1,R,26.78,27.25,6.64,691.5,OPTIMESS_DATA,2MTA ,T71614 ,RO_EQUIP     
T92656,10/19/2018 7:11,5653054,208,2,L,26.6,27.13,6.49,691.5,OPTIMESS_DATA,2MTA ,T71614 ,RO_EQUIP     
T92656,10/19/2018 7:11,5653054,208,2,R,26.61,27.45,6.75,691.6,OPTIMESS_DATA,2MTA ,T71614 ,RO_EQUIP     
T7L672,10/19/2018 7:11,5653054,208,3,L,26.58,27.14,6.58,644.4,OPTIMESS_DATA,2CTC ,T7L672 ,BOTH         
T7L672,10/19/2018 7:11,5653054,208,3,R,26.21,27.44,6.17,644.5,OPTIMESS_DATA,2CTC ,T7L672 ,BOTH             
6
  • can you add how is your csv?? Jan 10, 2020 at 19:08
  • how do you display it ? Show part of csv file.
    – furas
    Jan 10, 2020 at 19:11
  • cannot recreate. could you share a sample of the csv? Jan 10, 2020 at 19:47
  • Added a sample of the file.
    – SDHokie
    Jan 10, 2020 at 21:35
  • Try to export this partial data to CSV file, df.to_csv(File name.csv), and check the file, may be the result help to guess the problem.
    – nucsit026
    Jan 11, 2020 at 18:38

2 Answers 2

1

A simple workaround here is to just to take the transpose of the dataframe. Link to Pandas Documentation

df = pd.DataFrame.transpose(df)
3
  • Thank you. A workaround is fine but I am wondering what about my script is making Python import my columns as rows. To note, if I import the entire .csv with the standard pd.read_csv('measurement_file.csv'), the columns are imported properly.
    – SDHokie
    Jan 10, 2020 at 19:57
  • How is the file delimited? I have seen that sometimes the read_csv will read in tab delimited files in strange ways. Jan 10, 2020 at 20:40
  • The file is comma delimited.
    – SDHokie
    Jan 10, 2020 at 21:22
0

Can you try like this?

import pandas as pd

dataset = pd.read_csv('yourfile.csv')

#filterhere
dataset = dataset[cols_to_use]
1
  • Thanks. Unfortunately, it filters down to the columns of interest but puts them into rows.
    – SDHokie
    Jan 10, 2020 at 21:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.