134

I try to read the file into pandas. The file has values separated by space, but with different number of spaces I tried:

pd.read_csv('file.csv', delimiter=' ')

but it doesn't work

2

5 Answers 5

224

add delim_whitespace=True argument, it's faster than regex.

3
  • 4
    should add that, and remove delimiter=' ' as they are mutually exclusive in recent versions.
    – matanster
    Aug 8, 2018 at 13:05
  • 9
    @matanster: delimiter=' ' is very brittle, it says to expect one and only one space. No tabs, newsline, multiple spaces, nonbreaking whitespaces, combination of these etc. delimiter='\s+' is what pandas recommends and is more robust.
    – smci
    Jan 16, 2020 at 12:28
  • sep="\s+" argument also works
    – PeJota
    May 26, 2022 at 13:55
48

you can use regex as the delimiter:

pd.read_csv("whitespace.csv", header=None, delimiter=r"\s+")
3
  • 5
    This helps when you have more than just a space as delimiter. In current versions one should add engine = "python" to avoid a warning. Mar 20, 2018 at 9:45
  • 2
    Sorry for commenting old reply here, what does r before "\s+" mean?
    – AlphaF20
    Sep 1, 2021 at 4:23
  • 1
    @AlphaF20 it means read as raw string literal: stackoverflow.com/questions/2081640/…
    – PeJota
    May 26, 2022 at 13:57
2

Pandas read_fwf for the win:

import pandas as pd

df = pd.read_fwf(file_path)
1

You can pass a regular expression as a delimiter for read_table also, and it is fast :).

result = pd.read_table('file', sep='\s+')
0

If you can't get text parsing to work using the accepted answer (e.g if your text file contains non uniform rows) then it's worth trying with Python's csv library - here's an example using a user defined Dialect:

 import csv

 csv.register_dialect('skip_space', skipinitialspace=True)
 with open(my_file, 'r') as f:
      reader=csv.reader(f , delimiter=' ', dialect='skip_space')
      for item in reader:
          print(item)
2
  • That's not true. It works with python 3.8 and pandas. The question asks for reading a text file in pandas.
    – Spas
    Nov 30, 2020 at 10:40
  • Ah sorry - I have updated my answer to account for this.
    – Pierz
    Dec 9, 2020 at 18:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.