12

I have a problem when I try to remove ^M from a csv

When I type vim or vi file.csv, I get

A, TK,2015-04-06,14.4^M,14.7,10.0,0.0,54.0^M,13.3^M,135.0^M,33.8
B, NV,2015-04-06,14.4^M,14.7,5.4,0.0,47.0^M,14.8^M,97.0^M,31.3

I have tried with

tr -d '^M' < file.csv > file2.csv

But it doesn't remove, also with sed.

4
  • 3
    tr -d '\r' < file.csv > file2.csv? Commented Apr 6, 2015 at 14:46
  • Yes Etan, thanks, now I can remove ^M Commented Apr 6, 2015 at 14:54
  • How did you enter ^M ? Did you use char ^ followed by char M (wrong), or did you use CTRL-V CTRL-M (correct) ?
    – Walter A
    Commented Apr 6, 2015 at 15:25
  • Here is the answer which you are looking for stackoverflow.com/questions/811193/…
    – pkacprzak
    Commented Apr 6, 2015 at 16:07

3 Answers 3

13

You could use dos2unix command which is provided to do that.

Using GNU/sed just for fun :

sed -i -e "s/\r//g" file

Using tr :

tr -d '\r' <file1 >file2
4

You can try:

perl -pE 's/(\^M|\r)//g' < file >file2

should remove

  • the literal ^M - sequence of two characters ^ and M
  • and the ^M as \r character.
2
  • I used perl -pE 's/(\^M|\r)//g' /var/tmp/screwed_csv_file.csv and then i have a correct CSV file.
    – user285594
    Commented Feb 21, 2016 at 16:03
  • After doing that, i can use dev.mysql.com/doc/refman/5.7/en/… and LOAD DATA. Does it not mean MySQL has BUG? they should document it or should be treated as BUG? it took 4 hours for me to find out.
    – user285594
    Commented Feb 21, 2016 at 16:04
2

Where did you get that file from? It looks like an old System X file from a Mac. The old pre-OSX OS used <CR> as line endings. Unix uses <LF>, and Windows/DOS uses <CRLF>.

Do you have dos2unix. This program can convert line endings from Unix/Linux, DOS/Windows, or System X Macs to any of the formats you want. In your file, I take it you need to convert the ^M which are Control Ms and not a Caret-M to NL characters.

1
  • 1
    By the way, to do this you need to put dos2unix into Mac conversion mode: dos2unix -c mac <filename>
    – jonahb
    Commented Feb 15, 2017 at 23:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.