I have a script that is appending new fields to an existing CSV, however ^M
characters are appearing at the end of the old lines so the new fields end up on a new row instead of the same one. How do I remove ^M
characters from a CSV file using Perl?
11 Answers
Or a 1-liner:
perl -p -i -e 's/\r\n$/\n/g' file1.txt file2.txt ... filen.txt
-
3
-
On windows passing *.txt with this command does not work. It gives: Can't open *.txt: Invalid argument. Anyone?– mgouinDec 10, 2015 at 19:26
-
You found out you can also do this:
$line=~ tr/\015//d;
-
1not as readable as
\r
- anyone looking at that (or yourself in a year's time) would be glad of a comment stating what it does– plusplusJun 21, 2011 at 8:12
Slightly unrelated, but to remove ^M from the command line using Perl, do this:
perl -p -i -e "s/\r\n/\n/g" file.name
I prefer a more general solution that will work with either DOS or Unix input. Assuming the input is from STDIN:
while (defined(my $ln = <>))
{
chomp($ln);
chop($ln) if ($ln =~ m/\r$/);
# filter and write
}
This one liner replaces all the ^M characters:
dos2unix <file-name>
You can call this from inside Perl or directly on your Unix prompt.
To convert DOS style to UNIX style line endings:
for ($line in <FILEHANDLE>) {
$line =~ s/\r\n$/\n/;
}
Or, to remove UNIX and/or DOS style line endings:
for ($line in <FILEHANDLE>) {
$line =~ s/\r?\n$//;
}
-
-
I guess that depends on your goal. I edited to show both strategies.– spoulsonMar 16, 2009 at 15:23
This is what solved my problem. ^M is a carriage return, and it can be easily avoided in a Perl script.
while(<INPUTFILE>)
{
chomp;
chop($_) if ($_ =~ m/\r$/);
}
-
Does that remove ^M from a CSV file? Changing the input file? Does it create some output file that will not have them? Dec 7, 2017 at 0:23
Little script I have for that. A modification of it helped to filter out some other non-printable characters in cross-platform legacy files.
#!/usr/bin/perl
# run this as
# convert_dos2unix.pl < input_file > output_file
undef $/;
$_ = <>;
s/\r//ge;
print;
perl command to convert dos line ending to unix line ending with backup of the original file:
perl -pi.bak -e 's/\r\n/\n/g' filename
This command generates filename with unix line ending and leaves the original file as filename.bak.
In vi hit :
.
Then s/Control-VControl-M//g
.
Control-V
Control-M
are obviously those keys. Don't spell it out.
-
1It's a bad idea to include non-printing characters like carriage return verbatim in source code like this. Far better to use the \r escape that is (a) easy to see and (b) won't get lost if the source is reformatted. Dec 2, 2015 at 17:16
binmode(STDIN, ":crlf")
orPERLIO=:unix:crlf
(see [stackoverflow.com/a/21320709/424632]).