11

I have found that many of my files have DOS line endings. In VI they look like this: "^M". I don't want to modify files that don't have these DOS line endings. How do I do this using a bash script? Thanks!

EV

8 Answers 8

19
grep -URl ^M . | xargs fromdos

grep gets you a list of all files under the current directory that have DOS line endings.

-U makes grep consider line endings instead of stripping them away by default

-R makes it recursive

-l makes it list only the filenames and not the matching lines

then you're piping that list into the converter command (which is fromdos on ubuntu, dos2unix where i come from).

NOTE: don't actually type ^M. instead, you'll need to press <Ctrl-V> then <Ctrl-M> to insert the ^M character and make grep understand what you're going for. or, you could type in $'\r' in place of ^M (but i think that may only work for bash...).

4
  • 1
    grep -URl ^M . | xargs fromdos ?
    – Dalinaum
    Mar 7, 2013 at 16:06
  • 5
    The Ubuntu package, for installation, is called "tofrodos".
    – Apalala
    May 7, 2013 at 19:31
  • On tcsh, and probably on csh too, you can get the same effect with grep -URl "\r" . | xargs fromdos.
    – bdesham
    Nov 19, 2013 at 17:03
  • if you need it to work for files with spaces in their names, try grep -URl ^M . | xargs -I{} dos2unix "{}" instead. Jan 21, 2014 at 17:39
5

One way using GNU coreutils:

< file.txt tr -d '\r'
4

On ubuntu, you use the fromdos utility

fromdos test.txt

The above example would take a MS-DOS or Microsoft Windows file or other file with different line separators and format the file with new line separators to be read in Linux and Unix.

1

Many options are there..you can try with any of these.. http://www.theunixschool.com/2011/03/different-ways-to-delete-m-character-in.html

1
cat origin_file.txt | sed "s/^M//" > dest_file.txt

You have to do the same thing mentioned above, ctl-V then ctl-M to get that character. This is preferable for me because it is portable across many platforms and keeps it simple within bash.

on ubuntu I also find this works:

cat origin_file.txt | sed "s/\r//" > dest_file.txt

0

you can use the command:

   dos2ux file.in>file.out or:

in perl:

perl -pi -e 's/\r//g' your_file

alternatively you can do:

  • open in vi
  • go to command mode
  • type :%s/[ctrl-V][CTRL-M]//g
0

Note if you're converting multi-byte files you need to take extra care, and should probably try to use the correct iconv or recode from-encoding specifications.

If it's a plain ASCII file, both of the below methods would work.

The flip program, in Debian the package is also called flip, can handle line-endings. From the manual:

When asked to convert a file to the same format that  it already 
has, flip  causes  no change to the file. Thus to convert all
files to **IX format you can type

flip -u *

and all files will end up right, regardless of whether they were 
in MS-DOS or in **IX format to begin with. This also works in the
opposite direction.

Or you could use GNU recode:

< /etc/passwd recode ..pc | tee a b > /dev/null
file a b

Output:

a: ASCII text, with CRLF line terminators
b: ASCII text, with CRLF line terminators

Convert to unix line-endings:

recode pc.. a b
file a b

Output:

a: ASCII text
b: ASCII text

recode abbreviates dos line-endings as pc, so the logic with pc.. is: convert from pc format to the default, which is latin1 with unix line-endings.

0

A modification to the Winning answer if you need to filter by file ending

grep -URl ^M . | grep .php | xargs dos2unix

I used dos2unix instead of fromdos but the effect should be the same.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.