446

I have to convert an entire directory using dos2unix. I am not able to figure out how to do this.

2

11 Answers 11

817

find . -type f -print0 | xargs -0 dos2unix

Will recursively find all files inside current directory and call for these files dos2unix command

18
  • 1
    i gave like this find . -type f -exec dos2unix {} /home/venuk/Desktop/NEO_Src and it gave the error find: missing argument to `-exec'
    – Vivek Gaur
    Commented Aug 13, 2012 at 7:05
  • 2
    Note the backslash that escapes the semi-colon ensures the dos2unix commands are separated by semi-colons so that they don't end up mashed together. If you want to run another command afterwards you'll need another semi-colon, so \;; echo Hello
    – Josh
    Commented May 20, 2014 at 11:43
  • 5
    @vguzmanp True, the find invocation does not do this check (although that would be simple enough to add), but modern dos2unix correctly skips binary files. Commented Jan 7, 2016 at 16:58
  • 3
    Warning: If you run this command in a directory containing a .git index, your git index will be corrupted.
    – mbomb007
    Commented Mar 18, 2020 at 20:34
  • 10
    To avoid .git index corruption and preserve UTF-8 BOM, I've ended up doing find . -type f -print0 | xargs -0 dos2unix -ic0 | xargs -0 dos2unix -b. This command touches only those files (thanks to -ic0 filtering option), that contain windows line breaks, all other files are skipped. -b option is also helpful if your repository contains files with UTF-8 BOM, it preserves the BOM.
    – Konard
    Commented Nov 28, 2020 at 11:03
96

If it's a large directory you may want to consider running with multiple processors:

find . -type f -print0 | xargs -0 -n 1 -P 4 dos2unix 

This will pass 1 file at a time, and use 4 processors.

3
  • 1
    This method has the advantage, that it continues, even though dos2unix encounters any problems! Something like a "--force" method. Thank you for that!
    – freeo
    Commented Oct 10, 2014 at 16:29
  • 6
    Starting a new dos2unix process for each individual file will introduce massively unnecessary overhead. I'd bump that n up by an order of magnitude or two (depending on how many files we're talking about here) Commented Apr 23, 2015 at 12:24
  • 1
    Great answer! @JonoCoetzee, here I use -n 50. Commented Jul 19, 2023 at 19:56
35

A common use case appears to be to standardize line endings for all files committed to a Git repository:

git ls-files -z | xargs -0 dos2unix

Keep in mind that certain files (e.g. *.sln, *.bat) are only used on Windows operating systems and should keep the CRLF ending:

git ls-files -z '*.sln' '*.bat' | xargs -0 unix2dos

If necessary, use .gitattributes

2
  • 1
    I upvoted your answer, but one thing needs to be addressed: it fails on files with spaces in the path, since xargs defaults to having all whitespace be a delimiter. The xargs man page shows a --delimiter option, but suggests using the --null option when input might contain spaces. I have not tested this, but adding --null to yours produces the following: git ls-files | xargs --null dos2unix.
    – Pulseczar
    Commented Jun 22, 2021 at 19:37
  • 1
    Thanks @Pulseczar - Yes, you are correct, the command will fail if a filename contains spaces. Your comments helped me to find the answer (it also requires -z for ls-files). I've updated my answer. Thanks again! Commented Oct 12, 2021 at 0:20
31

As I happened to be poorly satisfied by dos2unix, I rolled out my own simple utility. Apart of a few advantages in speed and predictability, the syntax is also a bit simpler :

endlines unix *

And if you want it to go down into subdirectories (skipping hidden dirs and non-text files) :

endlines unix -r .

endlines is available here https://github.com/mdolidon/endlines

3
  • 1
    This is perfect! The closest one-liner I could get to this is here: unix.stackexchange.com/a/365679/112190
    – phyatt
    Commented May 17, 2017 at 17:33
  • 4
    How does it improve on dos2unix? Genuinely curious.
    – Walf
    Commented Jul 17, 2017 at 5:26
  • 3
    1/ Mainly, there are tons of different dos2unix, with varying capabilities (some read UTF32 for example, while some don't ; endlines does not). There's only one endlines, which capabilities are well known. 2/ liberal on input, not all dos2unix are. 3/ efficient file tree exploration, designed to be fast and practical on tens of thousands of files. 4/ runs out of the box on OSX - which is less important now that Brew package exists. Commented Jul 17, 2017 at 11:04
24

It's probably best to skip hidden files and folders, such as .git. So instead of using find, if your bash version is recent enough or if you're using zsh, just do:

dos2unix **

Note that for Bash, this will require:

shopt -s globstar

....but this is a useful enough feature that you should honestly just put it in your .bashrc anyway.

If you don't want to skip hidden files and folders, but you still don't want to mess with find (and I wouldn't blame you), you can provide a second recursive-glob argument to match only hidden entries:

dos2unix ** **/.*

Note that in both cases, the glob will expand to include directories, so you will see the following warning (potentially many times over): Skipping <dir>, not a regular file.

10
  • 1
    This didn't work for me. Does the globstar syntax work for dos2unix? I've used globstar elsewhere with success, but couldn't get this to work. I'm using Bash 4.3.11(1)
    – dutoitns
    Commented Jun 2, 2016 at 14:25
  • @NSduToit *NIX style shells don't allow executables to alter the behavior of argument-expansion, because (unlike in Windows) argument-expansion is performed before the executable ever receives the arguments. So the only thing I can think of is that you have some kind of dos2unix alias that's affecting how arguments are expanded. What is the output of type dos2unix on your system? Commented Jun 2, 2016 at 15:56
  • 1
    @NSduToit There's some confusion here. My answer explicitly states that the point of using ** instead of find is to "skip hidden files and folders, such as .git". dos2unix never sees the hidden files, because ** does not expand to show them. If you want to automatically run dos2unix on hidden files and folders, you can use find or dos2unix ** **/.* The **/.* will expand only the hidden files and folders, including . (the root dir), .. (the parent dir), and any other hidden entries in the current folder. Commented Jun 2, 2016 at 17:01
  • 1
    (Note that dos2unix simply prints Skipping <dir>, not a regular file. when run on a directory, so running on .. and . is safe.) Additionally, combining ls with a glob is not a good way to check how the glob is expanded; use echo instead: echo ** will print the arguments that dos2unix receives from dos2unix **. Commented Jun 2, 2016 at 17:02
  • 1
    Thank you! Makes sense! Okay, I had a totally wrong idea on what ** implied, and using echo makes it clear.
    – dutoitns
    Commented Jun 2, 2016 at 17:12
7

For any Solaris users (am using 5.10, may apply to newer versions too, as well as other unix systems):

dos2unix doesn't default to overwriting the file, it will just print the updated version to stdout, so you will have to specify the source and target, i.e. the same name twice:

find . -type f -exec dos2unix {} {} \;
7

I've googled this like a million times, so my solution is to just put this bash function in your environment.

.bashrc or .profile or whatever

dos2unixd() {
  find $1 -type f -print0 | xargs -0 dos2unix
}

Usage

$ dos2unixd ./somepath

This way you still have the original command dos2unix and it's easy to remember this one dos2unixd.

6

I think the simplest way is:

dos2unix $(find . -type f)
4

I have had the same problem and thanks to the posts here I have solved it. I knew that I have around a hundred files and I needed to run it for *.js files only. find . -type f -name '*.js' -print0 | xargs -0 dos2unix

Thank you all for your help.

2
for FILE in /var/www/html/files/*
do
 /usr/bin/dos2unix FILE
done
1
  • 2
    Welcome to Stack Overflow. Although your solution is a valid solution, it would be great if you could add some explanation to it. You might also consider to reference other answers to justify your answer. Please have a look at How to Answer for more information.
    – kvantour
    Commented May 14, 2018 at 16:04
-2

If there is no sub-directory, you can also take

ls | xargs -I {} dos2unix "{}"
2
  • 6
    If there are no subdirectories, dos2unix * is simpler and will actually be more robust than this. (It's generally not recommended to pipe the output of ls, because it's a formatting tool and * is more reliable for programmatic usage.) Commented Dec 9, 2016 at 14:55
  • 3
    the reason piping ls is not good
    – phuclv
    Commented May 14, 2018 at 13:55

Not the answer you're looking for? Browse other questions tagged or ask your own question.