38

Is there a way (say PowerShell, or a tool) in Windows that can recurse over a directory and convert any Unix files to Windows files.

I'd be perfectly happy with a way in PowerShell to at least detect a Unix file.

It's easy do this for one single file, but I'm after something a bit more scalable (hence leaning towards a PowerShellish solution).

2

11 Answers 11

47

Here is the pure PowerShell way if you are interested.

Finding files with at least one Unix line ending (PowerShell v1):

dir * -inc *.txt | %{ if (gc $_.FullName -delim "`0" | Select-String "[^`r]`n") {$_} }

Here is how you find and covert Unix line endings to Windows line endings. One important thing to note is that an extra line ending (\r\n) will be added to the end of the file if there isn't already a line ending at the end. If you really don't want that, I'll post an example of how you can avoid it (it is a bit more complex).

Get-ChildItem * -Include *.txt | ForEach-Object {
    ## If contains UNIX line endings, replace with Windows line endings
    if (Get-Content $_.FullName -Delimiter "`0" | Select-String "[^`r]`n")
    {
        $content = Get-Content $_.FullName
        $content | Set-Content $_.FullName
    }
}

The above works because PowerShell will automatically split the contents on \n (dropping \r if they exist) and then add \r\n when it writes each thing (in this case a line) to the file. That is why you always end up with a line ending at the end of the file.

Also, I wrote the above code so that it only modifies files that it needs to. If you don't care about that you can remove the if statement. Oh, make sure that only files get to the ForEach-Object. Other than that, you can do whatever filtering you want at the start of that pipeline.

3
  • 1
    By default PowerShell works in "Unicode". I'm no expert on text encoding, but I haven't run into problems with the defaults yet. If you wish, you can explicitly set an encoding for the Get-Content and Set-Content commands with the -Encoding parameter. Get-Help Get-Content -Parameter Encoding Apr 8, 2009 at 23:31
  • @PeterSeale Set-Content or Out-File have an -Encoding parameter that can be used to set the file encoding type. Jun 26, 2014 at 14:00
  • 1
    Might also want to add the -recurse flag to search nested folders as well. I.e. Get-ChildItem -Path .\ -recurse | ForEach-Object ...
    – Mark
    Feb 19, 2021 at 0:21
15

This seems to work for me.

Get-Content Unix.txt | Out-File Dos.txt
4
  • 4
    or Get-Content Unix.txt | Set-Content Dos.txt
    – js2010
    Aug 28, 2018 at 15:56
  • Out-File outputs utf-16 or "unicode" text. I would recommend Set-Content instead in PS 5.
    – js2010
    Sep 2, 2018 at 13:59
  • This gives me an empty file when the source is the same as destination. I had to go through a different file
    – Joffrey
    Oct 4, 2018 at 14:03
  • 2
    @Joffrey (Get-Content file.txt) | Set-Content file.txt
    – js2010
    Jun 27, 2019 at 16:32
14

There is dos2unix and unix2dos in Cygwin.

6
  • 2
    I'd recommend this technique as the unix utils will do a better job maintaining the original file encoding (UTF-8, ASCII, etc). I've had problems with PS in the past when I intended to keep ASCII files ASCII. Apr 8, 2009 at 19:56
  • or msys and then you can use the utilities from cmd.
    – Pod
    Jun 24, 2009 at 12:12
  • 3
    I use powershell to list the files and then pipe it to dos2unix.exe like this: dir -Recurse -File -Exclude .git | % { dos2unix --u2d --skipbin $_ }
    – orad
    Feb 7, 2014 at 3:42
  • @orad if you are already using the unix command dos2unix why not use unix find as well? Feb 8, 2014 at 9:55
  • Or download Win32 binary here and run it directly in cmd.exe/PowerShell! Mar 14, 2018 at 20:40
7

Download Vim, open your file, and issue

:se fileformat=dos|up

Batch for multiple files (all *.txt files in C:\tmp - recursive):

:args C:\tmp\**\*.txt
:argdo se fileformat=dos|up
3
  • can you do this for a folder or as a batch job?
    – ninesided
    Apr 7, 2009 at 4:48
  • Or download Eclipse, open the file and convert line delimeters to Unix. vim is no doubt a great tool and I use it every day. But don't you think it is a bit of overkill to use for converting endofline? Apr 7, 2009 at 13:58
  • 4
    It's just the first thing that came to my mind, it's on every box I own/administer. Btw: are you actually suggesting using eclipse (85MB) and doing it file-by-file instead of using vim (8.5MB) and doing it all at once?
    – soulmerge
    Apr 7, 2009 at 14:44
2

You can use Visual Studio. Menu FileAdvanced Save Options....

2

Converting to Windows text could be as simple as:

(Get-Content file) | Set-Content file

Use the following (with negative lookbehind). Without -nonewline, set-content puts an extra `r`n at the bottom. With the parentheses, you can modify the same file. This should be safe on doing to the same file twice accidentally.

function unix2dos ($infile, $outfile) {
    (Get-Content -raw $infile) -replace "(?<!`r)`n","`r`n" |
    Set-Content -nonewline $outfile
}

The reverse would be this, Windows to Unix text:

function dos2unix ($infile, $outfile) {
    (Get-Content -raw $infile) -replace "`r`n","`n" |
    Set-Content -nonewline $outfile
}

Here's another version for use with huge files that can't fit in memory. But the output file has to be different.

Function Dos2Unix ($infile, $outfile) {
  Get-Content $infile -ReadCount 1000 | % { $_ -replace '$',"`n" } |
  Set-Content -NoNewline $outfile
}

Examples (input and output file can be the same):

dos2unix dos.txt unix.txt
unix2dos unix.txt dos.txt
unix2dos file.txt file.txt

If you have Emacs, you can check it with esc-x hexl-mode. Notepad won't display Unix text correctly; it will all be on the same line. I have to specify the path for set-content, because -replace erases the pspath property.

0
1

If Cygwin isn't for you, there are numerous stand-alone executables for unix2dos under Windows if you google around, or you could write one yourself. See my similar (opposite direction for conversion) question here.

1

The result of testing the code given in a previous answer in a loop with 10,000 files, many of them more than 50 KB in size:

The bottom line is the PowerShell code is very inefficient/slow/unusable for large files and large number of files. It also does not preserve BOM bytes. I found unix2dos 7.2.3 to be the fastest and most practical solution.

2
  • This has to do with the fact that Get-Content is very slow (and inefficient), especially on larger files.
    – bluuf
    Jan 4, 2018 at 11:47
  • This ought to be converted to a comment (for the appropriate or closest matching answer) - this does not answer the question. Apr 8, 2021 at 13:00
0

It works for me:

 Get-ChildItem -Recurse -File | % { $tmp = Get-Content $_; $tmp | Out-File "$_" -Encoding UTF8 }
0

Building on js2010's answer I've created this script:

$excludeFolders = "node_modules|dist|.vs";
$excludeFiles = ".*\.map.*|.*\.zip|.*\.png|.*\.ps1"

Function Dos2Unix {
    [CmdletBinding()]
    Param([Parameter(ValueFromPipeline)] $fileName)

    Write-Host -Nonewline "."

    $fileContents = Get-Content -raw $fileName
    $containsCrLf = $fileContents | %{$_ -match "\r\n"}
    If($containsCrLf -contains $true)
    {
        Write-Host "`r`nCleaing file: $fileName"
        Set-Content -Nonewline -Encoding utf8 $fileName ($fileContents -replace "`r`n","`n")
    }
}

Get-Childitem -File "." -Recurse |
  Where-Object {$_.PSParentPath -notmatch $excludeFolders} |
  Where-Object {$_.PSPath -notmatch $excludeFiles} |
  foreach { $_.PSPath | Dos2Unix }
1
  • Why is there a semicolon in the first line of the script? Apr 8, 2021 at 13:24
-1

Opening a file with Unix line endings in WordPad and saving it will rewrite all the line endings as DOS. It is a bit laborious for large numbers of files, but it works well enough for a few files every once in a while.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.