329

How can I change the following code to look at all the .log files in the directory and not just the one file?

I need to loop through all the files and delete all lines that do not contain "step4" or "step9". Currently this will create a new file, but I'm not sure how to use the for each loop here (newbie).

The actual files are named like this: 2013 09 03 00_01_29.log. I'd like the output files to either overwrite them, or to have the SAME name, appended with "out".

$In = "C:\Users\gerhardl\Documents\My Received Files\Test_In.log"
$Out = "C:\Users\gerhardl\Documents\My Received Files\Test_Out.log"
$Files = "C:\Users\gerhardl\Documents\My Received Files\"

Get-Content $In | Where-Object {$_ -match 'step4' -or $_ -match 'step9'} | `
Set-Content $Out

4 Answers 4

477

Give this a try:

Get-ChildItem "C:\Users\gerhardl\Documents\My Received Files" -Filter *.log | 
Foreach-Object {
    $content = Get-Content $_.FullName

    #filter and save content to the original file
    $content | Where-Object {$_ -match 'step[49]'} | Set-Content $_.FullName

    #filter and save content to a new file 
    $content | Where-Object {$_ -match 'step[49]'} | Set-Content ($_.BaseName + '_out.log')
}
3
  • thanks. The BaseName part does not work (version 1) - but I managed by first creating a backup of the full file then making the change to the original file. Sep 17, 2013 at 12:25
  • 7
    For v1, you can use the following to extract the base name: [io.path]::GetFileNameWithoutExtension($name)
    – Shay Levy
    Sep 17, 2013 at 12:26
  • 1
    If I replace BaseName with FullName I get what I want.
    – M--
    Oct 31, 2018 at 13:57
134

To get the content of a directory you can use

$files = Get-ChildItem "C:\Users\gerhardl\Documents\My Received Files\"

Then you can loop over this variable as well:

for ($i=0; $i -lt $files.Count; $i++) {
    $outfile = $files[$i].FullName + "out" 
    Get-Content $files[$i].FullName | Where-Object { ($_ -match 'step4' -or $_ -match 'step9') } | Set-Content $outfile
}

An even easier way to put this is the foreach loop (thanks to @Soapy and @MarkSchultheiss):

foreach ($f in $files){
    $outfile = $f.FullName + "out" 
    Get-Content $f.FullName | Where-Object { ($_ -match 'step4' -or $_ -match 'step9') } | Set-Content $outfile
}
8
  • This creates one output file named "out.log". How can I create a separate output file for each input file? Sep 17, 2013 at 10:31
  • @user2725402 I added an input dependend output file that is named <input file name>out
    – PVitt
    Sep 17, 2013 at 10:39
  • 1
    okay 2 new issues... this now for some reason does the opposite of the match, i.e. it creates a file with only records NOT containing "step4" or "step9". Also, it is renaming using "out" as an extension, so files are named "XXX.log.out" - I tried using BaseName but can't get it right. Sep 17, 2013 at 12:01
  • 4
    You can also do foreach ($f in Get-ChildItem "C:\Users\gerhardl\Documents\My Received Files\") { ... }
    – Soapy
    Aug 19, 2015 at 12:55
  • 2
    or use ForEach ($f in $files){...} May 31, 2018 at 13:37
44

If you need to loop inside a directory recursively for a particular kind of file, use the below command, which filters all the files of doc file type

$fileNames = Get-ChildItem -Path $scriptPath -Recurse -Include *.doc

If you need to do the filteration on multiple types, use the below command.

$fileNames = Get-ChildItem -Path $scriptPath -Recurse -Include *.doc,*.pdf

Now $fileNames variable act as an array from which you can loop and apply your business logic.

2
-18

Other answers are great, I just want to add... a different approach usable in PowerShell: Install GNUWin32 utils and use grep to view the lines / redirect the output to file http://gnuwin32.sourceforge.net/

This overwrites the new file every time:

grep "step[49]" logIn.log > logOut.log 

This appends the log output, in case you overwrite the logIn file and want to keep the data:

grep "step[49]" logIn.log >> logOut.log 

Note: to be able to use GNUWin32 utils globally you have to add the bin folder to your system path.

6
  • 27
    This question was tagged as PowerShell, not Unix
    – SteveC
    Sep 21, 2016 at 15:13
  • 2
    Yes. Obviously. My take was fairly simple: You can use GNUWin32 utils in Windows therefore you can use *nix tools in Powershell easily. Works. No one ever said, that the answer had to entirely be native to Powershell. And even if they did, you still can call Win32 Utils directly from Powershell. AFAIK using whatever is available and compatible is not actually a bad thing. Nov 9, 2017 at 22:45
  • 3
    I would have edited my previous comment, but I passed the 5 minutes limit: SteveC: Are you saying that incorporating GNU tools that work fine under Windows is somehow defying Powershell? That's one thing. I understand that working this out purely in Powershell might be more productive in short term. In long term expanding the possibilities has actually more merit. You add more options and are working entirely in Windows and Powershell without any Linux emulators like Cygwin. So while I understand your reservation, I disagree. Nov 9, 2017 at 22:55
  • 8
    The question was about looking in PowerShell, not the multitude of other languages that could be used.
    – SteveC
    Nov 10, 2017 at 7:06
  • 6
    Answers are meant to be to the question. This question was explicit in that it wanted the answer in PowerShell. Your answer was install something and use it. You could have suggested install Ruby, etc.
    – SteveC
    Dec 13, 2019 at 14:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.