3

In PowerShell, it appears that the order of execution of cmdlets in a pipeline aren't executed in an obvious way. Rather than each cmdlet executing then passing the results to the next cmdlet in the pipeline, it seems that individual output objects of a cmdlet are passed to the input of the next cmdlet before the execution of the previous cmdlet complets. The following confirms this behavior:

1..5 | %{ Write-Host $_; $_ } | %{ Write-Host ([char]($_ + 64)) }

prints

1
A
2
B
3
C
4
D
5
E

It appears what's happening is that each execution of the ForEach-Object cmdlet will execute its script block and each subsequent command in its pipeline before iterating.

Is this what actually occurs, and is this behavior documented anywhere? Is this the case for all iterative cmdlets like ForEach-Object (e.g. Where-Object, etc.)?

I know I can wrap pieces in an expression ((1..5 | %{ Write-Host $_; $_ }) | %{ Write-Host ([char]($_ + 64)) }) or assign a piece to a variable and then pipe that to subsequent pipeline commands to avoid this behavior, but is there a way to perform an operation on each element of a collection and then pass that entire collection on to the next command in a pipeline?

2 Answers 2

4

This matches my understanding of how pipelines process objects. Objects travel "as far" in the pipeline as possible before the next item is touched. Some cmdlets don't use the "process" block but instead use the "end" block out of necessity (for example, sort-object has to have all of the items to actually perform the sort) so they block the pipeline. Others use write-output (which your second $_ is doing implicitly) and keep the pipeline moving.

5
  • Agreed with this; the output in the question is intuitive to myself. Notice how your output changes when you do (1..5 | %{ Write-Host $_; $_ }) | %{ Write-Host ([char]($_ + 64))}. This will cause everything in () to be executed first, and then each one of those items is handed to the outer pipeline.
    – SpellingD
    Commented Jul 27, 2012 at 20:42
  • I remember a video (Channel 9?) with Jeffrey Snover and Eric Meijer talking about PowerShell and Monads where Snover drew a diagram of how stuff is pushed down the pipeline. I'll see if I can find a link. Commented Jul 27, 2012 at 20:46
  • 1
    It wasn't very hard to find. Here's the link Commented Jul 27, 2012 at 20:48
  • This matches my experience and testing as well. Have you been able to find any documentation of this behavior, and how to know whether a command in a pipeline will block the pipeline or not? Commented Jul 30, 2012 at 14:23
  • 1
    Bruce Payette discusses this in PowerShell in Action 2nd Edition in a section called "Pipelines and streaming behavior" (section 2.5.1, p. 56-57). If you want to understand the mechanics of PowerShell, I can't recommend any book more highly. Commented Jul 31, 2012 at 2:04
1

Each object travels the pipeline as far as it can. That's why every cmdlet should write its output directly to the pipeline rather than accumulate it in some internal storage, except if it has to process all its input together like the Sort-Object.

It also has some interesting side effects. Eg the Get-Member cmdlet gives different output if it gets input from the pipeline and different if it gets its input from InputObject parameter.

And, because IT IS PowerShell, it's all documented here:

Get-Help about_pipelines
2
  • Is that supposed to be a link at the end? It doesn't work.
    – Tom Zych
    Commented Jan 21, 2017 at 21:58
  • Nope. It's a PowerShell command: Get-Help about_pipelines If you haven't updated the help locally, use the Get-Help about_pipelines -Online or search for the "about_pipelines"
    – Thanassis
    Commented Jan 21, 2017 at 23:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.