8

I'm writing a script to download several repositories from GitHub. Here is the command to download a repository:

git clone "$RepositoryUrl" "$localRepoDirectory"

When I run this command it displays some nice progress information in the console window that I want displayed.

The problem is that I also want to be able to detect if any errors have occurred while downloading. I found this post that talks about redirecting the various streams, so I tried:

(git clone "$RepositoryUrl" "$localRepoDirectory") 2> $errorLogFilePath

This pipes any errors from stderr to my file, but no longer displays the nice progress information in the console.

I can use the Tee-Object like so:

(git clone "$RepositoryUrl" "$localRepoDirectory") | Tee-Object -FilePath $errorLogFilePath

and I still get the nice progress output, but this pipes stdout to the file, not stderr; I'm only concerned with detecting errors.

Is there a way that I can store any errors that occur in a file or (preferably) a variable, while also still having the progress information piped to the console window? I have a feeling that the answer might lie in redirecting various streams into other streams as discusses in this post, but I'm not really sure.

======== Update =======

I'm not sure if the git.exe is different than your typical executable, but I've done some more testing and here is what I've found:

$output = (git clone "$RepositoryUrl" "$localRepoDirectory")

$output always contains the text "Cloning into '[localRepoDirectory]'...", whether the command completed successfully or produced an error. Also, the progress information is still written to the console when doing this. This leads me to think that the progress information is not written via stdout, but by some other stream?

If an error occurs the error is written to the console, but in the usual white foreground color, not the typical red for errors and yellow for warnings. When this is called from within a cmdlet function and the command fails with an error, the error is NOT returned via the function's -ErrorVariable (or -WarningVariable) parameter (however if I do my own Write-Error that does get returned via -ErrorVariable). This leads me to think that git.exe doesn't write to stderr, but when we do:

(git clone "$RepositoryUrl" "$localRepoDirectory") 2> $errorLogFilePath

The error message is written to the file, so that makes me think that it does write to stderr. So now I'm confused...

======== Update 2 =======

So with Byron's help I've tried a couple of more solutions using a new process, but still can't get what I want. When using a new process I never get the nice progress written to the console.

The three new methods that I've tried both use this bit of code in common:

$process = New-Object System.Diagnostics.Process
$process.StartInfo.Arguments = "clone ""$RepositoryUrl"" ""$localRepoDirectory"""
$process.StartInfo.UseShellExecute = $false
$process.StartInfo.RedirectStandardOutput = $true
$process.StartInfo.RedirectStandardError = $true
$process.StartInfo.CreateNoWindow = $true
$process.StartInfo.WorkingDirectory = $WORKING_DIRECTORY
$process.StartInfo.FileName = "git"

Method 1 - Run in new process and read output afterwards:

$process.Start()
$process.WaitForExit()
Write-Host Output - $process.StandardOutput.ReadToEnd()    
Write-Host Errors - $process.StandardError.ReadToEnd()

Method 2 - Get output synchronously:

$process.Start()
while (!$process.HasExited)
{
    Write-Host Output - $process.StandardOutput.ReadToEnd()
    Write-Host Error Output - $process.StandardError.ReadToEnd()

    Start-Sleep -Seconds 1
}

Even though this looks like it would write the output while the process is running, it doesn't write anything until after the process exits.

Method 3 - Get output asynchronously:

Register-ObjectEvent -InputObject $process -EventName "OutputDataReceived" -Action {Write-Host Output Data - $args[1].Data }
Register-ObjectEvent -InputObject $process -EventName "ErrorDataReceived" -Action { Write-Host Error Data - $args[1].Data }
$process.Start()
$process.BeginOutputReadLine()
$process.BeginErrorReadLine()
while (!$process.HasExited)
{
    Start-Sleep -Seconds 1
}

This does output data while the process is working which is good, but it still doesn't display the nice progress information :(

3 Answers 3

2

I think I have your answer. I'm working with PowerShell for a while and created several build systems. Sorry if the script is a bit long, but it works.

$dir = <your dir>
$global:log = <your log file which must be in the global scope> # Not global = won't work

function Create-Process {
    $process = New-Object -TypeName System.Diagnostics.Process
    $process.StartInfo.CreateNoWindow = $false
    $process.StartInfo.RedirectStandardError = $true
    $process.StartInfo.UseShellExecute = $false
    return $process
}

function Terminate-Process {
    param([System.Diagnostics.Process]$process)

    $code = $process.ExitCode
    $process.Close()
    $process.Dispose()
    Remove-Variable process
    return $code
}

function Launch-Process {
    param([System.Diagnostics.Process]$process, [string]$log, [int]$timeout = 0)

    $errorjob = Register-ObjectEvent -InputObject $process -EventName ErrorDataReceived -SourceIdentifier Common.LaunchProcess.Error -action {
        if(-not [string]::IsNullOrEmpty($EventArgs.data)) {
            "ERROR - $($EventArgs.data)" | Out-File $log -Encoding ASCII -Append
            Write-Host "ERROR - $($EventArgs.data)"
        }
    }
    $outputjob = Register-ObjectEvent -InputObject $process -EventName OutputDataReceived -SourceIdentifier Common.LaunchProcess.Output -action {
        if(-not [string]::IsNullOrEmpty($EventArgs.data)) {
            "Out - $($EventArgs.data)" | Out-File $log -Encoding ASCII -Append
            Write-Host "Out - $($EventArgs.data)"
        }
    }

    if($errorjob -eq $null) {
        "ERROR - The error job is null" | Out-File $log -Encoding ASCII -Append
        Write-Host "ERROR - The error job is null"
    }

    if($outputjob -eq $null) {
        "ERROR - The output job is null" | Out-File $log -Encoding ASCII -Append
        Write-Host "ERROR - The output job is null"
    }

    $process.Start() 
    $process.BeginErrorReadLine()

    if($process.StartInfo.RedirectStandardOutput) {
        $process.BeginOutputReadLine() 
    }

    $ret = $null
    if($timeout -eq 0)
    {
        $process.WaitForExit()
        $ret = $true
    }
    else
    {
        if(-not($process.WaitForExit($timeout)))
        {
            Write-Host "ERROR - The process is not completed, after the specified timeout: $($timeout)"
            $ret = $false
        }
        else
        {
            $ret = $true
        }
    }

    # Cancel the event registrations
    Remove-Event * -ErrorAction SilentlyContinue
    Unregister-Event -SourceIdentifier Common.LaunchProcess.Error
    Unregister-Event -SourceIdentifier Common.LaunchProcess.Output
    Stop-Job $errorjob.Id
    Remove-Job $errorjob.Id
    Stop-Job $outputjob.Id
    Remove-Job $outputjob.Id

    $ret
}

$repo = <your repo>

$process = Create-Process
$process.StartInfo.RedirectStandardOutput = $true
$process.StartInfo.FileName = "git.exe"
$process.StartInfo.Arguments = "clone $($repo)"
$process.StartInfo.WorkingDirectory = $dir
Launch-Process $process $global:log
Terminate-Process $process

The log file must be in the global scope because the routine which runs the event processing is not in the script scope.

Sample of my log file:

Out - Cloning into ''... ERROR - Checking out files: 22% (666/2971)
ERROR - Checking out files: 23% (684/2971)
ERROR - Checking out files: 24% (714/2971)

2
  • Hi romu, thanks for the code! This still doesn't give me quite what I want though. This does write everything (including the progress) to the log file as well as the console, but it doesn't display the progress in the console in real-time; it waits for the download to complete and then displays all of it at once. So if it takes 10 minutes to download the repo, I just see the flashing cursor for 10 minutes in the console, then when it finishes it displays all of the progress info :(
    – deadlydog
    Aug 8, 2013 at 18:52
  • Ok, sorry to not be able to help more.
    – romu
    Aug 13, 2013 at 12:02
1

You can do this by putting the git clone command inside an advanced function e.g.:

function Clone-Git {
    [CmdletBinding()]
    param($repoUrl, $localRepoDir)

    git clone $repoUrl $localRepoDir
}

Clone-Git $RepositoryUrl $localRepoDirectory -ev cloneErrors

$cloneErrors
4
  • 2
    I was very excited about this answer, as it seems pretty straight forward. Unfortunately it doesn't seem to work for my case. The git.exe doesn't seem to use stderr, as the $cloneErrors variable is always null. However it does write the error message to the console, but in the regular white foreground color, not the typical red for errors or yellow for warnings. However, using "2>" does pipe the error message to a file, so that makes it seem like it is using stderr. I'm confused. If I do my own Write-Error from the Clone-Git function the $cloneErrors variable shows my error. Any other tips?
    – deadlydog
    May 10, 2013 at 17:29
  • It sounds like git.exe is using stderr. IIRC this would work in ISE but not in the console. Doh! PowerShell.exe (as opposed to ISE) gives console exe's a handle write to so when git.exe outputs to stderr it is bypassing PowerShell's error stream. What you could do is redirect stderr to a file in the function and then if $LastExitCode is not equal to 0 (right after git clone call), Write-Error the contents of that file.
    – Keith Hill
    May 10, 2013 at 23:17
  • How should I redirect the stderr to a file while still getting the progress info to display to the console? Using "git clone $repoUrl $localRepoDirectory 2> C:\GitHubErrors.log" stops the progress info from being displayed.
    – deadlydog
    May 13, 2013 at 16:26
  • If you use a Diagnostics.Process you can choose independently to redirect stderr and stdout
    – Byron Ross
    Jun 14, 2013 at 6:03
0

If you use System.Diagnostics.Process to start Git, you can redirect all the error and output.

I just had to solve this problem for Inkscape:

$si = New-Object System.Diagnostics.ProcessStartInfo
$si.Arguments = YOUR PROCESS ARGS
$si.UseShellExecute = $false
$si.RedirectStandardOutput = $true
$si.RedirectStandardError = $true
$si.WorkingDirectory = $workingDir
$si.FileName = EXECUTABLE LOCATION
$process = [Diagnostics.Process]::Start($si)
while (!($process.HasExited))
{
    // Do what you want with strerr and stdout
    Start-Sleep -s 1  // Sleep for 1 second
}

You can, of course, wrap this in a function with proper arguments...

6
  • Looks like this doesn't work either; unless I'm doing something wrong. After $process.WaitForExit() I'm able to get the generated output via $process.StandardOutput.ReadToEnd() and StandardError.ReadToEnd(), but then once again I have no progress displayed while the operation is running; I get the information after the operation is complete. Also, it looks like the progress info is displayed via the StandardError.
    – deadlydog
    Jul 3, 2013 at 23:34
  • You may be able to wrap $process in a while loop and read the output as you go. - see updated answer
    – Byron Ross
    Jul 4, 2013 at 3:07
  • Thanks for the update Byron, but it still does not seem to work properly. Using StandardOutput.ReadToEnd() still does not display until after the process has exited for some reason, and StandardOutput.ReadToEnd() just returns an empty string this way :S I also tried registering an event handler to the process's OutputDataReceived and ErrorDataReceived events instead. This method did return data back while the process was running, but again the error data was always empty for some reason.
    – deadlydog
    Jul 5, 2013 at 0:04
  • You may need to create a thread for each output if you want to see them in 'real time'. See msdn.microsoft.com/en-us/library/…
    – Byron Ross
    Jul 5, 2013 at 4:23
  • Yeah, I forgot to mention it, but when using the event handlers I was using .BeginOutputReadLine() and .BeginErrorReadLine(), rather than .ReadToEnd(). Thanks for your persistence on this one with me! :)
    – deadlydog
    Jul 5, 2013 at 19:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.