2

I found online the following code to download all the pdfs on a webpage:

$psPage = Invoke-WebRequest "https://www.pi.infn.it/~rizzo/ingegneria/appunti_fisII_ing_mecc/
"
$urls = $psPage.ParsedHtml.getElementsByTagName("A") | ? {$_.href -like "*.pdf"} | Select-Object -ExpandProperty href

$urls | ForEach-Object {Invoke-WebRequest -Uri $_ -OutFile ($_ | Split-Path -Leaf)}

but PS gave me the error:

Invoke-WebRequest : The response content cannot be parsed because the Internet Explorer engine is not
available, or Internet Explorer's first-launch configuration is not complete. Specify the UseBasicParsing
parameter and try again.
At C:\Users\Raffaele\Desktop\Nuova cartella\a.ps1:1 char:11
+ $psPage = Invoke-WebRequest "https://www.pi.infn.it/~rizzo/ingegneria ...
+           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotImplemented: (:) [Invoke-WebRequest], NotSupportedException
    + FullyQualifiedErrorId : WebCmdletIEDomNotSupportedException,Microsoft.PowerShell.Commands.InvokeWebReq
   uestCommand

You cannot call a method on a null-valued expression.
At C:\Users\Raffaele\Desktop\Nuova cartella\a.ps1:2 char:1
+ $urls = $psPage.ParsedHtml.getElementsByTagName("A") | ? {$_.href -li ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : InvokeMethodOnNull

Split-Path : Cannot bind argument to parameter 'Path' because it is null.
At C:\Users\Raffaele\Desktop\Nuova cartella\a.ps1:4 char:66
+ ... h-Object {Invoke-WebRequest -Uri $_ -OutFile ($_ | Split-Path -Leaf)}
+                                                        ~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (:) [Split-Path], ParameterBindingValidationException
    + FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.S
   plitPathCommand

Invoke-WebRequest : Cannot validate argument on parameter 'Uri'. The argument is null or empty. Provide an
argument that is not null or empty, and then try the command again.
At C:\Users\Raffaele\Desktop\Nuova cartella\a.ps1:4 char:48
+ $urls | ForEach-Object {Invoke-WebRequest -Uri $_ -OutFile ($_ | Spli ...
+                                                ~~
    + CategoryInfo          : InvalidData: (:) [Invoke-WebRequest], ParameterBindingValidationException
    + FullyQualifiedErrorId : ParameterArgumentValidationError,Microsoft.PowerShell.Commands.InvokeWebReques
   tCommand

How can I overcome this problem? Why it requires internet explorer engine?


EDIT: I tried to modify the code in this way:

$site = "https://www.pi.infn.it/~rizzo/ingegneria/appunti_fisII_ing_mecc/"
$psPage = Invoke-WebRequest -Uri $site -UseBasicParsing
$urls = $psPage.ParsedHtml.getElementsByTagName("A")
$urls |  where {$_.pathname -like "*pdf"} | % {Invoke-WebRequest -Uri "$site$($_.pathname)" -OutFile $_.pathname }

and the error is:

You cannot call a method on a null-valued expression.
At C:\Users\Raffaele\Desktop\Nuova cartella\a.ps1:3 char:1
+ $urls = $psPage.ParsedHtml.getElementsByTagName("A")
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : InvokeMethodOnNull

EDIT 2 I tried to modify the code in this way. New code:

$psPage = Invoke-WebRequest -Uri -UseBasicParsing "https://www.pi.infn.it/~rizzo/ingegneria/appunti_fisII_ing_mecc/"
$urls = $psPage.ParsedHtml.getElementsByTagName("A") | ? {$_.href -like "*.pdf"} | Select-Object -ExpandProperty href
$urls | ForEach-Object {Invoke-WebRequest -Uri $_ -OutFile ($_ | Split-Path -Leaf)}

The Windows PowerShell gave me a new error:

Invoke-WebRequest : Missing an argument for parameter 'Uri'. Specify a parameter of type 'System.Uri' and
try again.
At C:\Users\Raffaele\Desktop\Nuova cartella\b.ps1:1 char:29
+ $psPage = Invoke-WebRequest -Uri -UseBasicParsing "https://www.pi.inf ...
+                             ~~~~
    + CategoryInfo          : InvalidArgument: (:) [Invoke-WebRequest], ParameterBindingException
    + FullyQualifiedErrorId : MissingArgument,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

You cannot call a method on a null-valued expression.
At C:\Users\Raffaele\Desktop\Nuova cartella\b.ps1:2 char:1
+ $urls = $psPage.ParsedHtml.getElementsByTagName("A") | ? {$_.href -li ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : InvokeMethodOnNull

Split-Path : Cannot bind argument to parameter 'Path' because it is null.
At C:\Users\Raffaele\Desktop\Nuova cartella\b.ps1:3 char:66
+ ... h-Object {Invoke-WebRequest -Uri $_ -OutFile ($_ | Split-Path -Leaf)}
+                                                        ~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (:) [Split-Path], ParameterBindingValidationException
    + FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.S
   plitPathCommand

Invoke-WebRequest : Cannot validate argument on parameter 'Uri'. The argument is null or empty. Provide an
argument that is not null or empty, and then try the command again.
At C:\Users\Raffaele\Desktop\Nuova cartella\b.ps1:3 char:48
+ $urls | ForEach-Object {Invoke-WebRequest -Uri $_ -OutFile ($_ | Spli ...
+                                                ~~
    + CategoryInfo          : InvalidData: (:) [Invoke-WebRequest], ParameterBindingValidationException
    + FullyQualifiedErrorId : ParameterArgumentValidationError,Microsoft.PowerShell.Commands.InvokeWebReques
   tCommand
5
  • Does this answer your question? The response content cannot be parsed because the Internet Explorer engine is not available, or Commented Mar 1, 2021 at 23:23
  • The answer is in the error message: either complete the first launch of IE OR use the -UseBasicParsing parameter.
    – bluuf
    Commented Mar 2, 2021 at 7:37
  • Hi @DougMaurer I added -UseBasicParsing, but I still have error (fortunately another kind of error) Commented Mar 2, 2021 at 8:59
  • I suggest you read the error messages, the new error message you recieve is clear (hint: you did -usebasicparsing but after that you forgot the space between that and the url)
    – bluuf
    Commented Mar 2, 2021 at 11:43
  • @bluuf thank you for the suggestion. I edited the code and wrote the "new" error. Commented Mar 2, 2021 at 11:46

1 Answer 1

1

In your way : First you have to remove "about:" in your URL or replace it by nothing :

$site = "https://www.pi.infn.it/~rizzo/ingegneria/appunti_fisII_ing_mecc/"
$psPage = Invoke-WebRequest $site
$urls = $psPage.ParsedHtml.getElementsByTagName("A") | ? {$_.href -like "*.pdf"} | Select-Object -ExpandProperty href | ForEach-Object {$_.replace("about:", "")}

Second you have to recreate full URL :

$urls | ForEach-Object {Invoke-WebRequest -Uri "$site$_" -OutFile $_ }

But you can simplify using "textcontent" or "pathname" :

$site = "https://www.pi.infn.it/~rizzo/ingegneria/appunti_fisII_ing_mecc/"
$psPage = Invoke-WebRequest $site
$urls = $psPage.ParsedHtml.getElementsByTagName("A")
$urls |  where {$_.pathname -like "*pdf"} | % {Invoke-WebRequest -Uri "$site$($_.pathname)" -OutFile $_.pathname }
9
  • Have you try mine ? Your URI is not complet.
    – JPBlanc
    Commented Mar 2, 2021 at 11:42
  • Hi @JPBlanc, in the first edit I reported your code and the corresponding error. Commented Mar 2, 2021 at 11:43
  • Strange it works for me. Try it again with URL hardcoded ?the last one works definitly in a brand new shell. Perhaps a trouble with an hidden character ?
    – JPBlanc
    Commented Mar 2, 2021 at 12:22
  • @GennaroArguzzi, just copy pas the last 4 lines.
    – JPBlanc
    Commented Mar 2, 2021 at 13:36
  • I did it, but PS gave me the error about internet explorer. If possible, upload a short video (to record with Windows press windows key+G). Commented Mar 2, 2021 at 19:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.