I want to scrape some stock data from a website https://dps.psx.com.pk/ using VBA in Excel, but the problem is the URL of this website does not change.
When I click on market summary as highlighted in image#1 Image#1
that will return the whole market summary, I just need to scrape data in Excel using VBA as highlighted in the image#2. Image#2
I tried to examine the network with fiddler as shown in image#3 Image#3
and develop the following code in VBA.
Option Explicit
Sub Test()
' Add references
' Microsoft HTML Object Library
' Microsoft XML, v6.0
Dim sResp As String
Dim rOutputCell As Range
Dim oElememnt
Dim cElements As IHTMLElementCollection
Dim oTableRow
Dim oTableCell
' Retrieve HTML from website
With New MSXML2.XMLHTTP60
' Send request
.Open "GET", "https://dps.psx.com.pk/webpages/mktSummary.php?r=REG", True
Do Until .ReadyState = 4: DoEvents: Loop
sResp = .ResponseText
End With
' Parse response and output
With New HTMLDocument
' Load response HTML into DOM
.body.innerHTML = sResp
' Clear first worksheet for output
ThisWorkbook.Sheets(1).Cells.Delete
Set rOutputCell = Cells(3, 1)
Set oElememnt = .getElementsByClassName("tableHead")(0)
For Each oTableRow In oElememnt.getElementsByTagName("tr")
For Each oTableCell In oTableRow.getElementsByTagName("td")
rOutputCell.Value = oTableCell.innerText
Set rOutputCell = rOutputCell.Offset(0, 1)
Next
Set rOutputCell = rOutputCell.Offset(1, 0).EntireRow.Cells(1, 1)
Next
End With
MsgBox "Completed"
End Sub
But when I run this code it just shows running but nothing happens even after waiting for sometime. I don't know whether it got stuck in Event Loop or some other problem is there please help.
Set rOutputCell = Cells(3, 1)
, run the code, when it stops at the breakpoint, begin to step into line by line, checking values of variables in the locals window. First of all checkoElememnt
,oTableRow
,oTableCell
. You should make sure that.getElementsByClassName("tableHead")
returns not empty collection, and thattableHead
is relevant here. – omegastripes Jul 8 '18 at 14:16