0

I'm stumped. I don't want to use a WebBrowser in my application, and I want to get a specific element by id. my code is:

Dim request As System.Net.HttpWebRequest = System.Net.HttpWebRequest.Create("http://www.google.com/finance?q=NASDAQ:GOOG")
Dim response As System.Net.HttpWebResponse = request.getresponse()
Dim sr As System.IO.StreamReader = New System.IO.StreamReader(response.GetResponseStream())
Dim sourcecode As String = sr.ReadToEnd()
TextBox1.Text = sourcecode

This gets me the source code. But how do I get a specific element? I would think that there is an easy way to do this... Btw I don't want to use Regex, or download HTML Agility Pack.

5
  • 1
    The solution is to download the HTML Agility Pack. That is the easiest way. By far. The alternative would be to build an HTML parser yourself. Jan 14, 2013 at 20:57
  • 1
    You need to use HTML Agility Pack. You can download it easily using NuGet.
    – SLaks
    Jan 14, 2013 at 20:58
  • 2
    Why wont you want to use Regex or HTML Agility Pack?
    – Magnus
    Jan 14, 2013 at 20:59
  • You do not have to use HTML agility pack if its a highly customizable project. I use the MSHTML libraries directly with a little modification of code retrieved from Microsoft forums. It's tricky to setup though. I have a project that uses it here: dais.codeplex.com. I was experimenting with downloading and performing analysis on HTML documents automatically. In order to get a specific html element though you need mshtml. Ironically, HTML agility pack just is a nice wrapper library for mshtml. Regex would be useful when combined together with them.
    – jeffery
    Mar 25, 2014 at 14:15
  • You could though do as people suggested and parse the html yourself which is so much harder. I've seen the code too much on msdn forums with and without MSHTML or html agility pack and without it is much more difficult (i.e. Lots of repeated code to perform a task I could perform with 5-10 lines sometimes with MSHTML).
    – jeffery
    Mar 25, 2014 at 14:23

2 Answers 2

0

You can make a parse table to recognize html tags, and search for id=elementname (plus possible whitespace characters) inside the tags. It's not the impossible task it may seem, because you can ignore most tags and you don't have to validate the html. Just consider <>, and ignore the contents of quotes, scripts, etc. There are lots more details and it takes a little work, but it's fun programming.

The alternative would be to download something like html agility pack, use a browser, or use a regex, which you'd like to avoid.

0

Heres a very rough idea and it does not work for BLOCK elements that need a SEPARATE closing tag (like ) but it works fine for self closing elements like

also i noted that some of tag id's are enclosed in speech marks and some are not, so you would have to tweak that possibly...

I just roughed this code up and copy pasted the routine to detect unenclosed id tags but it still needs work on it and could be shortened too.

<script runat="server">
Dim sourcecode As String
Dim bodycode As String
Dim RetVal As String

Protected Sub Page_Load(sender As Object, e As System.EventArgs)
    '
    LoadHttpStuff()
    If Request.Form("Button1") = "Submit" Then
        RetVal = MyGetElementById(Request("Text1"))
    End If

End Sub

Private Sub LoadHttpStuff()

    Dim request As System.Net.HttpWebRequest
    Dim response As System.Net.HttpWebResponse
    Dim sr As System.IO.StreamReader
    Dim finishat As Long
    Dim startat As Long

    request = System.Net.HttpWebRequest.Create("http://www.google.com/finance?q=NASDAQ:GOOG")
    response = request.GetResponse()
    sr = New System.IO.StreamReader(response.GetResponseStream())
    sourcecode = sr.ReadToEnd()
    startat = InStr(sourcecode, "<body>")
    finishat = InStr(sourcecode, "</body>") + 7
    bodycode = Mid(sourcecode, startat, finishat - startat)
    bodycode = LCase(bodycode)


End Sub

Private Function MyGetElementById(Id As String) As String
    Dim tagstart As Long
    Dim tagend As Long
    Dim posx As Long
    Dim item As System.Web.UI.HtmlControls.HtmlGenericControl
    Dim test As Boolean
    Dim letter As Char
    Dim text As String
    item = Nothing
    test = False
    text = ""
    If Trim(Id) <> "" Then
        '-> with SPEECHMARKS
        posx = InStr(bodycode, LCase("id=" & Chr(34) & Id & Chr(34)))
        If posx > 0 Then
            'find start of tag
            Do
                posx = posx - 1
                letter = Mid(bodycode, posx, 1)
                If letter = "<" Then
                    'found tag start
                    tagstart = posx
                    Exit Do
                End If
            Loop Until posx < 1
            If tagstart > 0 Then
                posx = InStr(bodycode, LCase("id=" & Chr(34) & Id & Chr(34)))
                Do
                    posx = posx + 1
                    letter = Mid(bodycode, posx, 1)
                    If letter = ">" Then
                        tagend = posx + 1
                        Exit Do
                    End If
                Loop Until posx >= Len(bodycode)
                If tagend > 0 Then
                    text = Mid(bodycode, tagstart, tagend - tagstart)
                    test = True
                End If
            End If
        Else
            posx = InStr(bodycode, LCase("id=" & Id))
            If posx > 0 Then
                'find start of tag
                Do
                    posx = posx - 1
                    letter = Mid(bodycode, posx, 1)
                    If letter = "<" Then
                        'found tag start
                        tagstart = posx
                        Exit Do
                    End If
                Loop Until posx < 1
                If tagstart > 0 Then
                    posx = InStr(bodycode, LCase("id=" & Id))
                    Do
                        posx = posx + 1
                        letter = Mid(bodycode, posx, 1)
                        If letter = ">" Then
                            tagend = posx + 1
                        End If
                    Loop Until posx >= Len(bodycode)
                    If tagend > 0 Then
                        text = Mid(bodycode, tagstart, tagend - tagstart)
                        test = True
                    End If
                End If
            End If
        End If
    End If
    Return Text
End Function
</script>

<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
  <title></title>
</head>
<body>
  <form id="form1" runat="server">
    <table style="width: 100%;">
        <tr>
            <td style="text-align:left; vertical-align: top; width: 75%;"><textarea rows="20" cols="80" style="width: 90%;" disabled="disabled"><%=sourcecode%></textarea></td>
            <td style="width: 25%; text-align: left; vertical-align: top;">
                <table style="width:100%;">
                    <tr>
                        <td>Element Id&nbsp;<input id="Text1" name="Text1" type="text" /></td>
                    </tr><tr>
                        <td>&nbsp;</td>
                    </tr><tr>
                        <td>&nbsp;</td>
                    </tr><tr>
                        <td><input id="Button1" type="Submit" value="Submit" name="Button1" /></td>
                    </tr><tr>
                        <td>&nbsp;</td>
                    </tr><tr>
                        <td>&nbsp;</td>
                    </tr>
                </table>
            </td>
        </tr><tr>
            <td style="width: 75%;">&nbsp;</td>
            <td style="width: 25%;">&nbsp;</td>
        </tr><tr>
            <td style="width: 100%;" colspan="2"><textarea rows="20" cols="80" style="width: 75%;" disabled="disabled"><%=RetVal%></textarea></td>
            <td style="width: 25%;">&nbsp;</td>
        </tr>
    </table>
</form>
</body>
</html>

Hope it helps a little

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.