0

i need to open an URL that is inside another URL, but i dont know how to do it

this is the code i wrote (im a beginner), and it send me a traceback: URLError: <urlopen error unknown url type: "http>

please help me

thank u

import re 
import urllib.request, urllib.parse, urllib.error

fhand=urllib.request.urlopen("http://dr-chuck.com/page1.htm")
for line in fhand:
    words=line.decode().strip()
    if re.search("href=", words):
        print(words)
        w=words.split()
        print(w)
        i=w[1]
        print(i)
        p=i[5:40]
        print(p)
        q=urllib.request.urlopen(p)
        print(q)
1
  • Hello, please include the output of the above so we know what the code is doing and what is stored in various variables above. In general, the approach of searching for links can be problematic. You may find that searching for links, storing in a list, and then post-processing may help you separate tasks of "finding" links vs "opening" the links.
    – jatal
    Sep 9, 2020 at 22:41

1 Answer 1

0

The question is too broad to give a direct answer to, however as a general approach, you would need to use a scraping library such as Scrapy or BeautifulSoup.

You would then perform a GET request on the parent URL (i.e. http://dr-chuck.com/page1.htm) and parse that page's HTML. You would then target and extract the href HTML tag and save the URL itself into another variable and then perform another GET request on that.

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.