I am doing a CA and I have to parse the page using beautiful soup, I did with the code
r = urlopen(url) # download the page
res1 = str(r.read()) # put the content into a variable
soup = BeautifulSoup(res1,'html.parser')
for link in soup.find_all('a'):
print(link.get('href'))
but then I have to print how many different pages have been crawled.
Is anybody has a tip to give me ?
Thank you very much
# download the **page**
soup = BeautifulSoup(res1.read(),'html.parser')
read
,BeautifulSoup(urlopen(url), ...)
is sufficient - BeautifulSoup accepts file-like objects.