-3

I need to take link to the latest user tiktok video. it worked, but then stopped working

import bs4
import requests
from fake_useragent import UserAgent
from bs4 import BeautifulSoup

link = "https://www.tiktok.com/@dava_m?"
headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36 OPR/81.0.4196.61'}
full_usd_page = requests.get(link, headers=headers).text
soup = BeautifulSoup(full_usd_page, 'html.parser')
product_link = soup.find("a", class_="").get("href")
print(link)
product_link = soup.find("a", class_="").get("href")
AttributeError: 'NoneType' object has no attribute 'get'

it returns none, but an hour ago it worked. pls help to get latest video link https://www.tiktok.com/@dava_m?

7
  • How did it stop working? Does it print nothing? Does it crash? Does it print someting unexpected?
    – Bill Lynch
    Nov 29, 2021 at 16:24
  • product_link = soup.find("a", class_="").get("href") AttributeError: 'NoneType' object has no attribute 'get'
    – ggindinson
    Nov 29, 2021 at 16:25
  • forgot to write
    – ggindinson
    Nov 29, 2021 at 16:25
  • Run soup.find("a", class_="") by itself. It is None, so you need to check this before calling .get("href") Nov 29, 2021 at 16:26
  • but it just can t be none, im getting latest video, which definetely is not none in my tiktok example
    – ggindinson
    Nov 29, 2021 at 16:26

2 Answers 2

0

This means soup.find("a", class_="") is returning a nonetype object. Try running it by itself without the get bit and check the output. Make sure the output of the following is not None.

soup.find("a", class_="")

It's possible that the function can't find anything.

4
  • The thing is it really returns None, but one hour ago it worked! IDK how. I can t understand which tag i should put.
    – ggindinson
    Nov 29, 2021 at 16:31
  • tiktok.com/@dava_m? can you please make a code of getting the latest video link and compare it with mine, pls
    – ggindinson
    Nov 29, 2021 at 16:32
  • You said it worked an hour ago - That's around the time that user "Dava_m" uploaded a new video. My guess is the upload of a new video changed the html code, meaning the find method no longer works. Use a debugger and check through the value of variable soup (the html).
    – Munchies
    Nov 29, 2021 at 16:36
  • I thought the same way, but since i turned my computer on, it made a new video link, and then just stopped working, idk how
    – ggindinson
    Nov 29, 2021 at 16:38
0

Probably the page is dynamically generated, and it's not sufficient to use simply requests. But other methods are quite more sophisticated. They might be selenium, scrapy-splash, etc. Search the web for python scrape dynamic content/page.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.