Use select_one()
in combo with SelectorGadget Chrome extension to grab CSS
selectors by clicking on the desired element in your browser:
soup.select_one('.YwPhnf').text
# 09:06
Or you can use stripped_strings
, but it's not as pretty as using CSS
selectors:
html = '''
<div class="content-cell mdl-cell mdl-cell--6-col mdl-typography--body-1">
Contents
<a href="https://www.google.com/search?q=Google+what+time+is+it">
Google what time is it
</a>
<br/>
It's 4:38.
<br/>
2018. 2. 5. 5:38:41 PM
</div>
'''
soup = BeautifulSoup(html, 'lxml')
# returns a generator object
current_time = list(soup.select_one('.content-cell').stripped_strings)[2]
print(current_time)
# It's 4:38.
Code:
from bs4 import BeautifulSoup
import requests, lxml
headers = {
'User-agent':
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.19582"
}
params = {
"q": "what time it is", # query
"gl": "us", # country to search from
"hl": "en" # language
}
html = requests.get("https://www.google.com/search", headers=headers, params=params)
soup = BeautifulSoup(html.text, 'lxml')
current_time = soup.select_one('.YwPhnf').text
current_date = soup.select_one('.KfQeJ:nth-child(1)').text
print(f'{current_time}\n{current_date}')
# 2:11 AM
# September 11, 2021
Alternatively, you can achieve the same thing by using Google Direct Answer Box API from SerpApi. It's a paid API with a free plan.
The difference in your case is that you only need to get the data you want from the structured JSON, rather than figuring out to extract things and maintain the parser over time if something won't work correctly because of some changes in the HTML.
Code to integrate:
from serpapi import GoogleSearch
params = {
"api_key": "YOUR_API_KEY",
"engine": "google",
"q": "what time it is",
"gl": "us",
"hl": "en"
}
search = GoogleSearch(params)
results = search.get_dict()
print(results['answer_box']['result'])
# 2:07 AM
Disclaimer, I work for SerpApi.