0

I am new to beautifulsoup and I am having trouble using it on basketball reference. I am trying to store the entire dataframe of the Advanced stats into a pandas dataframe, but I cannot even select it. Here is my code so far:

from urllib.request import urlopen
from bs4 import BeautifulSoup
import pandas as pd


url='http://www.basketball-reference.com/teams/ATL/2016.html'
html = urlopen(url)
soup = BeautifulSoup(html)

soup.findAll('table',attrs={'id': 'advanced'})

I see the html of what I need after selecting advanced from the code above, but I cannot actually parse and pull the data.

0

1 Answer 1

0

Find the table element and let the read_html() do the parsing and dataframe initializing job:

table = soup.find('table', attrs={'id': 'advanced'})

df = pd.read_html(str(table))
print(df)  # prints a dataframe with 15 rows x 27 columns
4
  • Thank you for your response. I am getting an 'ImportError: html5lib not found, please install it' error. However I install and I import it and still get this error. What should I do? Jan 14, 2016 at 2:01
  • @PippensPips you may either need to install html5lib: pip3 install html5lib or try setting the parser to bs4: df = pd.read_html(StringIO(str(table)), flavor="bs4").
    – alecxe
    Jan 14, 2016 at 2:07
  • None of those solutions work. Is there another way to do this? Jan 14, 2016 at 2:16
  • @PippensPips I don't know your setup. You just need to install html5lib module to the same python environment you are executing your script in. Apparently, you have already somehow installed BeautifulSoup and pandas - it is absolutely the same way with html5lib.
    – alecxe
    Jan 14, 2016 at 2:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.