1

I use a python script to webscrape for "Show Notes" and an mp3. When I encounter a page that has no show notes, this means the show was a Best Of, so I want to skip the download of the notes and mp3. I am not sure where the best place to insert the test would be. The snippet is as follows:

for show_html in showpage_htmls:
    try:
        p_html = s.get(show_html)
        p_soup = BeautifulSoup(p_html.content, 'html.parser')

        # set title for SHOW NOTES
        title = ''
        title = p_soup.title.contents[0]

        # get SHOW NOTES chunk and remove unwanted characters (original mp3notes not changed)
        mp3notes = ''
        mp3notes = p_soup.find('div', {'class': 'module-text'}).find('div')
        mp3notes = str(title) + str('\n') + str(mp3notes).replace('<div>','').replace('<h2>','').replace('</h2>','\n').replace('<p>','').replace('<br/>\n','\n').replace('<br/>','\n').replace('</p>','').replace('</div>','').replace('\u2032','')
        # FIXME need to skip d/l if no notes

        # set basename, mp3named and mp3showtxt
        mp3basename = '{0}{1}{2}'.format(show_html.split('/')[3],show_html.split('/')[4],show_html.split('/')[5])
        if (os.name == 'nt'):
            mp3showtxt = mp3dir + '\\' + mp3basename + '.txt'
            mp3named = mp3dir + '\\' + mp3basename + '.mp3'
        else:
            mp3showtxt = mp3dir + '/' + mp3basename + '.txt'
            mp3named = mp3dir + '/' + mp3basename + '.mp3'

        # save show notes to local
        with open(mp3showtxt, 'w') as f:
            try:
                f.write(mp3notes)
                print("Show notes " + mp3basename + " saved.")
            except UnicodeEncodeError:
                print("A charmap encoding ERROR occurred.")
                print("Show notes for " + mp3basename + ".mp3 FAILED, but continuing")
            finally:
                f.close()
        # FIXME need eyed3 to set mp3 tags since B&T are lazy

        # get Full Show mp3 link
        mp3url = p_soup.find('a', href = True, string = 'Full Show').get('href')

        # get and save mp3
        r = requests.get(mp3url)
        with open(mp3named, 'wb') as f:
            f.write(r.content)
        print("Downloaded " + mp3basename + ".mp3.")
    except AttributeError:
        print(show_html + " did not exist as named.")

I would think an

if not (len(mp3notes) >= 50)

would work; just not sure where to put it or there is better way (more Pythonic).

Ideally, if the mp3notes are less than expected, no notes or mp3 for that show_html would be saved, and the script would start at the next show_html page.

Since I am new to Python, feel free to offer suggestions to making this more Pythonic as well; I am here to learn! Thanks.

3
  • 1
    Hi David, welcome to StackOverflow. I think this sort of question would be more at home on the CodeReview stack exchange. Dec 8, 2017 at 15:56
  • I agree that the "feel free to offer suggestions" portion would be more relevant to being on CodeReview, but that was not the root of my presence here - I have a question of location for a test and was seeking help in writing it. [CodeReview specifically states "... asking for advice about code not yet written are off-topic, as the code is not ready for review."]
    – David
    Dec 8, 2017 at 23:37
  • The reason I mention CodeReview is that this seems to me to be a question asking about best practices which are off-topic for StackOverflow as seen here. meta.stackoverflow.com/questions/296542/…. You've written all the code except that one line that you want which you have an idea for already. If you make your best guess and then say you are unsure about the placement and quality of that line you will be entirely on topic CodeReview. Best of luck either way! Dec 9, 2017 at 3:32

0

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.