0

I came across this website and tried to test-run the code presented there so I could adjust it later.

Upon realising it was written in Python2, I converted it into Python 3 manually. I fixed "print" and "urlparse" statements and removed

reload(sys)
sys.setdefaultencoding("utf-8")"

(Google said it was unnecessary in Python3. Really?).

When I now run the code, I get the following output and I have no idea what to do to fix this since I am very new to Python. I looked at similar questions but the solutions there couldn't fix this code.

[*] Retrieved 89 possible stored URLs
[*] Retrieving https://web.archive.org/web/20110823161411/http://www.oct282011.com/ (1 of 89)
[*] Retrieving https://web.archive.org/web/20110830211214/http://www.oct282011.com/ (2 of 89)
[+] Adding new image: https://web.archive.org/web/20110830211214im_/http://www.oct282011.com/st.jpg
[+] Adding new image: https://web.archive.org/web/20110830211214im_/http://www.oct282011.com/itswhatyouknow_itsnotamystery_mostofyouhavefoundit.png
[+] Adding new image: https://web.archive.org/web/20110830211214im_/http://www.oct282011.com/sngl.jpg
[+] Adding new image: https://web.archive.org/web/20110830211214im_/http://www.oct282011.com/qst.png
[*] Retrieving https://web.archive.org/web/20110831172158/http://www.oct282011.com/ (3 of 89)
[+] Adding new image: https://web.archive.org/web/20110831172158im_/http://www.oct282011.com/0light.png
[+] Adding new image: https://web.archive.org/web/20110831172158im_/http://www.oct282011.com/itswhatyouknow_itsnotamystery_mostofyouhavefoundit.png
[+] Adding new image: https://web.archive.org/web/20110831172158im_/http://www.oct282011.com/sngl.jpg
[+] Adding new image: https://web.archive.org/web/20110831172158im_/http://www.oct282011.com/0.png
[*] Retrieving https://web.archive.org/web/20110903151702/http://www.oct282011.com/ (4 of 89)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-7152635ed042> in <module>
    123 print("[*] Retrieved %d possible stored URLs" % len(results.assets))
    124 
--> 125 image_paths = get_image_paths(results)
    126 
    127 print("[*] Retrieved %d image paths." % len(image_paths))

<ipython-input-3-7152635ed042> in get_image_paths(packed_results)
     54 
     55         # parse out all image tags
---> 56         soup = bs4.BeautifulSoup(result)
     57         image_list = soup.findAll("img")
     58 

~\Anaconda3\lib\site-packages\bs4\__init__.py in __init__(self, markup, features, builder, parse_only, from_encoding, exclude_encodings, **kwargs)
    265         if hasattr(markup, 'read'):        # It's a file-type object.
    266             markup = markup.read()
--> 267         elif len(markup) <= 256 and (
    268                 (isinstance(markup, bytes) and not b'<' in markup)
    269                 or (isinstance(markup, str) and not '<' in markup)

TypeError: object of type 'NoneType' has no len()

My questions are:

  1. Why does Python point at three code lines (126,56,267) and why in this strange order? Are there three errors or is only the lowest-numbered line (56) of importance here?
  2. What is the problem and how can it be fixed? Please keep in mind I am a total beginner.

I don't know if this is important but am using Python3 in a Jupyter notebook.

1
  • 1
    Please avoid asking multiple questions at once. The 1. code lines in strange order are a "Traceback" as the heading says – this word should allow searching for explanations. As for the 2. problem, please see the minimal reproducible example and help center pages how to help us help you. Oct 26, 2020 at 8:46

3 Answers 3

0

It looks like the result = asset.fetch() call returns None and that breaks things down the line.

You could add something like

if not result:
    print('Result was None:', asset)
    continue

after that line to skip the assets which couldn't get fetched.

0

1)

So actually the three lines is a path to identify the real error: The first line is the function which was called that failed, but it throws an error because of another function in that first function, which is the second line/arrow. The second line/function throws an error due to an error in the second function, which is the third and final line. An example would be


def f1(a):
    b = f2(a)
   

def f2(a):
   c = 1/a 

if you call f1(0) you'll get the error


f1(0)
---------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
c:\Users\test.py in <module>
----> 1 f1(0)

c:\Users\test.py in f1(a)
      1 def f1(a):
----> 2     b = f2(a)

c:\Users\test.py in f2(a)
      1 def f2(a):
----> 2    c = 1/a

ZeroDivisionError: division by zero

which says "there is an error in the call f1(0)" (first line/arrow) "but that is due to the call b=f2(a)" (second line/arrow) "which fails on the line c=1/a" (third line/arrow).

That way you can back-trace your error. If it just said "there is an error in f1(0)" it would be very difficult to trace what the real error is (c=1/0)

2) It might be due to results being empty i.e does not exist

0

Just adding to other answers:

I believe the first part of your question has been answered but as to the second part, in line 267 you could specify:

elif markup and len(markup) >= 256 and ((isinstance(markup, bytes) and not b'<' in markup) or (isinstance(markup, str) and not '<' in markup))

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.