2

I have got the following error:

ProtocolError: ('Connection aborted.', OSError(0, 'Error'))

During handling of the above exception, another exception occurred:

ConnectionError Traceback (most recent call last) in

---> 16 df['List'] = df['Link'].apply(get_all_links)

/anaconda3/lib/python3.7/site-packages/pandas/core/series.py in apply(self, func, convert_dtype, args, **kwds) 4106
else: 4107 values = self.astype(object)._values -> 4108 mapped = lib.map_infer(values, f, convert=convert_dtype) 4109 4110 if len(mapped) and isinstance(mapped[0], Series):

pandas/_libs/lib.pyx in pandas._libs.lib.map_infer()

in get_all_links(url) 7 # but that's outside the scope here 8 print(url) ----> 9 soup = BeautifulSoup(requests.get(url).content, "html.parser") 10 11 return [a.attrs.get('href', '') for a in soup.find_all('a')]

/anaconda3/lib/python3.7/site-packages/requests/api.py in get(url, params, **kwargs) 74 75 kwargs.setdefault('allow_redirects', True) ---> 76 return request('get', url, params=params, **kwargs) 77 78

ConnectionError: ('Connection aborted.', OSError(0, 'Error'))

I would say that a try/except condition might fix the issue. Does anyone know how to fix it?

def get_all_links(url):
    soup = BeautifulSoup(requests.get(url).content, "html.parser")

    return [a.attrs.get('href', '') for a in soup.find_all('a')]

df['List'] = df['Link'].apply(get_all_links)

I think all the information are shared. The website that seems causing the issue should be 'https://www.puppetstringnews.com/'.

An example of urls to test:

https://www.stackoverflow.com
https://deepclips.com/
https://www.puppetstringnews.com/
0

1 Answer 1

1

Did you try to open that website in the browser ? It first prompts you with an certificate error, and when you click "accept risk and continue", it gives an 404 Not Found error. The error probably is at the server side of that website, not in your code. To verify that, you can try another webpage. But with this one, you will never get the expected output. To skip such erroneous websites, wrap your get_all_links function body into a try / except Exception as e. Additionally, in the except clause, you should print("Error with url %s : %s" % (url, e)).

5
  • 1
    @Val you have to move the Return statement before the except (inside the try block)
    – TheEagle
    Feb 7, 2021 at 20:36
  • @Val no problem !
    – TheEagle
    Feb 7, 2021 at 20:44
  • @Val could you maybe post some of the urls for testing purposes ?
    – TheEagle
    Feb 7, 2021 at 21:03
  • 1
    I updated the question including three urls (just for testing). Thanks, Programmer
    – V_sqrt
    Feb 7, 2021 at 21:46
  • good day dar Programmer - thanks alot! this looks aweseome
    – zero
    Feb 9, 2021 at 12:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.