27

I am using this statement in Python:

 jsonreq = json.dumps({'jsonrpc': '2.0', 'id': 'qwer', 'method': 'aria2.pauseAll'})
 jsonreq = jsonreq.encode('ascii')
 c = urllib.request.urlopen('http://localhost:6800/jsonrpc', jsonreq)

I am getting this warning/error when I perform code quality test

Audit url open for permitted schemes. Allowing use of "file:" or custom schemes is often unexpected.

3 Answers 3

25

Because I stumbled upon this question and the accepted answer did not work for me, I researched this myself:

Why urlib is a security risk

urlib not only opens http:// or https:// URLs, but also ftp:// and file://. With this it might be possible to open local files on the executing machine which might be a security risk if the URL to open can be manipulated by an external user.

How to fix this

You are yourself responsible to validate the URL before opening it with urllib. E.g.

if url.lower().startswith('http'):
  req = urllib.request.Request(url)
else:
  raise ValueError from None

with urllib.request.urlopen(req) as resp:
  [...]

How to fix this so the linter (e.g. bandit) does no longer complain

At least bandit has a simple blacklist for the function call. As long as you use urllib, the linter will raise a warning. Even if you DO validate your input like shown above. (Or even use hardcoded URLs).

Add a #nosec comment to the line to suppress the warning from bandit or look up the suppression keyword for your linter/code-checker. It's best practice to also add additional comments stating WHY you think this is not worth a warning in your case.

1
  • Thanks a lot for the explanation. There are two typos in your code: 'urlib.request.Request' (Caps inverted) and ':' just after 'else'. Commented Jul 21, 2020 at 10:48
9

I think this is what you need

import urllib.request

req = urllib.request.Request('http://www.example.com')
with urllib.request.urlopen(req) as response:
    the_page = response.read()
1
  • 3
    Thanks Lamrin, it's working cool with test cases. So the file type read is this? Commented Feb 14, 2018 at 6:22
4

For the people who couldn't solve it by above answers. You could use requests library instead, which is not black listed in bandit.

https://bandit.readthedocs.io/en/latest/blacklists/blacklist_calls.html#b310-urllib-urlopen

import requests

url = 'http://www.example.com'
the_page = requests.get(url)

print(the_page.json()) # if the response is json
print(the_page.text) # if the response is some text

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.