2

I'm trying to grab some data from a server inside my company's network with the requests library. I'm using a VPN to get into my company's network and there is also a corporate proxy set up. The address I am trying to access is only visible from within that company network.

import requests

url = "http://some.private.server.net"

r = requests.get(url)

I have both HTTP_PROXY and HTTPS_PROXY set in my environment. I am running Windows 7.

I get error 502 from this code. It worked when I was physically connected to the company network so I think the problem is to do with connecting through the VPN rather than the proxy. I can access the address with my web browser.

I don't know much about networking but I tried changing the source address for the requests to be the IP address used by the VPN network adapter.

import requests
from requests_toolbelt.adapters.source import SourceAddressAdapter

s = requests.Session()
ip = "y.y.y.y"
s.mount('http://', SourceAddressAdapter(ip))
s.mount('https://', SourceAddressAdapter(ip))

url = "http://some.private.server.net"

response = s.get(url)

This still led to the same error (502) as above.

EDIT: Looked more closely at the 502 error and the message "the host name of the page you are looking for does not exist" is being returned by the proxy server.

Therefore I tried replacing the URL with the IP address of the server I'm trying to reach and instead get a 502 due to "Gateway Timeout".

I can ping that IP from the command line no problems.

1 Answer 1

6

Ironically the issue was that I needed to not be using the proxy to access this server. You can do this with,

import requests

s = requests.Session()
s.trust_env = False

url = "http://some.private.server.net"

response = s.get(url)
3
  • Thanks, this was exactly what I needed this morning. A lot of people wouldn't care to post the correct answer after they find it, well done.
    – Copas
    Jun 30, 2020 at 10:48
  • I got connection refused when I used this
    – etotientz
    Sep 7, 2020 at 12:49
  • I got the same issue I had before adding this.
    – fadingbeat
    Nov 5, 2020 at 11:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.