4

I was trying to send http/https requests via proxy (socks5), but I can't understand if the problem is in my code or in the proxy.

I tried using this code and it gives me an error:

requests.exceptions.ConnectionError: SOCKSHTTPSConnectionPool(host='www.google.com', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.contrib.socks.SOCKSHTTPSConnection object at 0x000001B656AC9608>: Failed to establish a new connection: Connection closed unexpectedly'))

This is my code:

import requests

url = "https://www.google.com"


proxies = {
    "http":"socks5://fsagsa:[email protected]:31112",
    "https":"socks5://fsagsa:[email protected]:31112",
    }

headers = {
        "Upgrade-Insecure-Requests": "1",
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36",
        "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
        "Sec-Gpc": "1",
        "Sec-Fetch-Site": "same-origin",
        "Sec-Fetch-User": "?1",
        "Accept-Encoding": "gzip, deflate, br",
        "Sec-Fetch-Mode": "navigate",
        "Sec-Fetch-Dest": "document",
        "Accept-Language": "en-GB,en;q=0.9"
    }


r = requests.get(url, headers = headers, proxies = proxies)

print(r)

Then, I checked the proxy with an online tool The tool manages to send requests through the proxy. Online tool output.

So the problem is in this code? I can't figure out what's wrong.

Edit (15/09/2021)

I added headers but the problem is still there.

2
  • 1
    have you tried adding your headers to tell your site what you want? : accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9 accept-encoding: gzip, deflate, br accept-language: en-GB,en;q=0.9 Sep 13, 2021 at 9:12
  • Yes, I have tried. Unfortunately it didn't solve my problem.
    – SAL
    Sep 13, 2021 at 9:21

2 Answers 2

6
+50

Create a local server/mock to handle the request using pytest or some other testing framework with responses library to eliminate variables external to your application/script. I’m quite sure Google will reject requests with empty headers. Also, ensure you installed the correct dependencies to enable SOCKS proxy support in requests (python -m pip install requests[socks]). Furthermore, if you are making a remote request to connect to your proxy you must change socks5 to socks5h in your proxies dictionary.

References

pytest: https://docs.pytest.org/en/6.2.x/

responses: https://github.com/getsentry/responses

requests[socks]: https://docs.python-requests.org/en/master/user/advanced/#socks

1
  • I've already tried to add headers to that request and also, I've already tried all the steps you mentioned.
    – SAL
    Sep 14, 2021 at 16:51
4

In addition to basic HTTP proxies, Requests also supports proxies using the SOCKS protocol. This is an optional feature that requires that additional third-party libraries be installed before use.

You can get the dependencies for this feature from pip:

$ python -m pip install requests[socks]

Once you’ve installed those dependencies, using a SOCKS proxy is just as easy as using a HTTP one:

proxies = {
            'http': 'socks5://user:pass@host:port',
            'https': 'socks5://user:pass@host:port'
          }

Using the scheme socks5 causes the DNS resolution to happen on the client, rather than on the proxy server. This is in line with curl, which uses the scheme to decide whether to do the DNS resolution on the client or proxy. If you want to resolve the domains on the proxy server, use socks5h as the scheme.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.