The docs of Scrapy
says about HttpProxyMiddleware
is like this:
This middleware sets the HTTP
proxy
to use for requests, by setting theproxy
meta value forRequest
objects.Like the Python standard library modules
urllib
andurllib2
, it obeys the following environment variables:http_proxy https_proxy no_proxy
You can also set the meta key proxy per-request, to a value like
http://some_proxy_server:port
orhttp://username:password@some_proxy_server:port
. Keep in mind this value will take precedence overhttp_proxy/https_proxy
environment variables, and it will also ignoreno_proxy
environment variable.
But there are no examples in the docs.
I have no ideas how to write a HttpProxyMiddleware
.
Are there any suggestions?