119

Using pip install for any module apparently on my Ubuntu 16.04 system with python 2.7.11+ throws this error:

TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'

What is wrong with pip? How could I reinstall it, if necessary?

Update: Full traceback is below

sunny@sunny:~$ pip install requests
Collecting requests
Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 209, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 328, in run
    wb.build(autobuilding=True)
  File "/usr/lib/python2.7/dist-packages/pip/wheel.py", line 748, in build
    self.requirement_set.prepare_files(self.finder)
  File "/usr/lib/python2.7/dist-packages/pip/req/req_set.py", line 360, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/usr/lib/python2.7/dist-packages/pip/req/req_set.py", line 512, in _prepare_file
    finder, self.upgrade, require_hashes)
  File "/usr/lib/python2.7/dist-packages/pip/req/req_install.py", line 273, in populate_link
    self.link = finder.find_requirement(self, upgrade)
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 442, in find_requirement
    all_candidates = self.find_all_candidates(req.name)
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 400, in find_all_candidates
    for page in self._get_pages(url_locations, project_name):
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 545, in _get_pages
    page = self._get_page(location)
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 648, in _get_page
    return HTMLPage.get_page(link, session=self.session)
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 757, in get_page
    "Cache-Control": "max-age=600",
  File "/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/sessions.py", line 480, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 378, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "/usr/share/python-wheels/CacheControl-0.11.5-py2.py3-none-any.whl/cachecontrol/adapter.py", line 46, in send
    resp = super(CacheControlAdapter, self).send(request, **kw)
  File "/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/adapters.py", line 376, in send
    timeout=timeout
  File "/usr/share/python-wheels/urllib3-1.13.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 610, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/share/python-wheels/urllib3-1.13.1-py2.py3-none-any.whl/urllib3/util/retry.py", line 228, in increment
    total -= 1
TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'
13
  • @MartijnPieters Sure, appending that info
    – devautor
    May 28, 2016 at 5:10
  • So where did those wheels come from? pip normally would use an embedded requests package, and requests normally would use an embedded urllib3 package. Those wheels are not embedded versions, and are probably incompatible with the pip subclass of some of this code. move /usr/share/python-wheels aside perhaps?
    – Martijn Pieters
    May 28, 2016 at 5:16
  • I'd expect pip to use pip/_vendor/requests/ and pip/_vendor/requests/packages/urllib3/, not /usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests or /usr/share/python-wheels/urllib3-1.13.1-py2.py3-none-any.whl/urllib3.
    – Martijn Pieters
    May 28, 2016 at 5:17
  • I have totally no idea about this! Where should I be looking for info on this, or what could I do?
    – devautor
    May 28, 2016 at 5:18
  • Sorry, no easy answer here; just trying to diagnose your issues. The traceback is unexpected and points to a clash between versions. Which is why normally requests and pip embed their dependencies. Your traceback shows any embedded versions are not being used, but why that is I don't know.
    – Martijn Pieters
    May 28, 2016 at 5:21

19 Answers 19

90

Ubuntu comes with a version of PIP from precambrian and that's how you have to upgrade it if you do not want to spend hours and hours debugging pip related issues.

apt-get remove python-pip python3-pip
wget https://bootstrap.pypa.io/get-pip.py
python get-pip.py
python3 get-pip.py

As you observed I included information for both Python 2.x and 3.x

9
  • 2
    Doing python get-pip.py fails with this "OSError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/site-packages/pip-8.1.2.dist-info' "
    – devautor
    Jul 4, 2016 at 16:53
  • 2
    @mirror, use sudo because get-pip.py will try to install pip Aug 24, 2016 at 18:49
  • 47
    This problem has nothing to do with pip. the stack trace shows that the request to get the package fails, indicating a network problem. Telling the OP to reinstall pip will just be a detour.
    – Thom
    Oct 16, 2016 at 10:08
  • 2
    this is the right answer, the culprit is canonical patching of pip that has remain unfixed for several releases now: github.com/pypa/pip/issues/4779
    – lurscher
    Apr 19, 2018 at 13:29
  • 2
    Used it on raspberian. Couldn't run "pip" anymore. Could only run pip3. I had to add a symbolic link: ln -s /usr/local/bin/pip /usr/bin/pip taken (and modified) from this answer: stackoverflow.com/questions/29712519/…
    – TheTrowser
    May 30, 2018 at 0:05
39

If you are behind a proxy, you must do some extra configuration steps before starting the installation. You must set the environment variable http_proxy to the proxy address. Using bash this is accomplished with the command

export http_proxy="http://user:[email protected]:port/" 

You can also provide the

--proxy=[user:pass@]url:port 

parameter to pip. The [user:pass@] portion is optional.

26

Updating setuptools has worked out fine for me.

sudo pip install --upgrade setuptools
4
  • Worked for me too. Actually, after doing this, I realised that the errors used to appear after: Collecting setuptools (from kiwisolver>=1.0.1->matplotlib)
    – mayid
    May 24, 2019 at 13:50
  • 3
    This worked for me as well on an Raspian Stretch. Please accept this answer to make it easier to find Oct 13, 2019 at 1:39
  • needed a reboot after that to bake it work with raspbian
    – Pipo
    Dec 16, 2019 at 20:45
  • 5
    This command cause the same TypeError for me on Debian9. This is not universal solution. Feb 3, 2020 at 7:17
17

First of all, this problem exists because of network issues, and uninstalling and re-installing everything won't be of much help. Probably you are behind proxy, and in that case you need to set proxy.

But in my case, I was facing the problem because I wasn't behind proxy. Generally, I work behind proxy, but when working from home, I set the proxy to None in Network settings.

But I was still getting the same errors even after removing the proxy settings.

So, when I did type

env | grep proxy

I found something like this :

http_proxy=http://127.0.0.1:1234/

And this was the reason I was still getting the very same error, even when I thought I had removed the proxy settings.

To unset this proxy, type

unset http_proxy

Follow the same approach for all the other entries, such as https_proxy.

14

What happens here is that the the vendored versions of request/urllib3 clash when imported in two different places (same code, but different names). If you then have a network error, it doesn't retry to get the wheel, but fails with the above error. See here for a deeper dive into this error.

For the solution with system pip, see above.

If you have this problem in a virtualenv built by python -m venv (which still copies the wheels from /usr/share/python-wheels, even if you have pip installed separately), the easiest way to "fix" it seems to be:

  1. create the virtualenv: /usr/bin/python3.6 -m venv ...
  2. install requests into the environment (this might raise the above error): <venv>/bin/pip install requests
  3. remove the copied versions of requests which would be used by pip: rm <venv>/share/python-wheels/{requests,chardet,urllib3}-*.whl

Now a <venv>/bin/pip uses the installed version of requests which has urllib3 vendored.

3
  • Thanks, I just removed this /usr/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl and it's fixed now (without virtualenv).
    – krsoni
    Oct 29, 2017 at 12:59
  • Actually in my case, I was not using the virtual env.
    – krsoni
    Nov 3, 2017 at 7:26
  • If you remove the whl files directly in /usr/share/python-wheels, python -m venv will not create a useable <venv>/bin/pip anymore. So if you remove these files you should be sure that you do not need it. If you do need both a system pip and a venv pip, you need to do install the system one with get-pip.py (See stackoverflow.com/a/37531821/1380673) and you need to remove the whl files in each venv.
    – Jan Katins
    Nov 6, 2017 at 9:47
7

port 443 is not open, just allow custom tcp port 443 if on AWS else open the port 443 for the outbound connections ...

1
  • This helped me big time in my OpenStack cluster !! Good one Deepak !
    – Arun Das
    Sep 6, 2017 at 1:00
6

Just upgrade pip worked for me:

pip install --upgrade pip

5
  • 5
    the error happen to me while running this very exact command :( Apr 16, 2018 at 15:32
  • It fixed the error for me on a Raspbian Sketch Lite. Nov 6, 2018 at 18:36
  • 1
    I am also running Raspian Stretch Lite and it did not fix the error. Mar 21, 2019 at 9:24
  • I concur this worked beautifully "python -m pip install --upgrade pip"
    – imbatman
    Jun 27, 2019 at 14:10
  • If the top answer, "PIP from precambrian" is causing it, then this is an appropriate simple fix. Worked for me while attempting to install awscli...in Raspbian Stretch.
    – gbarry
    Oct 15, 2020 at 21:59
2

I have the same problem when installing a RaspberryPI TFT from Adafruit with pitft.sh / adafruit-pitft.sh.

I am not happy about coding-styles with errors from somewhere to be interpreted somehow - as could be seen by the previous answers.

Remark: The type error exception of retry.py is obviously a bug, caused by an unappropriate assignement and calculation of an instance of the class Reply to an int with the default value of 10 - somewhere in the code... Should be fixed either by adding an inplace-operator, or fixing the erroneous assignment.

So tried to analyse and patch the error itself first. The actual error in my case case is the same - retry.py called by pip.

The installation script adafruit-pitft.sh / pitft.sh tries to apply urllib3 which itself tries to install nested dependencies by pip, so the same error.

adafruit-pitft.sh # or pitft.sh

...

_stacktrace=sys.exc_info()[2]) File "/usr/share/python-wheels/urllib3-1.13.1-py2.py3 none-any.whl/urllib3/util/retry.py", line 228, in increment

total -= 1

TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'

For the current distribution(based on debian-9.6.0/stretch):

File "/usr/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/util/retry.py", line 315, in increment

total -= 1

TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'

The following - dirty *:) - patch enables a sounding error trace:

# File: retry.py - in *def increment(self, ..* about line 315
# original: total = self.total

# patch: quick-and-dirty-fix
# START:
if isinstance(self.total, Retry):
    self.total = self.total.total

if type(self.total) is not int:
    self.total = 2 # default is 10
# END:

# continue with original:
total = self.total

if total is not None:
    total -= 1

connect = self.connect
read = self.read
redirect = self.redirect
cause = 'unknown'
status = None
redirect_location = None

if error and self._is_connection_error(error):
    # Connect retry?
    if connect is False:
        raise six.reraise(type(error), error, _stacktrace)
    elif connect is not None:
        connect -= 1

The sounding output with the temporary patch is(displayed twice...?):

Retrying (Retry(total=1, connect=None, read=None, redirect=None)) after connection broken by 'ConnectTimeoutError(<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at/

Retrying (Retry(total=0, connect=None, read=None, redirect=None)) after connection broken by 'ConnectTimeoutError(<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at/

Could not find a version that satisfies the requirement evdev (from versions: )

No matching distribution found for evdev

WARNING : Pip failed to install software!

So in my case actually two things cause the error, this may vary in other environments:

  1. Missing evdev => try to install
  2. Failed to connect a repo/dist containing evdev in order to download. => finally give it up

My installation environment is offline from an internal debian+raspbian mirror, thus do not want to set the proxy...

So I proceeded by manual installation of the missing component evdev:

  1. download evdev from PyPI(or e.g. from github.com):

    https://pypi.org/project/evdev/

    https://files.pythonhosted.org/packages/7e/53/374b82dd2ccec240b7388c65075391147524255466651a14340615aabb5f/evdev-1.1.2.tar.gz

  2. Unpack and install manually as root user - for all local accounts, so detected as installed:

    sudo su -

    tar xf evdev-1.1.2.tar.gz

    cd evdev-1.1.2

    python setup.py install

  3. Call install script again:

    adafruit-pitft.sh # or pitft.sh

    ...Answer dialogues...

    ...that's it.

If you proceed online by direct PyPI access:

  1. check your routing + firewall for access to pypi.org

  2. set a proxy if required (http_proxy/https_proxy)

And it works..

Hope this helps in other cases too.

Arno-Can Uestuensoez

----------------------------------------------

See also: issue - 35334: https://bugs.python.org/issue35334

----------------------------------------------

See now also: issue - 1486: https://github.com/urllib3/urllib3/issues/1486

for file: https://github.com/urllib3/urllib3/blob/master/src/urllib3/util/retry.py

2

check for network issues, to bypass the exception case code

In my case, I was using a custom index, that index had no route and such would trigger the exception case code. The exception case bug still exists and still masks the real issue, however I was able to work around this by testing the connectivity with other tools such as nc -vzw1 myindex.example.org 443 and retrying when the network was up.

1

I was facing similar issue while trying to install awscli tool on ec2 instance. I changed security group to allow port 443 inbound and outbound access and that solved the issue for me.

2
1

I got this error when I was trying to create a virtualenv with command virtualenv myVirtualEnv. I just added a sudo before the command; it solved everything.

2
  • It's not a solution: I run as root in docker and have the same issue. Also you don't explain why running as root helps.
    – Eric
    Oct 12, 2017 at 10:00
  • 1
    @Eric, I just used try&error approach; I don't know the underlying purpose. It worked for me, so I suggested. Oct 16, 2017 at 6:10
1

Solution:
1. sudo apt remove python-pip
2. pip3 install pip (or install pip by get-pip.py)

Why:
This error occurred on pip 8.0.1 which installed by apt-get. And happened only when your network is unstable.

If you have a pip installed with apt, it hides the pip you installed by other ways, so you should remove the apt one first.

I disconnected the network and tested 8.0.1, 9.0.3, 10.x the 3 versions installed with pip3 or get-pip.py, no error occurred.   So, I think only the apt version of pip 8.0.1 has that bug, the others is ok.

1

I tried the solution answered above:

apt-get remove python-pip python3-pip
wget https://bootstrap.pypa.io/get-pip.py
python get-pip.py
python3 get-pip.py

When I tried

python get-pip.py 
python3 get-pip.py

I got this message

 Could not install packages due to an EnvironmentError:
[Errno 13] Permission denied: /usr/bin/pip3 Consider using the --user
 option or check the permissions.

I did the following and it works

python3 -m venv env
source ./env/bin/activate
Sudo apt-get update 
apt-get remove python-pip python3-pip
wget https://bootstrap.pypa.io/get-pip.py
python get-pip.py
python3 get-pip.py
pip3 install pip
sudo easy_install pip
pip install --upgrade pip
1

In my case, i had opened Pycharm in sudo mode, and was running pip install nltk in pycharm terminal which showed this error. running with sudo pip install solves the error.

0

I also had this issue. Initially, a proxy was set and work fine. Then I connected to a network where it doesn't go through a proxy. After unsetting proxy pip again get works.

unset http_proxy; unset http_prox;  unset HTTP_PROXY; unset HTTPS_PROXY
0

Bizarrely if I remove the proxy from the environment and add it to the command line it works for me. For example to upgrade pip itself:

env http_proxy= https_proxy= pip install pip --upgrade --proxy 'http://proxy-url:80'

My issue was having the proxy in the environment. It seems that pip only honors the one in argument.

0

This is the working solution to this problem I found.

sudo apt-get clean
cd /var/lib/apt
sudo mv lists lists.old
sudo mkdir -p lists/partial
sudo apt-get clean
sudo apt-get update
0

For myself, it turns out that wlan0 was down, which resulted in me being unable to connect out. So, ensuring that wlan0 was up, allowed pip / pip3 to work without issue.

0

fixed it temporary:

pip install requests -i http://a.b.com/pypi/simple --trusted-host a.b.com

fixed it permanent:

linux OS: add these in ~/.pip/pip.conf(create it if no exist)

[global]
index-url = http://a.b.com/pypi/simple
[install]
trusted-host = a.b.com

ps: http://a.b.com/pypi/simple your proxy_http_address

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.