9

I am using python 3.5.2 and scrapy 1.1.1.

There is an error when running the code below:

#-*- coding:utf-8-*-

import random
import base64


class ProxyMiddleware(object):
    def process_request(self, request, spider):
        proxy = random.choice(PROXIES)
        if proxy['user_pass'] is not None:
            request.meta['proxy'] = "http://%s" % proxy['ip_port']
            encoded_user_pass = base64.encodebytes(proxy['user_pass'])
            request.headers['Proxy-Authorization'] = 'Basic ' + encoded_user_pass
            print("ok!" + proxy['ip_port'])
        else:
            print("fail!" + proxy['ip_port'])
            request.meta['proxy'] = "http://%s" % proxy['ip_port']

error:

  File "C:\Users\dell\AppData\Local\Programs\Python\Python35\lib\base64.py", line 518, in _input_type_check
    m = memoryview(s)
    TypeError: memoryview: a bytes-like object is required, not 'str'

I think the error is related to this sentence:

encoded_user_pass = base64.encodebytes(proxy['user_pass'])

But I don't know how to solve it.
Some help please,
thanks in advance!

Edit:

encoded_user_pass = base64.encodebytes(proxy['user_pass'])

was changed to

encoded_user_pass = base64.encodebytes(proxy['user_pass'].encode())

there is another error:

    request.headers['Proxy-Authorization'] = 'Basic ' + encoded_user_pass
TypeError: Can't convert 'bytes' object to str implicitly

what should I do?

2
  • Just FYI base64 encoding the password is insecure and pretty much does nothing in terms of security. You should look into salt hashing for a more secure way of doing it; check out this question: stackoverflow.com/questions/9594125/… Jul 24, 2016 at 15:45
  • @Granitosaurus thanks,I will try it later.now there is another error,see my edit ,thanks.
    – zwl1619
    Jul 24, 2016 at 15:59

2 Answers 2

8

Function base64.encodebytes() is expecting bytes value and it seems like you are providing it a string.

To fix that you can simply encode your string value (encode() function turns your string object into bytes object):

base64.encodebytes('foo'.encode())

or in your case:

encoded_user_pass = base64.encodebytes(proxy['user_pass'].encode())
1
  • Regarding your edit: Your error here means that one value is string the other one is bytes and when you try to add them python gets a bit confused. You should either convert both to bytes with str.encode() or convert both to string with bytes.decode(). You can just fix it with b'Basic' + encoded_user_pass in this specific case. Jul 24, 2016 at 16:01
2

As the name suggests, base64.encodebytes encodes bytes, not strings. As outlined in the documentation, it requires a bytes-like object (just like the error message also tells you).

Your proxy['user_pass'] value is a string however. In order to convert that into bytes, call str.encode:

encoded_user_pass = base64.encodebytes(proxy['user_pass'].encode())

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.