2

While trying to access a file whose name contain utf-8 chars from browser I get the error

The requested URL /images/0/04/×¤×ª×¨×•× ×•×ª_תרגילי×_על_משטחי×_דיפ'_2014.pdf was not found on this server.

Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.`

In order to access the files I wrote the following python script:

# encoding: utf8
__author__ = 'Danis'
__date__ = '20/10/14'

import urllib

curr_link = u'http://math-wiki.com/images/0/04/2014_\'דיפ_משטחים_על_פתרונות.nn uft8pdf'

urllib.urlretrieve(link, 'home/danisf/targil4.pdf')

but when I run the code I get the error URLError:<curr_link appears here> contains non-ASCII characters

How can I fix the code to get him work? (by the way I don't have access to the server or to the webmaster) maybe the browser failed not because the bad encoding of the name for the file?

10
  • So, 404 is your expected result?
    – falsetru
    Oct 20, 2014 at 13:10
  • No. I want to get the file. receiving files with names like ex20-2014.pdf works fine through the browser. I want to get the file since I think something is corrupted with the GET in this html page. Oct 20, 2014 at 13:13
  • What the original url that you can download?
    – falsetru
    Oct 20, 2014 at 13:15
  • 'math-wiki.com/images/0/04/…' Oct 20, 2014 at 13:16
  • That does give me 404. (with / without trailing quote). Without knowing the exact url, how can you download it?
    – falsetru
    Oct 20, 2014 at 13:17

1 Answer 1

3

You cannot just pass Unicode URLs into urllib functions; URLs must be valid bytestrings instead. You'll need to encode to UTF-8, then url quote the path of your URL:

import urllib
import urlparse

curr_link = u'http://math-wiki.com/images/0/04/2014_\'דיפ_משטחים_על_פתרונות.nn uft8pdf'
parsed_link = urlparse.urlsplit(curr_link.encode('utf8'))
parsed_link = parsed_link._replace(path=urllib.quote(parsed_link.path))
encoded_link = parsed_link.geturl()

urllib.urlretrieve(encoded_link, 'home/danisf/targil4.pdf')

The specific URL you provided in your question produces a 404 error however.

Demo:

>>> import urllib
>>> import urlparse
>>> curr_link = u'http://math-wiki.com/images/0/04/2014_\'דיפ_משטחים_על_פתרונות.nn uft8pdf'
>>> parsed_link = urlparse.urlsplit(curr_link.encode('utf8'))
>>> parsed_link = parsed_link._replace(path=urllib.quote(parsed_link.path))
>>> print parsed_link.geturl()
http://math-wiki.com/images/0/04/2014_%27%D7%93%D7%99%D7%A4_%D7%9E%D7%A9%D7%98%D7%97%D7%99%D7%9D_%D7%A2%D7%9C_%D7%A4%D7%AA%D7%A8%D7%95%D7%A0%D7%95%D7%AA.nn%20uft8pdf

Your browser usually decodes UTF-8 bytes encoded like this, to present a readable URL, but when sending the URL to the server to retrieve, it is encoded in the exact same manner.

3
  • where the file will be saved? Oct 20, 2014 at 13:25
  • if the url redirects to 404, it means that the file doesn't exist? Oct 20, 2014 at 13:28
  • @DanisFischer: it means the file doesn't exist. What page was this linked from?
    – Martijn Pieters
    Oct 20, 2014 at 13:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.