I am trying to urlencode this string before I submit.
queryString = 'eventName=' + evt.fields["eventName"] + '&' + 'eventDescription=' + evt.fields["eventDescription"];
What you're looking for is urllib.quote_plus
:
safe_string = urllib.quote_plus('string_of_characters_like_these:$#@=?%^Q^$')
#Value: 'string_of_characters_like_these%3A%24%23%40%3D%3F%25%5EQ%5E%24'
In Python 3, the urllib
package has been broken into smaller components. You'll use urllib.parse.quote_plus
(note the parse
child module)
import urllib.parse
safe_string = urllib.parse.quote_plus(...)
import urllib.parse ... urllib.parse.quote_plus(query)
python3 -c "import urllib.parse, sys; print(urllib.parse.quote_plus(sys.argv[1])) "string to encode"
for a one liner on the command line
Commented
May 8, 2019 at 12:19
"
just after the double round closing brackets ))
, it should be: python3 -c "import urllib.parse, sys; print(urllib.parse.quote_plus(sys.argv[1]))" "string to encode"
You need to pass your parameters into urlencode()
as either a mapping (dict), or a sequence of 2-tuples, like:
>>> import urllib
>>> f = { 'eventName' : 'myEvent', 'eventDescription' : 'cool event'}
>>> urllib.urlencode(f)
'eventName=myEvent&eventDescription=cool+event'
Python 3 or above
>>> urllib.parse.urlencode(f)
eventName=myEvent&eventDescription=cool+event
Note that this does not do url encoding in the commonly used sense (look at the output). For that use urllib.parse.quote_plus
.
Try requests instead of urllib and you don't need to bother with urlencode!
import requests
requests.get('http://youraddress.com', params=evt.fields)
EDIT:
If you need ordered name-value pairs or multiple values for a name then set params like so:
params=[('name1','value11'), ('name1','value12'), ('name2','value21'), ...]
instead of using a dictionary.
The following is a complete solution, including how to deal with some pitfalls.
### ********************
## init python (version 2.7.2 )
import urllib
### ********************
## first setup a dictionary of name-value pairs
dict_name_value_pairs = {
"bravo" : "True != False",
"alpha" : "http://www.example.com",
"charlie" : "hello world",
"delta" : "1234567 !@#$%^&*",
"echo" : "[email protected]",
}
### ********************
## setup an exact ordering for the name-value pairs
ary_ordered_names = []
ary_ordered_names.append('alpha')
ary_ordered_names.append('bravo')
ary_ordered_names.append('charlie')
ary_ordered_names.append('delta')
ary_ordered_names.append('echo')
### ********************
## show the output results
if('NO we DO NOT care about the ordering of name-value pairs'):
queryString = urllib.urlencode(dict_name_value_pairs)
print queryString
"""
echo=user%40example.com&bravo=True+%21%3D+False&delta=1234567+%21%40%23%24%25%5E%26%2A&charlie=hello+world&alpha=http%3A%2F%2Fwww.example.com
"""
if('YES we DO care about the ordering of name-value pairs'):
queryString = "&".join( [ item+'='+urllib.quote_plus(dict_name_value_pairs[item]) for item in ary_ordered_names ] )
print queryString
"""
alpha=http%3A%2F%2Fwww.example.com&bravo=True+%21%3D+False&charlie=hello+world&delta=1234567+%21%40%23%24%25%5E%26%2A&echo=user%40example.com
"""
urllib.parse.quote()
myself since it uses %20
rather than +
.
Try this:
urllib.pathname2url(stringToURLEncode)
urlencode
won't work because it only works on dictionaries. quote_plus
didn't produce the correct output.
my string
to my%20string
. Your solution works like a charm for that!
Note that the urllib.urlencode does not always do the trick. The problem is that some services care about the order of arguments, which gets lost when you create the dictionary. For such cases, urllib.quote_plus is better, as Ricky suggested.
>>> import urllib >>> urllib.urlencode([('name', 'brandon'), ('uid', 1000)]) 'name=brandon&uid=1000'
Commented
Mar 16, 2017 at 21:07
In Python 3, this worked with me
import urllib
urllib.parse.quote(query)
import urllib.parse
, otherwise I get "AttributeError: module 'urllib' has no attribute 'parse'".
Commented
Nov 28, 2023 at 16:41
import urllib.parse
query = 'Hellö Wörld@Python'
urllib.parse.quote(query) # returns Hell%C3%B6%20W%C3%B6rld%40Python
urllib.parse.quote
is already mentioned in this earlier answer.
Commented
Nov 19, 2021 at 3:38
import urllib
when it should be urllib.parse
. Yes, they could also have suggested an edit, but as it is now, this answer is the most correct of the two.
Commented
Nov 28, 2023 at 16:46
for future references (ex: for python3)
>>> import urllib.request as req
>>> query = 'eventName=theEvent&eventDescription=testDesc'
>>> req.pathname2url(query)
>>> 'eventName%3DtheEvent%26eventDescription%3DtestDesc'
'c:/2 < 3'
on Windows is '///C://2%20%3C%203'
. I want something that would just output 'c:/2%20%3C%203'
.
The syntax is as follows :
import urllib3
urllib3.request.urlencode({"user" : "john" })
For use in scripts/programs which need to support both python 2 and 3, the six module provides quote and urlencode functions:
>>> from six.moves.urllib.parse import urlencode, quote
>>> data = {'some': 'query', 'for': 'encoding'}
>>> urlencode(data)
'some=query&for=encoding'
>>> url = '/some/url/with spaces and %;!<>&'
>>> quote(url)
'/some/url/with%20spaces%20and%20%25%3B%21%3C%3E%26'
Another thing that might not have been mentioned already is that urllib.urlencode()
will encode empty values in the dictionary as the string None
instead of having that parameter as absent. I don't know if this is typically desired or not, but does not fit my use case, hence I have to use quote_plus
.
For Python 3 urllib3 works properly, you can use as follow as per its official docs :
import urllib3
http = urllib3.PoolManager()
response = http.request(
'GET',
'https://api.prylabs.net/eth/v1alpha1/beacon/attestations',
fields={ # here fields are the query params
'epoch': 1234,
'pageSize': pageSize
}
)
response = attestations.data.decode('UTF-8')
If you don't want to use urllib.
https://github.com/wayne931121/Python_URL_Decode
URL_RFC_3986 = {
"!": "%21", "#": "%23", "$": "%24", "&": "%26", "'": "%27", "(": "%28", ")": "%29", "*": "%2A", "+": "%2B",
",": "%2C", "/": "%2F", ":": "%3A", ";": "%3B", "=": "%3D", "?": "%3F", "@": "%40", "[": "%5B", "]": "%5D",
}
def url_encoder(b):
# https://zh.wikipedia.org/wiki/%E7%99%BE%E5%88%86%E5%8F%B7%E7%BC%96%E7%A0%81
if type(b)==bytes:
b = b.decode(encoding="utf-8") #byte can't insert many utf8 charaters
result = bytearray() #bytearray: rw, bytes: read-only
for i in b:
if i in URL_RFC_3986:
for j in URL_RFC_3986[i]:
result.append(ord(j))
continue
i = bytes(i, encoding="utf-8")
if len(i)==1:
result.append(ord(i))
else:
for c in i:
c = hex(c)[2:].upper()
result.append(ord("%"))
result.append(ord(c[0:1]))
result.append(ord(c[1:2]))
result = result.decode(encoding="ascii")
return result
#print(url_encoder("我好棒==%%0.0:)")) ==> '%E6%88%91%E5%A5%BD%E6%A3%92%3D%3D%%0.0%3A%29'
urllib parse wasn't quite sufficient in my case.
Two additions were needed:
use safe
kwarg to unmark /
as a safe character (urllib.parse.quote(string, safe='')
)
use .replace() for remaining special characters - in particular, note that javascript's encodeURIComponent and C# Uri.EscapeDataString behave differently. For working with a C# backend that expects EscapeDataStrings, then the following all need to be additionally replaced: _-.~
Ultimately I have something like this
import urllib
def aggressive_urlencode(inp: str) -> str:
return (
urllib.parse.quote(inp, safe='')
.replace('-', '%2D')
.replace('.', '%2E')
.replace('_', '%5F')
.replace('~', '%7E')
)