54

I want to send JSON encoded data to a server using only native Python libraries. I love requests but I simply can't use it because I can't use it on the machine which runs the script. I need to do it without.

newConditions = {"con1":40, "con2":20, "con3":99, "con4":40, "password":"1234"} 
params = urllib.parse.urlencode(newConditions)
params = params.encode('utf-8')

req = urllib.request.Request(conditionsSetURL, data=params)
urllib.request.urlopen(req)        

My server is a local WAMP server. I always get an

urllib.error.HTTPError: HTTP Error 500: Internal Server Error

I am 100% sure that this is NOT a server issue, because the same data, with the same url, on the same machine, with the same server works with the requests library and Postman.

1 Answer 1

76

You are not posting JSON, you are posting a application/x-www-form-urlencoded request.

Encode to JSON and set the right headers:

import json

newConditions = {"con1":40, "con2":20, "con3":99, "con4":40, "password":"1234"} 
params = json.dumps(newConditions).encode('utf8')
req = urllib.request.Request(conditionsSetURL, data=params,
                             headers={'content-type': 'application/json'})
response = urllib.request.urlopen(req)

Demo:

>>> import json
>>> import urllib.request
>>> conditionsSetURL = 'http://httpbin.org/post'
>>> newConditions = {"con1":40, "con2":20, "con3":99, "con4":40, "password":"1234"} 
>>> params = json.dumps(newConditions).encode('utf8')
>>> req = urllib.request.Request(conditionsSetURL, data=params,
...                              headers={'content-type': 'application/json'})
>>> response = urllib.request.urlopen(req)
>>> print(response.read().decode('utf8'))
{
  "args": {}, 
  "data": "{\"con4\": 40, \"con2\": 20, \"con1\": 40, \"password\": \"1234\", \"con3\": 99}", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept-Encoding": "identity", 
    "Connection": "close", 
    "Content-Length": "68", 
    "Content-Type": "application/json", 
    "Host": "httpbin.org", 
    "User-Agent": "Python-urllib/3.4", 
    "X-Request-Id": "411fbb7c-1aa0-457e-95f9-1af15b77c2d8"
  }, 
  "json": {
    "con1": 40, 
    "con2": 20, 
    "con3": 99, 
    "con4": 40, 
    "password": "1234"
  }, 
  "origin": "84.92.98.170", 
  "url": "http://httpbin.org/post"
}
8
  • 2
    Where is the type of request (i.e. post) identified?
    – Startec
    Jul 21, 2015 at 7:44
  • 6
    @Startec: by using the data parameter the Request object switches from GET to POST.
    – Martijn Pieters
    Jul 21, 2015 at 7:45
  • 1
    @Startec: You can't. See How to make HTTP DELETE method using urllib2? for work-arounds.
    – Martijn Pieters
    Jul 21, 2015 at 7:48
  • 2
    @Startec: newConditions is a Python dictionary, not a JSON string.
    – Martijn Pieters
    Aug 19, 2015 at 19:29
  • 3
    @Startec: no, json.dumps() produces one JSON encoded string, in which the integers are still (JSON) integers. Any compliant JSON decoder will give you integers again.
    – Martijn Pieters
    Aug 21, 2015 at 21:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.