I am streaming data from a sensor using socket library in Json format, and trying to parse it and load it into Database. When I print the stream I get this Json in this format:
b'[{"metadata":{"timezone":{"location":"Etc/UTC"},"serial_number":"00:07:32:52:09:fc","device_type":"SPIDER"},"timestamp":"2019-08-29T13:53:05.895Z","framenumber":"2290718","tracked_objects":[{"id":2592,"is_at_border":true,"type":"PERSON","position":{"x":233,"y":262,"type":"FOOT","coordinate_system":"PROCESSING_IN_PIXEL"},"person_data":{"height":1728}}]}]'
Based on my research prefix b stands for the byte type. So when I try to parse it with code below:
while True:
message, address = server_socket.recvfrom(1024)
message = message.upper()
# loading json file.
objs_json = json.loads(message)
# using if looop to prevent script of trying to to parse data without any object being tracked.
if "tracked_objects" in objs_json:
# Parsing json file with json_normalize object
objs_df = json_normalize(
objs_json, record_path='tracked_objects',
meta=[['metadata', 'serial_number'], 'timestamp']
)
# Renaming columns
objs_df = objs_df.rename(
columns={
"id": "object_id", "position.x": "x_pos",
"position.y": "y_pos", "person_data.height": "height",
"metadata.serial_number": "serial_number",
"timestamp": "timestamp"
}
)
# Selecting columns of interest
objs_df = objs_df.loc[:, ["timestamp", "serial_number", "object_id", "x_pos", "y_pos", "height"]]
# Writting the data into SQlite db
objs_df.to_sql('data_object', con=engine, if_exists='append', index=False)
# In case there is no tracks, print No Tracks in console.
else:
print("No Tracks")
I get this error message:
Traceback (most recent call last):
File "/home/pi/ProRail-PMS/Test_Spider2.py", line 20, in <module>
objs_json = json.loads(message)
File "/usr/lib/python3.7/json/__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.7/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.7/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 215 (char 214)
However if I save that data into json file and remove the prefix b my parsing code works. How do i go around this so when I receive the data from socket library I want to be able to parse it and feed it into database?