I have events being received in Azure Event Hub which are in the Format of IBM MQ Text. Source is Mainframe Oracle tables so data is being routed through Oracle Golden Date on the IBM MQ and arrives in Event Hub. I am interested in the body of the message which has an Event Name followed by Delimiter and also Tags which are followed by delimited data. There are no column names associated with the data in the message. The idea is to be able parse, map and further transform this data and make it available to other applications. The problem is how to Parse the Body of the event message which has delimited text and map it to a fixed schema using Azure Data Bricks or Stream Analytics. Is it only possible by creating a custom parser in Python or Java or another coding language?
Here is the body of a sample message -
{"GDSFE001\u00031\u0003N\u00030\u0003Confirm_Shipment_Closed\u00035572214\u0003B\u0003I7EPM0XV1Z8KB\u0003TAG0000\u0001\u000220190516\u00011409\u0001GCSS\u0001Message Broker\u0001\u0001\u0001\u0001O\u0001\u0001\u0001N\u0001BKG\u0001\u0001\u0001\u000163.0\u0002TAGT100\u0001HDZKG4XV1Z9KB\u0001BNILG4XV1Z9KB\u0001\u0001\u0001N\u0001N\u00010\u0001Y\u0001N\u00010\u00010\u0001SGPPM0XV1Z8KB\u0002TAG0100\u0001I7EPM0XV1Z8KB\u0001\u0001\u0001O\u0001\u0001\u00011\u0001Order Handling\u00011\u0001Active\u00013\u0001Ordinary Transport Order\u00012019-05-16 13.49.24.683955\u0001NayanKumar\u00012019-05-16 14.09.32.539936\u0001NayanKumar\u00012019-05-16 14.09.32.539936\u0001NayanKumar\u00011\u0001MSL\u00012019-05-16 13.49.24.683955\u0001NayanKumar\u00012019-05-16 14.09.32.539936\u0001NayanKumar\u0001\u0001Y\u0001N\u00011\u0001Booking Confirmation\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u00012019-05-16 13.49.24.683955\u0001NayanKumar\u0001\u0001\u0001\u0001\u0001\u0001\u0001A1\u0001\u0001\u0001\u0001T2\u00015\u0001GCSS\u00011\u0001CY\u00011\u0001CY\u0001\u0001\u0001\u00014\u0001Telephone\u0001\u00012019-05-09\u0001\u0001\u0001\u000110\u0001Transdoc Invoicing\u0001\u0001\u0001\u0001Y\u00012019-05-16 13.49.24.683955\u0001\u0001\u0001\u0001\u0001N\u0001\u0001\u0001Y\u0001SGPPM0XV1Z8KB\u0001\u00012019-05-09 21.00.00.000000\u0001T2\u0001233\u0001North Europe - United States\u0001MAEU\u0001\u0001\u0001\u0001N\u0001Y\u00012019-05-16 13.49.24.683955\u00011\u0001Fixed Date\u00016\u0001Failed\u0001W\u0001N\u0001Y\u0001N\u0001Y\u0001N\u0002TAG0110\u0001J20PM0XV1Z8KB\u0001\u0001\u0001O\u00011\u0001Booking Management\u00012019-05-16 13.49.24.683955\u0001\u0001\u0001\u0001\u00012019-05-16 13.49.24.683955\u0001NayanKumar\u00012019-05-16 13.54.16.577754\u0001AUTOUSER\u000112084\u0001Export Order Handling\u0001AUTOUSER\u0001AUTOUSER\u00011021\u0001AP Moller Copenhagen (MSL)\u0001panama_kvinder@hotmail.com\u0001DKCPHMSL1\u00013D9XA9RMF1PJ2\u0001DK\u0001Denmark\u0002TAG0120\u0001OKZHN0XV1Z8KB\u0001\u0001\u0001O\u00015HKDN0XV1Z8KB\u0001\u0001\u0001\u00010\u00011\u0001Booking Number\u0001\u0001\u0001510185665\u00012019-05-16 13.49.24.683955\u0001NayanKumar\u00012019-05-16 13.49.24.683955\u0001NayanKumar\u0001SGPPM0XV1Z8KB\u0001\u0001\u0001\u0001101\u0001Shipment\u0002TAG0120\u0001D8BMG4XV1Z9KB\u0001\u0001\u0001O\u0001\u0001\u0001\u0001\u00010\u00015\u0001Transport Document Number\u0001\u0001\u0001510185665\u00012019-05-16 13.51.42.301735\u0001NayanKumar\u00012019-05-16 13.51.42.301735\u0001NayanKumar\u0001\u0001\u0001BNILG4XV1Z9KB\u0001\u0001151\u0001Transport_Doc