0

Say, that we have the following text:

Something1 { "prop": "value" } Something2 <?xml version="1.0" encoding="UTF-8"?>
<root><subnode /></root> Something3

Is there a simple way to break this string into parts:

  • Text, Something1
  • JSON, { "prop": "value" }
  • Text, Something2
  • XML, <?xml version="1.0" encoding="UTF-8"?> <root><subnode /></root>
  • Text, Something3

I'm thinking of JSON and XML formats only.

Currently I have two ideas:

  • Parse string character by character; if the character is '{', check if it is JSON manually (parse) and if so, parse using, say, Newtonsoft JSON. If the character is '<', check if it is XML manually (parse) and if so, parse using System.XML. The downside is that I'll have to implement (simplified) parsers of JSON and XML, and the second one may be a tough one (control sequences etc.)
  • Since it is clear, when JSON and XML ends, JSON and XML deserializers should be able to say "parseable data ends here". If so, when encountering '{' or '<', I can run them, extract, where they left off and continue.

I treat first option as a last resort - this is a lot of work and I'm not sure I'll cover all JSON/XML caveats. Is the second possible (especially using System.XML and some popular JSON deserializer, such as Newtonsoft.JSON)?

Or is there a better way of solving this problem?

3
  • This feels like a XY Problem - meta.stackexchange.com/questions/66377/what-is-the-xy-problem . Why do you need to do this? Where did the data come from in that format?
    – mjwills
    Commented Feb 20, 2018 at 9:10
  • With some efforts you should be able to implement option 2. For example: JToken.ReadFrom() (from Newtonsort.JSON library) will not throw exception you feed it json object followed by anything - it will only parse json object.
    – Evk
    Commented Feb 20, 2018 at 9:39
  • @mjwillis This data is a log entry, which may look like following: Server response: {"field": "value"}. I'd like to break it to parts, format automatically and visualize. Since my application is used to display logs in general, I have no way of knowing, what format log entries will have.
    – Spook
    Commented Feb 20, 2018 at 9:39

2 Answers 2

0

Splitting this particular text into its parts is easy. Writing a program that takes a set of possible inputs similar to this one and performs similar processing on each of them is trickier: because it depends on how "similar" they are.

Specifying a class of "similar" inputs is done by writing a grammar in BNF, and splitting texts that conform to that grammar is done by writing a parser. This is all well-established computer science, well within what is taught in any undergraduate course. Write an unambiguous BNF specification of the set of messages you want to handle, and the rest is plain sailing.

Unfortunately it isn't common practice for parsers for languages such as JSON or XML to leave the input stream in a state where you can continue reading from the point where the JSON or XML finished (on the contrary, they will throw an error if there is any more content beyond the end of the JSON or XML structure). It would sometimes be a handy feature, but it's not usually available.

1
  • "(on the contrary, they will throw an error if there is any more content beyond the end of the JSON or XML structure)" - I actually used it to implement a solution :)
    – Spook
    Commented Feb 20, 2018 at 18:23
0

I managed to implement a solution. In a nutshell, when it encounteres { or < character, it uses Json or XML parser to parse it from that point onwards. If Json or XML parser throws an exception, I extract line and position on which exception occurred, then cut new part of processed string - from start position to one pointed to by the exception and try parsing again. If I fail, I simply accept { or < as a regular character and move on.

I guess it won't work in every single case and sometimes it may yield unexpected results (eg. JSON inside XML), but for my needs it's enough.

Uses ValueTuple. You'll have to implement your own containers for recognized parts as well (SimpleTextPart, XmlTextPart, JsonTextPart).

private int Move(string text, int current, int line, int position)
{
    while (current < text.Length && (line > 1 || position > 1))
    {
        if (text[current] == '\n')
        {
            if (line == 1)
                return -1;

            line--;
        }
        else if (line == 1)
        {
            position--;
        }

        current++;
    }

    return current;
}

private (bool jsonParseResult, BaseTextPart part, int newIndex) TryParseJson(string text, int current)
{
    try
    {
        string textPart = text.Substring(current);

        JObject obj = JObject.Parse(textPart);

        return (true, new JsonTextPart(obj), text.Length);
    }
    catch (JsonReaderException e)
    {
        int end = Move(text, current, e.LineNumber, e.LinePosition);

        try
        {
            string textPart = text.Substring(current, end - current);

            JObject obj = JObject.Parse(textPart);

            return (true, new JsonTextPart(obj), end);
        }
        catch (JsonReaderException)
        {
            return (false, null, 0);
        }
    }
}

private (bool xmlParseResult, BaseTextPart part, int newIndex) TryParseXml(string text, int current)
{
    XmlDocument doc = new XmlDocument();

    try
    {
        string textPart = text.Substring(current);

        doc.Load(new StringReader(textPart));    

        return (true, new XmlTextPart(doc), text.Length);
    }
    catch (XmlException e)
    {
        int end = Move(text, current, e.LineNumber, e.LinePosition);

        try
        {
            string textPart = text.Substring(current, end - current);

            doc.Load(new StringReader(textPart));

            return (true, new XmlTextPart(doc), end);
        }
        catch (XmlException)
        {
            return (false, null, 0);
        }
    }
}


private List<BaseTextPart> Parse(string text)
{
    var result = new List<BaseTextPart>();

    int current = 0;
    StringBuilder buffer = new StringBuilder();
    while (current < text.Length)
    {
        if (text[current] == '{')
        {
            (bool jsonParseResult, BaseTextPart part, int newIndex) = TryParseJson(text, current);

            if (jsonParseResult)
            {
                if (buffer.Length > 0)
                {
                    result.Add(new SimpleTextPart(buffer.ToString()));
                    buffer.Clear();
                }

                result.Add(part);
                current = newIndex;
                continue;
            }
        }

        if (text[current] == '<')
        {
            (bool xmlParseResult, BaseTextPart part, int newIndex) = TryParseXml(text, current);

            if (xmlParseResult)
            {
                if (buffer.Length > 0)
                {
                    result.Add(new SimpleTextPart(buffer.ToString()));
                    buffer.Clear();
                }

                result.Add(part);
                current = newIndex;
                continue;
            }
        }

        buffer.Append(text[current]);
        current++;
        continue;
    }

    if (buffer.Length > 0)
    {
        result.Add(new SimpleTextPart(buffer.ToString()));
        buffer.Clear();
    }

    return result;
}

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.