3

I need to deserialize raw binary data (BinaryFormatter), then serialize into JSON (for editing) and then serialize it back into binary again. Obviously, I lose on floats. Original float value 0xF9FF4FC1 (big endian, roughly -12.9999933) gets rounded to 0xF6FF4FC1 (-12.99999) when I serialize from original binary (correct data and intermediated data is 1:1 in memory) to JSON. I know it is not a big loss and I know floats are problematic but I want to keep the precision as close as possible due to possible incompatibility issues later.

Anyone tackled this problem before with JSON? How can I force it to write float with max precision? I've tried built in option for handling floats as either decimal or double but there is no difference in output, unfortunately, and I cant change target values because they still need to be written as floats when I do binary serialization so there will be rounding regardless during implicit conversion.

The specific type containing floats I am trying to round-trip is Vector2 from https://github.com/FNA-XNA/FNA/blob/master/src/Vector2.cs.

tl:dr have a float, want JsonNET serialize it as precise as possible into final json string.

P.S. I'e read tons of questions here and blog entries elsewhere but haven't found anyone trying to solve the same issue, most of the search hits were with float-reading issues (which I'm gonna need to solve later on too).

UPDATE: As @dbc below pointed out - Jsont.NET respects "TypeConverter" attribute thus I had to make my own converter that overrides it.

12
  • 2
    Swearing is not okay on SO. Commented Aug 6, 2018 at 8:32
  • The question I tagged as a duplicate is the opposite case - restricting the number of decimal places. But both are controlling the number of decimal places. Instead of ` writer.WriteRawValue(JsonConvert.ToString(value));` you'd use ` writer.WriteRawValue(((float)value).ToString('r'));` for the roundtrip format. Commented Aug 6, 2018 at 8:39
  • 1
    Can't reproduce, see dotnetfiddle.net/qBqBdW. I think we may need to see a minimal reproducible example to help you. Also, what version of Json.NET are you using?
    – dbc
    Commented Aug 6, 2018 at 9:16
  • 1
    Oh darn, you are right it DOES have TypeConverter: https://github.com/FNA-XNA/FNA/blob/c77c82837af89e28e9e71106a3637236c215ada3/src/Vector2.cs --- is there a way to bypass it/ignore?
    – KreonZZ
    Commented Aug 6, 2018 at 9:40

1 Answer 1

3

Json.NET will serialize float values using the round-trip precision format "R" (source). However, the type you are using, https://github.com/FNA-XNA/FNA/blob/master/src/Vector2.cs, has a TypeConverter applied:

    [Serializable]
    [TypeConverter(typeof(Vector2Converter))]
    [DebuggerDisplay("{DebugDisplayString,nq}")]
    public struct Vector2 : IEquatable<Vector2>
    {
        //...

As explained in the Newtonsoft docs, such types will be serialized as a string using the converter:

Primitive Types

.Net: TypeConverter (convertible to String)
JSON: String

And, inspecting the code for https://github.com/FNA-XNA/FNA/blob/master/src/Design/Vector2Converter.cs, it appears this converter is not using round-trip precision format when converting to string, at around line 60:

    return string.Join(
        culture.TextInfo.ListSeparator,
        new string[]
        {
            vec.X.ToString(culture),
            vec.Y.ToString(culture)
        }
    );

Thus the built-in TypeConverter itself is where you are losing precision.

To avoid this problem you can either

  1. Create a custom JsonConverter for Vector2 such as the following:

    public class Vector2Converter : JsonConverter
    {
        class Vector2DTO
        {
            public float X;
            public float Y;
        }
    
        public override bool CanConvert(Type objectType)
        {
            return objectType == typeof(Vector2) || objectType == typeof(Vector2?);
        }
    
        public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
        {
            // A JSON object is an unordered set of name/value pairs so the converter should handle
            // the X and Y properties in any order.
            var dto = serializer.Deserialize<Vector2DTO>(reader);
            if (dto == null)
                return null;
            return new Vector2(dto.X, dto.Y);
        }
    
        public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
        {
            var vec = (Vector2)value;
            serializer.Serialize(writer, new Vector2DTO { X = vec.X, Y = vec.Y });
        }
    }
    

    If you add the converter to JsonSerializerSettings.Converters it will supersede the TypeConverter.

    Working sample fiddle here.

    The converter serializes the Vector2 as an object with X and Y properties, but you could also serialize it as an array with two values if you prefer. I would not recommend serializing as a primitive string since Vector2 does not correspond to a JSON primitive.

  2. Use a custom contract resolver that does not create a primitive contract for types with TypeConverter applied, such as the one shown in the answer to Newtonsoft.JSON cannot convert model with TypeConverter attribute.

Notes:

  • The round-trip format "R" apparently does not preserve the difference between +0.0 and -0.0, and so Json.NET does not either, see https://dotnetfiddle.net/aereJ2 or https://dotnetfiddle.net/DwoGyX for a demo. The Microsoft docs make no mention of whether the zero sign should be preserved when using this format.

    Thus, this is one case where round-tripping a float might result in binary changes.

    (Thanks to @chux for raising this issue in comments.)

  • As an aside, Json.NET also uses "R" when writing double (as shown in the source for JsonConvert.ToString(double value, ...)):

    internal static string ToString(double value, FloatFormatHandling floatFormatHandling, char quoteChar, bool nullable)
    {
        return EnsureFloatFormat(value, EnsureDecimalPlace(value, value.ToString("R", CultureInfo.InvariantCulture)), floatFormatHandling, quoteChar, nullable);
    }
    

    For double only this format is documented to possibly lose precision. From The Round-trip ("R") Format Specifier:

    In some cases, Double values formatted with the "R" standard numeric format string do not successfully round-trip if compiled using the /platform:x64 or /platform:anycpu switches and run on 64-bit systems.

    Thus round-tripping a double via Json.NET might result in small binary differences. However, this does not strictly apply to the question here, which is specifically about float.

3

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.