0

I am trying to register a custom converter to avoid the inherited properties in TObjectList - mainly FListHelper and FOwnsObjects. But I cannot get the custom converter to register, and the documentation shows no real examples.

(No this is not a duplicate of: How to hide "ownsObjects" and "listHelper" TObjectList's properties from a Json using Delphi (Rest.JSON)?)

I am trying to get a converter Registered - but it seems to never run.

I have wrapped it in a custom class of mine that looks like this:

TMyJsonConverter = class(TJsonConverter)
  class function JsonConvert(ObjectToConvert:TObject): string;
  private
    type
      TListOfObjectInterceptor = class(TJSONInterceptor)
        function ObjectsConverter(Data: TObject; Field:string): TListOfObjects; override;
    end;
  end;


function TMyJsonConverter.TListOfObjectInterceptor.ObjectsConverter(Data: TObject; Field:string): TListOfObjects;
begin
  raise Exception.Create('converter found');
end;

class function TMyJsonConverter.JsonConvert(ObjectToConvert: TObject): string; 
begin
  var customConverter := TMyJsonConverter.TListOfObjectInterceptor.Create();
  var otherConverter := TMyJsonConverter.Create;
  var marshaller := TJSONMarshal.Create(otherconverter);

  marshaller.RegisterConverter(
    ObjectToConvert.ClassType,
    '*',
    customConverter.ObjectsConverter
  );

  var json := marshaller.Marshal(ObjectToConvert);

  try
    exit(json.ToString);
  finally
    marshaller.Free;
  end;
end;

I have tried to register the types of TObjectConverter, TObjectsConverter, TTypeObjectsConverter but I never seem to get into the conversion function. I can see that the call to register does register the converter, but when I marshal the JSON, it does not find the custom converter again.

Here is a sample structure that highlights the issue, I want to marshal TMySampleDTO as JSON:

type
  TEmployee = class
  public
    Id: Integer;
    Name: string;
  end;
  TEmployeeList = class(TObjectList<TEmployee>);

  TWorktime = class
  public
    EmployeeId: Integer;
    DepartmentId: Integer;
    StartTime: TDateTime;
    StopTime: TDateTime;
  end;
  TWorktimeList = class(TObjectList<TWorktime>);

  TDepartment = class
  public
    Id: Integer;
    Address: string;
    Employees: TEmployeelist;
  end;
  TDepartmentList = class(TObjectList<TDepartment>);

  TMySampleDTO = class
  public
    Departments: TDepartmentList;
    Worktimes: TWorktimeList;
    Employees: TEmployeeList;
  end;

UPDATE: I got the Converter to run, apparently even though Embarcadero defined the const FIELD_ANY as '*', it doesn't run if you don't specify the exact fieldname, in my case FListHelper. This raises the next issue though, I also have to give the exact type, as it doesn't check for inheritance. So if my object structure has properties derived from TObjectList<T> all those lists will be serialized as objects with a list as a property.

6
  • Could you provide sample data structure that you're trying to serialize to JSON?
    – Peter Wolf
    Sep 25 at 7:52
  • @PeterWolf I have added some data that might show my issue :)
    – Matt Baech
    Sep 25 at 8:06
  • Unfortunately not, I did read that answer thoroughly and it was what got me on the track of the custom converter. My issue is that I want this to be my generic converter for all my entities. And I must say that I think that it is ugly having to annotate all my models with JsonReflect attributes. So essentially I want to do what Stefan did in his answer, but without the attribute.
    – Matt Baech
    Sep 25 at 8:27
  • @RemyLebeau Thanks for the cleanup - however I didn't really answer the question, as mentioned it is now called but still does not work properly.
    – Matt Baech
    Sep 26 at 5:58

2 Answers 2

1

The documentation for TTypeMarshaller.RegisterConverter lacks any detailed information leaving you no option but to study Delphi RTL source code. At first look we can divide all overloaded versions of the method into 3 groups:

  1. An overload with 3 parameters with last parameter of type TConverterEvent - registers a converter, which performs conversion based on its ConverterType property. This property read-only, but its value is set along with setting any of its conversion delegates (*Converter properties).
  2. Overloads with 3 parameters with last parameter of type T...Converter - registers a converter for FieldName (2nd parameter) within class type clazz (1st parameter). Let's call it a "field converter".
  3. Overloads with 2 parameters with last parameter of type TType...Converter - registers a converter for class type clazz (1st parameter). This basically calls the first overload with FieldName set to '*', which is reserved for type converters.

In your sample code you are registering a field converter for field with name * within class type of an instance provided as an argument to your TMyJsonConverter.JsonConvert method. That explains why your conversion routine is not invoked.

Furthermore you create an instance TListOfObjectInterceptor in order to pass a reference to its ObjectsConverter method to TTypeMarshaller.RegisterConverter method. This way you leak this TListOfObjectInterceptor instance. Interceptors should be used in combination with JsonReflect attribute.

The correct way to register a field converter is:

function ListFieldConverter(Data: TObject; Field: string): TListOfObjects;
var
  RttiContext: TRttiContext;
  List: TList<TObject>;
begin
  List := TList<TObject>(RttiContext.GetType(Data.ClassInfo).GetField(Field).GetValue(Data).AsObject);
  Result := TListOfObjects(List.List);
  SetLength(Result, List.Count); // makes unique copy
end;

{ ... }

marshaller.RegisterConverter(TMySampleDTO, 'Employees', ListFieldConverter);
marshaller.RegisterConverter(TMySampleDTO, 'Departments', ListFieldConverter);
marshaller.RegisterConverter(TDepartment, 'Employees', ListFieldConverter);
marshaller.RegisterConverter(TMySampleDTO, 'Worktimes', ListFieldConverter);

The correct way to register a type converter is:

function ListTypeConverter(Data: TObject): TListOfObjects;
var
  List: TList<TObject>;
begin
  List := TList<TObject>(Data);
  Result := TListOfObjects(List.List);
  SetLength(Result, List.Count); // makes unique copy
end;

{ ... }

marshaller.RegisterConverter(TEmployeeList, ListTypeConverter);
marshaller.RegisterConverter(TDepartmentList, ListTypeConverter);
marshaller.RegisterConverter(TWorktimeList, ListTypeConverter);

As you can see, this is as much work as using JsonReflect attributes. In theory you could register converters dynamically based on RTTI, but this can be cumbersome due to nested types and inheritance.

To sum that up, even though Delphi JSON library provides ways to hook into the marshalling process, it's not very flexible and supports only very basic scenarios (see also recent question Spring4D Nullable JSON serialization).

The last thing I'd like to point out is that you should pay more attention to conventions. Fields of records/classes should be prefixed with F by convention. Delphi JSON library honors that and removes those F's in process of serialization. Serializing type TData = class Foo: string; end; to JSON yields: {"oo":""}.

1
  • I did somehow manage to get it to work using much the same method, but dynamic so I don't have to have hard coupling between my json converter and all entities in my entire codebase. As for your last point, that was a copy paste mistake and I have updated the example in my question. I will create a self answer with my new converter, but it still doesnt quite work, but it gets me 95% there, it still refuses to call the converter on the top level List object, but everything else is converted as i expect.
    – Matt Baech
    Sep 26 at 7:51
0

Here is something that works - for some reason it doesnt work when the top level object is a list, but I can live with that for now. It uses RTTI and some questionable methods for finding Lists in an object, but it works for me - somewhat.

If ObjectToConvert is a TList<T> descendant, it will still return json as an object with the property listHelper, containing the elements of the list.

But only for the top level object

If anyone can figure out why the converter doesn't work for the top list (outermost json value), feel free to tell me why, and I will add it to solution.

procedure FindObjectListsInHierarchy(obj: TObject; var result: TList<TClass>);

  function IsListDescendant(clazz:TClass) : boolean;
  begin
    result := false;
    while clazz <> nil do
    begin
      if clazz.ClassName.StartsWith('TList<') then
         exit(true);
      clazz := clazz.ClassParent;
    end;
  end;

begin
  if not assigned(result) then
    Result := TList<TClass>.Create;

  var ctx := TRttiContext.Create;
  var objType := ctx.GetType(obj.ClassType);

  for var prop in objType.GetProperties do
  begin
    if not ((prop.Visibility = mvPublic) or (prop.Visibility = mvPublished)) then
      continue;

    if prop.PropertyType.TypeKind in [tkClass, tkDynarray] then
    begin
      if IsListDescendant(TRttiInstanceType(Prop.Parent).MetaclassType) then
      begin
        if not result.Contains(obj.ClassType) then
          Result.Add(obj.ClassType);

        for var item in TList(Obj) do
          FindObjectListsInHierarchy(item, result);
      end;

      if prop.PropertyType.TypeKind = tkClass then
        FindObjectListsInHierarchy(prop.GetValue(obj).AsObject, result);
    end;
  end;
end;

function TMyJsonConverter.TypeObjectsConverter(Data: TObject): TListOfObjects;
begin
  var list := TList<TObject>(Data);
  Result := TListOfObjects(list.List);
  SetLength(Result, list.Count);
end;

function TMyJsonConverter.JsonConvert(ObjectToConvert: TObject): string;
begin
  var marshaller := TJSONMarshal.Create;
  try
    var listObjects: TList<TClass> := nil;
    try
      FindObjectListsInHierarchy(ObjectToConvert, listObjects);
      for var o in listObjects do
      begin
        marshaller.RegisterJSONMarshalled(o, 'FOwnsObjects', false);
        marshaller.RegisterConverter(o, TypeObjectsConverter);
      end;
    finally
      listObjects.Free;
    end;

    var json := marshaller.Marshal(ObjectToConvert);
    result := json.ToString;
    json.Free;
  finally
    marshaller.Free;
  end;
end;
7
  • The question was why your converter registration didn't work. In my answer I explained in detail where did you go wrong and how to do it correctly. I also advised you to drop usage of JsonInterceptor in combination with RegisterConverter which you seem to ignore.
    – Peter Wolf
    Sep 26 at 8:44
  • You have updated original post with new question which in my opinion should go to a separate post. Anyway, answer to that question is that Delphi JSON library is flawed and it will only work if you register interceptor using JsonReflect attribute with ctTypeObjects converter in your case, e.g.:[JsonReflect(ctTypeObjects, rtTypeObjects, TMyJsonInterceptor)] TEmployeeList = class(TObjectList<TEmployee>);. With that said, if you really need to use Delphi JSON library, I'd recommend you to mark all your custom list types with JsonReflect attribute and that's all you have to do.
    – Peter Wolf
    Sep 26 at 8:44
  • @PeterWolf My original question was how I got a converter to work with specifically descendants of TObjectList<T>, that I had an error in my converter registration was simply part of the solution. The JsonInterceptor was again an error on my side, since I saw that the TJsonInterceptor had virtual methods that matches the convertertion routines needed, I have now omitted that and will update my answer accordingly. I will not use attributes on hundreds or thousands of classes in a multi million line code base as I see no benefit of these classes to hold serialization specific knowledge.
    – Matt Baech
    Sep 26 at 9:40
  • @PeterWolf I DO recognize that the attributes are the intended way for this to work, but it does not solve my use case. Therefore this question.
    – Matt Baech
    Sep 26 at 9:41
  • That's the point - it's not possible with TJSONMarshal. You can check it yourself in TTypeMarshaller<TSerial>.MarshalData method (System.JsonReflect.pas). After checking for nil value it calls GetTypeConverter method to resolve type interceptor declared by JsonReflect attribute. If there's none, it ignores any registered converters and proceeeds directly with marshalling the type's fields.
    – Peter Wolf
    Sep 26 at 10:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.