JSON structures in Smart Pascal

JSON is more or less built into the JavaScript virtual machine [JSVM] which makes it very convenient to use. A simple call to JSON.stringify() is just about all you need to turn any JS object into a pristine JSON structure. And to turn it back again into an instance you just call JSON.parse().

Now arguing that JSON could be made simpler or in any way improved within Smart Mobile Studio would be, well, a bit lame. But what can be done is to abstract the use of JSON a bit. Most SMS coders are working quite close to the JSVM, mixing object pascal with JavaScript. That is one of the strengths of Smart Pascal, but it can also be regarded as a weakness.

It doesnt get easier than this, or does it..

If you ever used Turbo Pascal, High Speed Pascal or any of the “old time” compilers, you no doubt remember what happened when Delphi 1 came out? All that code written for Turbo became obsolete faster than it had to. Why? Because back then we mixed and matched Pascal with assembler. And when you wanted to move that code from DOS to Windows, you could kiss old-school assembler good-bye. The registers didn’t match up, the new platform was 32 bit, the way Delphi and Turbo used the stack and organized instances was different — long story short: you couldn’t just copy the code over.

While JavaScript is not going to change any time soon, it will not stay the same forever. And with that in mind I try my best to abstract the RTL as much as possible from the low-level code that actually get’s the job done. Who knows, one day we may actually turn around and implement that LLVM compiler that’s been floating around 😉

Abstraction

One of the units that is in the new RTL deals with structures. Not just JSON but also pure binary structures just like Delphi or Freepascal would allocate.

So there is an abstract baseclass simply called TW3Structure in the unit “System.Structure.pas” (yeah i know, the names are cunning). And it simply looks like this:

  EW3Structure = class(EW3Exception);
  TW3Structure = class(TObject)
  public
    procedure   WriteString(Name: string; Value: string; const Encode: boolean);
    procedure   WriteInt(const Name: string; value: integer);
    procedure   WriteBool(const Name: string; value: boolean);
    procedure   WriteFloat(const Name: string; value: float);
    procedure   WriteDateTime(const Name: string; value: TDateTime);

    function    ReadString(const Name: string): string;
    function    ReadInt(const Name: string): integer;
    function    ReadBool(const Name: string): boolean;
    function    ReadFloat(const Name: string): float;
    function    ReadDateTime(const Name: string): TDateTime;

    function    Read(const Name: string): variant;virtual;abstract;
    procedure   Write(const Name: string; const Value: variant);virtual;abstract;

    procedure   Clear;virtual;abstract;

    procedure   SaveToStream(const Stream: TStream);virtual;abstract;
    procedure   LoadFromStream(const Stream: TStream);virtual;abstract;
  end;

Next you have the JSON implementation in System.Structure.JSON, the XML version in System.Structure.XML – and raw binary in System.Structure.binary.

JSON serialization without baggage

When serializing with JSON we have so far operated either with variants (which you can turn into a JSON object with TVariant.CreateObject) or records. This is because raw serialization affects everything, including the VMT (virtual method table). So if you serialize a TObject based instance, it will include the VMT and whatever state that is in. Which will cause problems when de-serializing the object back depending on the complexity of the class.

In the last major update Eric (our compiler wizard) introduced classes that does not root in TObject. As you probably know, TObject has been the absolute base object in object-pascal for ages. You simply could not create a class that does not inherit from TObject.

Well, that is no longer the case and Smart Pascal allows you to define classes that are not based on TObject at all. This class form serves an important purpose: namely that they can be defined as “external”, so you can map these to JavaScript code directly. Like say a library you have included that exposes objects you can create. If you know the structure, then just define a class for it – and you can create it like any other object (!)

But what about creating pure JS based objects via code? Not defined classes or typed structures, but using code to carve out objects from scratch? Imagine it as being able to define what a record should look like – but at runtime. Sounds interesting? Indeed it does. And without much fuss I give you the TW3JSONObject class.

While somewhat pointless at first sight, it does make writing packets easier for NodeJS services. And extracting info is likewise a snap:

type

  TW3JSONEnumProc = function (Id: string; Data: variant): boolean;

  TW3JSONObject = class
  private
    function GetPropertyCount: integer;
  public
    function    ToString: string;
    procedure   FromString(const JSonText: string);
    function    Equals(const OtherObject: TW3JSONObject): boolean;
    function    Compare(const OtherObject: TW3JSONObject): boolean;

    function    &Read(const PropertyName: string): variant;
    procedure   &Write(const PropertyName: string; const PropertyData: variant);overload;
    function    AddOrSet(const PropertyName: string; const PropertyData: variant): TW3JSONObject;

    function    QueryPropertyNames: TStrArray;
    procedure   Clear;

    function    ForEach(const Callback: TW3JSONEnumProc): boolean;overload;

    function    &Contains(const PropertyName: string): boolean;overload;

    class function ForEach(const ObjReference: variant;
      const Callback: TW3JSONEnumProc): boolean;overload;

    class function &Contains(const ObjReference: variant;
      const PropertyName: string): boolean;overload;

    property    Values[const Id:string]: variant read &Read write &Write;default;
    property    Count:integer read GetPropertyCount;

    constructor Create;overload;virtual;
    constructor Create(const ObjReference: JObject);overload;virtual;
    constructor Create(const FromPrototype: Variant);overload;virtual;

    destructor Destroy;override;
  end;

{$R 'object.keys.shim.js'}

var
__RESERVED: array of string = [
          '$ClassName',
          '$Parent',
          '$Init',
          'toString',
          'toLocaleString',
          'valueOf',
          'indexOf',
          'hasOwnProperty',
          'isPrototypeOf',
          'propertyIsEnumerable',
          'constructor'
        ];

//############################################################################
// TW3JSONObject
//###########################################################################

constructor TW3JSONObject.Create;
begin
  inherited Create;
end;

constructor TW3JSONObject.Create(const ObjReference: JObject);
begin
  Create(Variant(ObjReference));
end;

constructor TW3JSONObject.Create(const FromPrototype: Variant);
begin
  Create;

  (* perform a non-recursive prototype clone of values *)
  if FromPrototype.Valid then
  begin
    ForEach(FromPrototype,
      function (Id: string; Data: variant): boolean
      begin
        if not (id in __RESERVED) then
        AddOrSet(Id,Data);
        result := true;
      end);
  end;
end;

destructor TW3JSONObject.Destroy;
begin
  clear;
  inherited;
end;

class function TW3JSONObject.&Contains(const ObjReference: variant;
      const PropertyName: string): boolean;
begin
  result := ObjectPrototype.hasOwnProperty.call(PropertyName);
end;

class function TW3JSONObject.ForEach(const ObjReference: variant;
      const Callback: TW3JSONEnumProc): boolean;
begin
  if assigned(Callback) then
  begin
    if  ObjReference.Valid then
    begin
      if ObjReference.IsObject then
      begin
        for var LItem in ObjReference do
        begin
          result := Callback(LItem, ObjReference[LItem]);
          if not result then
          break;
        end;
      end else
      raise Exception.Create('Enumeration failed, render is not an object error');
    end else
    Raise Exception.Create('Enumeration failed, reference is null or unassigned error');
  end;
end;

function TW3JSONObject.ForEach(const Callback: TW3JSONEnumProc): boolean;
begin
  if assigned(callback) then
  begin
    for var LItem in ThisContext do
    begin
      result := Callback(LItem, ThisContext[LItem]);
      if not result then
      break;
    end;
  end;
end;

procedure TW3JSONObject.Clear;
begin
  var LItems := QueryPropertyNames;
  for var LProp in LItems do
  begin
    asm
      delete this[@LProp];
    end;
  end;
end;

function TW3JSONObject.QueryPropertyNames: TStrArray;
begin
  (* Perform manual enumeration *)
  for var LItem in ThisContext do
  begin
    // Avoid standard items
    if not (LItem in __RESERVED) then
    begin
      if not (Variant(ThisContext[LItem]).Datatype in [vdUnknown, vdFunction]) then
      begin
        if ThisContext.hasOwnProperty.call(ThisContext,LItem) then
          result.add(LItem);
      end;
    end;
  end;
end;

function TW3JSONObject.GetPropertyCount: integer;
begin
  if (ObjectPrototype.keys) then
  begin
    (* Object.Keys() is supported by all modern browsers, but equally
       important: NodeJS and IO as well.
       There are however older runtime environments in circulation, so we need
       fallback mechanisms to ensure behavior. *)
    result := ObjectPrototype.keys.call(ThisContext).length;
  end else
  begin
    for var LItem in ThisContext do
    begin
      if ThisContext.hasOwnProperty.call(ThisContext,LItem) then
        inc(result);
    end;
  end;
end;

function TW3JSONObject.&Contains(const PropertyName: string): boolean;
begin
  result := thisContext.hasOwnProperty.call(PropertyName);
end;

function TW3JSONObject.Read(const PropertyName: string): variant;
begin
  result := ThisContext[PropertyName];
end;

procedure TW3JSONObject.Write(const PropertyName: string;
  const PropertyData: variant);
begin
  ThisContext[PropertyName] := PropertyData;
end;

function  TW3JSONObject.AddOrSet(const PropertyName: string;
      const PropertyData: variant): TW3JSONObject;
begin
  ThisContext[PropertyName] := PropertyData;
  result := self;
end;

function TW3JSONObject.ToString: string;
begin
  result := JSON.Stringify(self);
end;

(* Compare does a property-name and property-value check *)
function TW3JSONObject.Compare(const OtherObject: TW3JSONObject): boolean;
begin
  if assigned(otherobject) then
  begin
    result := ForEach(
      function (Name: string; Data: variant): boolean
      begin
        result := ObjectPrototype.hasOwnProperty.call(OtherObject,Name)
        and OtherObject.Read(Name) = Read(Name);
      end);
  end;
end;

(* Equals does a structural compatability check only *)
function TW3JSONObject.Equals(const OtherObject: TW3JSONObject): boolean;
begin
  if assigned(otherobject) then
  begin
    result := ForEach(
      function (Name: string; Data: variant): boolean
      begin
        result := ObjectPrototype.hasOwnProperty.call(OtherObject,Name)
      end);
  end;
end;

procedure TW3JSONObject.FromString(const JSONText: string);
var
  LTemp: variant;
  LObj: variant;
begin
  Clear;
  LObj := JSON.Parse(JSONText);
  if (LObj) then
  begin
    for LTemp in LObj do
    begin
      if not (LTemp in __RESERVED) then
        ThisContext[LTemp] := LObj[LTemp];
    end;
  end;
end;

Here is a little example code of how it’s used:

  var raw := TW3JSONObject.Create;
  raw.Write("hello",true);
  raw.Write("values",1200);
  raw.Write("SubElement",TW3JSonObject.Create);
  raw.AddOrSet("Another!",TW3JSonObject.Create)
     .AddOrSet("Yet another!", TW3JSonObject.Create);

And extracting property information is likewise easy:

  var props := raw.QueryPropertyNames.join(",");
  showmessage(props);