Archive

Posts Tagged ‘Object Pascal’

A Delphi propertybag

July 7, 2019 8 comments

A long, long time ago, way back in the previous century, I often had to adjust a Visual Basic project my company maintained. Going from object-pascal to VB was more than a little debilitating; Visual Basic was not a compiled language like Delphi is, and it lacked more or less every feature you needed to produce good software.

source

I could probably make a VB clone using Delphi pretty easily. But I think the world has experienced enough suffering, no need to add more evil to the universe

Having said that, I have always been a huge fan of Basic (it was my first language after all, it’s what schools taught in the 70s and 80s). I think it was a terrible mistake for Microsoft to retire Basic as a language, because it’s a great way to teach kids the fundamentals of programming.

Visual Basic is still there though, available for the .Net framework, but to call it Basic is an insult of the likes of GFA Basic, Amos Basic and Blitz Basic; the mighty compilers of the past. If you enjoyed basic before Microsoft pushed out the monstrosity that is Visual Basic, then perhaps swing by GitHub and pick up a copy of BlitzBasic?  BlitzBasic is a completely different beast. It compiles to machine-code, allows inline assembly, and has been wildly popular for game developers over the years.

A property bag

The only feature that I found somewhat useful in Visual Basic, was an object called a propertybag. It’s just a fancy name for a dictionary, but it had a couple of redeeming factors beyond lookup ability. Like being able to load name-value-pairs from a string, recognizing datatypes and exposing type-aware read/write methods. Nothing fancy but handy when dealing with database connection-strings, shell parameters and the like.

So you could feed it strings like this:

first=12;second=hello there;third=3.14

And the class would parse out the names and values, stuff it in a dictionary, and you could easily extract the data you needed. Nothing fancy, but handy on rare occasions.

A Delphi version

Im mostly porting code from Delphi to Oxygene these days, but here is my Delphi implementation of the propertybag object. Please note that I haven’t bothered to implement the propertybag available in .Net. The Delphi version below is based on the Visual Basic 6 version, with some dependency injection thrown in for good measure.

unit fslib.params;

interface

{.$DEFINE SUPPORT_URI_ENCODING}

uses
  System.SysUtils,
  System.Classes,
  Generics.Collections;

type

  (* Exceptions *)
  EPropertybag           = class(exception);
  EPropertybagReadError  = class(EPropertybag);
  EPropertybagWriteError = class(EPropertybag);
  EPropertybagParseError = class(EPropertybag);

  (* Datatypes *)
  TPropertyBagDictionary = TDictionary ;

  IPropertyElement = interface
    ['{C6C937DF-50FA-4984-BA6F-EBB0B367D3F3}']
    function  GetAsInt: integer;
    procedure SetAsInt(const Value: integer);

    function  GetAsString: string;
    procedure SetAsString(const Value: string);

    function  GetAsBool: boolean;
    procedure SetAsBool(const Value: boolean);

    function  GetAsFloat: double;
    procedure SetAsFloat(const Value: double);

    function  GetEmpty: boolean;

    property Empty: boolean read GetEmpty;
    property AsFloat: double read GetAsFloat write SetAsFloat;
    property AsBoolean: boolean read GetAsBool write SetAsBool;
    property AsInteger: integer read GetAsInt write SetAsInt;
    property AsString: string read GetAsString write SetAsString;
  end;

  TPropertyBag = Class(TInterfacedObject)
  strict private
    FLUT:       TPropertyBagDictionary;
  strict protected
    procedure   Parse(NameValuePairs: string);
  public
    function    Read(Name: string): IPropertyElement;
    function    Write(Name: string; Value: string): IPropertyElement;

    procedure   SaveToStream(const Stream: TStream);
    procedure   LoadFromStream(const Stream: TStream);
    function    ToString: string; override;
    procedure   Clear; virtual;

    constructor Create(NameValuePairs: string); virtual;
    destructor  Destroy; override;
  end;

implementation

{$IFDEF SUPPORT_URI_ENCODING}
uses
  system.NetEncoding;
{$ENDIF}

const
  cnt_err_sourceparameters_parse =
  'Failed to parse input, invalid or damaged text error [%s]';

  cnt_err_sourceparameters_write_id =
  'Write failed, invalid or empty identifier error';

  cnt_err_sourceparameters_read_id =
  'Read failed, invalid or empty identifier error';

type

  TPropertyElement = class(TInterfacedObject, IPropertyElement)
  strict private
    FName:      string;
    FData:      string;
    FStorage:   TPropertyBagDictionary;
  strict protected
    function    GetEmpty: boolean; inline;

    function    GetAsInt: integer; inline;
    procedure   SetAsInt(const Value: integer); inline;

    function    GetAsString: string; inline;
    procedure   SetAsString(const Value: string); inline;

    function    GetAsBool: boolean; inline;
    procedure   SetAsBool(const Value: boolean); inline;

    function    GetAsFloat: double; inline;
    procedure   SetAsFloat(const Value: double); inline;

  public
    property    AsFloat: double read GetAsFloat write SetAsFloat;
    property    AsBoolean: boolean read GetAsBool write SetAsBool;
    property    AsInteger: integer read GetAsInt write SetAsInt;
    property    AsString: string read GetAsString write SetAsString;
    property    Empty: boolean read GetEmpty;

    constructor Create(const Storage: TPropertyBagDictionary; Name: string; Data: string); overload; virtual;
    constructor Create(Data: string); overload; virtual;
  end;

//#############################################################################
// TPropertyElement
//#############################################################################

constructor TPropertyElement.Create(Data: string);
begin
  inherited Create;
  FData := Data.Trim();
end;

constructor TPropertyElement.Create(const Storage: TPropertyBagDictionary;
  Name: string; Data: string);
begin
  inherited Create;
  FStorage := Storage;
  FName := Name.Trim().ToLower();
  FData := Data.Trim();
end;

function TPropertyElement.GetEmpty: boolean;
begin
  result := FData.Length < 1;
end;

function TPropertyElement.GetAsString: string;
begin
  result := FData;
end;

procedure TPropertyElement.SetAsString(const Value: string);
begin
  if Value  FData then
  begin
    FData := Value;
    if FName.Length > 0 then
    begin
      if FStorage  nil then
        FStorage.AddOrSetValue(FName, Value);
    end;
  end;
end;

function TPropertyElement.GetAsBool: boolean;
begin
  TryStrToBool(FData, result);
end;

procedure TPropertyElement.SetAsBool(const Value: boolean);
begin
  FData := BoolToStr(Value, true);

  if FName.Length > 0 then
  begin
    if FStorage  nil then
      FStorage.AddOrSetValue(FName, FData);
  end;
end;

function TPropertyElement.GetAsFloat: double;
begin
  TryStrToFloat(FData, result);
end;

procedure TPropertyElement.SetAsFloat(const Value: double);
begin
  FData := FloatToStr(Value);
  if FName.Length > 0 then
  begin
    if FStorage  nil then
      FStorage.AddOrSetValue(FName, FData);
  end;
end;

function TPropertyElement.GetAsInt: integer;
begin
  TryStrToInt(FData, Result);
end;

procedure TPropertyElement.SetAsInt(const Value: integer);
begin
  FData := IntToStr(Value);
  if FName.Length > 0 then
  begin
    if FStorage  nil then
      FStorage.AddOrSetValue(FName, FData);
  end;
end;

//#############################################################################
// TPropertyBag
//#############################################################################

constructor TPropertyBag.Create(NameValuePairs: string);

begin
  inherited Create;
  FLUT := TDictionary.Create();

  NameValuePairs := NameValuePairs.Trim();
  if NameValuePairs.Length > 0 then
    Parse(NameValuePairs);
end;

destructor TPropertyBag.Destroy;
begin
  FLut.Free;
  inherited;
end;

procedure TPropertyBag.Clear;
begin
  FLut.Clear;
end;

procedure TPropertyBag.Parse(NameValuePairs: string);
var
  LList:      TStringList;
  x:          integer;
  LId:        string;
  LValue:     string;
  LOriginal:  string;
  {$IFDEF SUPPORT_URI_ENCODING}
  LPos:       integer;
  {$ENDIF}
begin
  // Reset content
  FLUT.Clear();

  // Make a copy of the original text
  LOriginal := NameValuePairs;

  // Trim and prepare
  NameValuePairs := NameValuePairs.Trim();

  // Anything to work with?
  if NameValuePairs.Length > 0 then
  begin
    {$IFDEF SUPPORT_URI_ENCODING}
    // Check if the data is URL-encoded
    LPos := pos('%', NameValuePairs);
    if  (LPos >= low(NameValuePairs) )
    and (LPos  0 then
    Begin
      (* Populate our lookup table *)
      LList := TStringList.Create;
      try
        LList.Delimiter := ';';
        LList.StrictDelimiter := true;
        LList.DelimitedText := NameValuePairs;

        if LList.Count = 0 then
          raise EPropertybagParseError.CreateFmt(cnt_err_sourceparameters_parse, [LOriginal]);

        try
          for x := 0 to LList.Count-1 do
          begin
            LId := LList.Names[x].Trim().ToLower();
            if (LId.Length > 0) then
            begin
              LValue := LList.ValueFromIndex[x].Trim();
              Write(LId, LValue);
            end;
          end;
        except
          on e: exception do
          raise EPropertybagParseError.CreateFmt(cnt_err_sourceparameters_parse, [LOriginal]);
        end;
      finally
        LList.Free;
      end;
    end;
  end;
end;

function TPropertyBag.ToString: string;
var
  LItem: TPair;
begin
  setlength(result, 0);
  for LItem in FLut do
  begin
    if LItem.Key.Trim().Length > 0 then
    begin
      result := result + Format('%s=%s;', [LItem.Key, LItem.Value]);
    end;
  end;
end;

procedure TPropertyBag.SaveToStream(const Stream: TStream);
var
  LData: TStringStream;
begin
  LData := TStringStream.Create(ToString(), TEncoding.UTF8);
  try
    LData.SaveToStream(Stream);
  finally
    LData.Free;
  end;
end;

procedure TPropertyBag.LoadFromStream(const Stream: TStream);
var
  LData: TStringStream;
begin
  LData := TStringStream.Create('', TEncoding.UTF8);
  try
    LData.LoadFromStream(Stream);
    Parse(LData.DataString);
  finally
    LData.Free;
  end;
end;

function TPropertyBag.Write(Name: string; Value: string): IPropertyElement;
begin
  Name := Name.Trim().ToLower();
  if Name.Length > 0 then
  begin
    if not FLUT.ContainsKey(Name) then
      FLut.Add(Name, Value);

    result := TPropertyElement.Create(FLut, Name, Value) as IPropertyElement;
  end else
  raise EPropertybagWriteError.Create(cnt_err_sourceparameters_write_id);
end;

function TPropertyBag.Read(Name: string): IPropertyElement;
var
  LData:  String;
begin
  Name := Name.Trim().ToLower();
  if Name.Length > 0  then
  begin
    if FLut.TryGetValue(Name, LData) then
      result := TPropertyElement.Create(LData) as IPropertyElement
    else
      raise EPropertybagReadError.Create(cnt_err_sourceparameters_read_id);
  end else
  raise EPropertybagReadError.Create(cnt_err_sourceparameters_read_id);
end;


end.

Enumerating network adapters in DWScript/Smart under Node.js

July 5, 2019 Leave a comment

This is something I never had the time to implement under Smart Pascal, but it should be easy enough to patch. If you are using DWScript with the QTX Framework this is already in place. But for Smart users, here is a quick recipe.

First, we need access to the node.js OS module:

unit qtx.node.os;

//#############################################################################
// Quartex RTL for DWScript
// Written by Jon L. Aasenden, all rights reserved
// This code is released under modified LGPL (see license.txt)
//#############################################################################

unit NodeJS.os;

interface

uses
  NodeJS.Core;

type

  TCpusResultObjectTimes = class external
    property user: Integer;
    property nice: Integer;
    property sys: Integer;
    property idle: Integer;
    property irq: Integer;
  end;

  TCpusResult = class external
    property model: String;
    property speed: Integer;
    property times: TcpusResultObjectTimes;
  end;

  JNetworkInterfaceInfo = class external
    property address:  string;
    property netmask:  string;
    property family:   string;
    property mac:      string;
    property scopeid:  integer;
    property internal: boolean;
    property cidr:     string;
  end;

  Jos_Exports = class external
  public
    function tmpDir: String;
    function hostname: String;
    function &type: String;
    function platform: String;
    function arch: String;
    function release: String;
    function uptime: Integer;
    function loadavg: array of Integer;
    function totalmem: Integer;
    function freemem: Integer;
    function cpus: array of TCpusResult;
    function networkInterfaces: variant;
    property EOL: String;
  end;

function NodeJSOsAPI: Jos_Exports;

implementation

function NodeJSOsAPI: Jos_Exports;
begin
  result := Jos_Exports(RequireModule("os") );
end;

end.

With that in place, we can start enumerating through the adapters. Remember that a PC can have several adapters attached, from a dedicated card to X number of USB wifi sticks.

Here is a little routine that goes through the adapters, and returns the first IPv4 LAN address it finds. This is very useful when writing servers, since you need the IP + port to setup a binding. And yes, you can just call HostName(), but the point here is to know how to run through the adapter array.

function GetMyV4LanIP: string;
begin
  var OSAPI := NodeJSOsAPI();
  var NetAdapters := OSAPI.networkInterfaces();

  for var Adapter in NetAdapters do
  begin
    // Skip loopback device
    if Adapter.Contains('Loopback') then
      continue;

    for var netIntf in NetAdapters[Adapter] do
    begin
      var address = JNetworkInterfaceInfo( NetAdapters[Adapter][netIntf] );
      if not address.internal then
      begin
        // force copy of string
        var lFam: string := string(address.family) + " ";

        // make sure its ipv4
        if lFam.ToLower().Trim() = 'ipv4' then
        begin
          result := address.address + " ";
          result := result.trim();
          break;
        end;
      end;
    end;
  end;

  if result.length < 1 then
    result := '127.0.0.1';
end;

Raspberry PI 4 at last!

June 25, 2019 2 comments

It was with astonishment that I opened up my browser this morning to read some daily IT news, only to discover that the Raspberry PI v4 has finally arrived! And boy what a landslide update to the 3.x family it is!

Three times the fun

There are plenty of sites that entertains page-up and page-down with numbers, but I will save all that for an article where I have the physical kit in my posession. But looking at the preliminaries I think it’s safe to say that we are looking at a solid 3x the speed of the older yet capable PI 3b+.

Raspberry-Pi-4-4

The PI returns, and what a joy it is!

While the 3x speed boost is enough to bump the SoC up, from entertaining to serious for business applications – it’s ultimately the memory footprint that will make all the difference. While the Raspberry PI is probably the most loved SBC (single board computer) of all time, it’s always been cut short due to lack of memory. 512 megabyte can only do so much in 2019, and even the slimmest of Linux distributions quickly consumes more ram that older versions could supply.

VideoCore 6, two screens and 4k video

The new model ships in three different configurations, with 1, 2 and 4 gigabytes of ram respectively. I strongly urge people to get the 4Gb version, because with that amount of memory coupled with a good solid-state-disk, means you can enable a proper swap-partition. No matter how fast a SoC might be, without memory to compliment it – the system simply wont be able to deliver on its potential. But with 4Gb, a nice solid state disk (just use a SSD-To-USB with one of the sexy new USB 3.x ports) and you are looking at an OK mini-computer capable of most desktop applications.

I have to admit I never expected the PI to ship with support for two monitors, but lo-and-behold, the board has two mini-hdmi out ports! The board is also fitted with the VideCore 6 rather than VideoCore 4.

Not missing the boat with Oxygene and Elements

One of the most frustrating episodes in the history of Delphi, is that we never got a Delphi edition that could target Raspberry PI (or ARM-Linux in general). It was especially frustrating since Allen Bauer actually demonstrated Delphi generating code that ran on a PI in 2012. The result of not directly supporting the PI, even on service level without a UI layer – is that Delphi developers have missed the IOT market completely.

Before Delphi developers missed the IOT revolution, Delphi also missed out on iOS and Android. By the time Delphi developers could target any of these platforms, the market was completely saturated, and all opportunities to make money was long gone. In other words, Delphi has missed the boat on 3 revolutionary platforms in a row. Something which is borderline unforgivable.

The good news though is that Oxygene, the object-pascal compiler from RemObjects, supports the Raspberry PI SoC. I have yet to test this on v4, but since the PI v4 is 100% backwards compatible I don’t see any reason why there should be any issues. The code generated by Oxygene is not bound to just the PI either. As long as it runs on a debian based distro, it should run just fine on most ARM-Linux SoC’s that have working drivers.

And like I have written about elsewhere, you can also compile for WebAssembly, running either in node.js or in the browser — so there are plenty of ways to get your products over!

Stay tuned for the lineup

This week im going to do a lot of testing on various ARM devices to find out just how many SBC’s Oxygene can target, starting with the ODroid N2. But for Raspberry PI, that should be a slam-dunk. Meaning that object-pascal developers can finally make use of affordable off-the-shelves parts in their hardware projects.

As of writing im preparing the various boards I will be testing. We have the PI 3b+, the Tinkerboard from ASUS, NanoPI, Dragonboard, Odroid XU4 – and the latest power-board, the ODroid N2. Out of these offerings only the N2 is en-par with the Raspberry PI v4, although I suspect the Videocore 6 GPU will outperform the Mali G52.

RemObjects VCL, mind blown!

June 12, 2019 12 comments

For a guy that spends most of his time online, and can talk for hours about the most nerdy topics known to mankind – being gobsmacked and silenced is a rare event. But this morning that was exactly what happened.

Now, Marc Hoffman has blogged regularly over the years regarding the evolution of the RemObjects toolchain; explaining how they decoupled the parts that make up a programming language, such as syntax, rtl and target, but I must admit haven’t really digested the full implications of that work.

Like most developers I have kept my eyes on the parts relevant for me, like the Remoting SDK, Data Abstract and Javascript support. Before I worked at Embarcadero I pretty much spent 10 years contracting -and building Smart Mobile Studio on the side together with the team at The Smart Company Inc.

xo

Smart Pascal gained support for RemObjects SDK servers quite early

Since both the Remoting SDK and Data Abstract were part of our toolbox as Delphi developers, those were naturally more immediate than anything else. We also added support for RemObjects Remoting SDK inside Smart Mobile Studio, so that people could call existing services from their Javascript applications.

Oxygene then

Like most Delphi developers I remember testing Oxygene Pascal when I bought Delphi 2005. Back then Oxygene was licensed by Borland under the “Prism” name and represented their take on dot net support. I was very excited when it came out, but since my knowledge of the dot net framework was nil, I was 100% relient on the documentation.

In many ways Oxygene was a victim of Rad Studio’s abhorrent help-file system. Documentation for Rad Studio (especially Delphi) up to that point had been exemplary since Delphi 4; but by the time Rad Studio 2005 came out, the bloat had reached epic levels. Even for me as a die-hard Delphi fanatic, Delphi 2005 and 2006 was a tragic experience.

image

Removing Oxygene was a monumental mistake

I mean, when it takes 15 minutes (literally) just to open the docs, then learning a whole new programming paradigm under those conditions was quite frankly impossible. Like most Delphi developers I was used to Delphi 7 style documentation, where the docs were not just reference material – but actually teaches you the language itself.

In the end Oxygene remained very interesting, but with a full time job, deadlines and kids to take care of, I stuck to what I knew – namely the VCL.

Oxygene today

Just like Delphi has evolved and improved radically since 2005, Oxygene has likewise evolved above and beyond its initial form. Truth be told, we copied a lot of material from Oxygene when we made Smart Pascal, so I feel strangely at home with Oxygene even after a couple of days. The documentation for Oxygene Pascal (and Elements as a whole) is very good: https://docs.elementscompiler.com/Oxygene/

But Oxygene Pascal, while the obvious “first stop” for Delphi developers looking to expand their market impact, is more than “just a language”. It’s a language that is a part of a growing family of languages that RemObjects support and evolve.

As of writing RemObjects offers the following languages. So even if you don’t have a background in Delphi, or perhaps migrated from Delphi to C# years ago – RemObjects will have solutions and benefits to offer:

  • Oxygene (object pascal)
  • C#
  • Swift
  • Java
water

Water is a sexy, slim new IDE for RemObjects languages on Windows. For the OS X version you want to download Fire.

And here is the cool thing: when you hear “Java” you automatically expect that you are bound hands and feet to the Java runtime-libraries right? Same also with C#, you expect C# to be purely limited to the dot-net framework. And if you like me dabbed in Oxygene back in 2005-2006, you probably think Oxygene is purely a dot-net adapted version of Object Pascal right? But RemObjects have turned that on it’s head!

Remember the decoupling I mentioned at the beginning of this post? What that means in practical terms is that they have separated each language into three distinct parts:

  1. The syntax
  2. The RTL
  3. The target

What this means, is that you can pick your own combinations!

Let’s say you are coming from Delphi. You have 20 years of Object Pascal experience under your belt, and while you dont mind learning new things – Object Pascal is where you will be most productive.

Well in that case picking Oxygene Pascal covers the syntax part. But you don’t have to use the dot-net framework if you don’t want to. You can mix and match these 3 parts as you see fit! Let’s look at some combinations you could pick:

  • Oxygene Pascal -> dot net framework -> CIL
  • Oxygene Pascal -> “VCL” -> CIL
  • Oxygene Pascal -> “VCL” -> WinAPI
  • Oxygene Pascal -> “VCL” -> WebAssembly

(*) The “VCL” here is a compatibility RTL closely modeled on the Freepascal LCL and Delphi VCL. This is written from scratch and contains no proprietary code. It is purely to get people productive faster.

The whole point of this tripartite decoupling is to allow developers to maximize the value of their existing skill-set. If you know Object Pascal then that is a natural starting point for you. If you know the VCL then obviously the VCL compatibility RTL is going to help you become productive much faster than calling WinAPI on C level. But you can, if you like, go all native. And you can likewise ignore native and opt for WebAssembly.

Sound cool? Indeed it is! But it gets better, let’s look at some of the targets:

  • Microsoft Windows
  • Apple OS X
  • Apple iOS
  • Apple WatchOS
  • Android
  • Android wearables
  • Linux x86 / 64
  • Linux ARM
  • tvOS
  • WebAssembly
  • * dot-net
  • * Java

In short: Pick the language you want, pick the RTL or framework you want, pick the target you want — and start coding!

(*) dot-net and Java are not just frameworks, they are also targets since they are Virtual Machines. WebAssembly also fall under the VM category, although the virtual machine there is bolted into Chrome and Firefox (also node.js).

Some example code

Webassembly is something that interest me more than native these days. Sure I love the speed that native has to offer, but since Javascript has become “the defacto universal platform”, and since most of my work privately is done in Javascript – it seems like the obvious place to start.

Webassembly is a bit like Javascript was 10 years ago. I remember it was a bit of a shock coming from Delphi. We had just created Smart Mobile Studio, and suddenly we realized that the classes and object the browser had to offer were close to barren. We were used to the VCL after all. So my work there was basically to implement something with enough similarity to the VCL to be familiar to to Delphi developer, without wandering too far away from established JS standards.

Webassembly is roughly in the same ballpark. Webassembly is just a runtime engine. It doesn’t give you all those nice and helpful classes out of the box. You are expected to either write that yourself – or (as luck would have it) rely on what language vendors provide.

RemObjects have a lot to offer here, because their “Delphi VCL” compatibility RTL compiles just fine for Webassembly. There is no form designer though, but I haven’t used a form designer in years. I prefer to do everything in code because that’s ultimately what works when your codebase grows large enough anyways. Even my Delphi projects are done mainly as raw code, because I like to have the option to compile with Freepascal and Lazarus.

My first test code for Oxygene Pascal with Webassembly as the target is thus very bare-bone. If there is something that has bugged me to no end, it’s that bloody HTML5 canvas. It’s a powerful thing, but it’s also overkill for per-pixel operations. So I figured that a nice, ad-hoc DIB (device independent bitmap) class will do wonders.

Note: Oxygene supports pointers, even under WebAssembly (!), but out of old habit I have avoided it. I want my code to compile for all the targets, without marking a class as “unsafe” in the dot-net paradigm. So I have avoided pointers and just use offsets instead.

namespace qtxlib;

interface

type

  // in-memory pixel format
  TPixelFormat = public (
      pf8bit  = 0,  //___8 -- palette indexed
      pf15bit = 1,  //_555 -- 15 bit encoded
      pf16bit = 2,  //_565 -- 16 bit encoded
      pf24bit = 3,  //_888 -- 24 bit native
      pf32bit = 4   //888A -- 32 bit native
      );

  TPixelBuffer = public class
  private
    FPixels:  array of Byte;
    FDepthLUT: array of Integer;
    FScanLUT: array of Integer;
    FStride:  Integer;
    FWidth:   Integer;
    FHeight:  Integer;
    FBytes:   Integer;
    FFormat:  TPixelFormat;
  protected
    function  CalcStride(const Value, PixelByteSize, AlignSize: Integer): Integer;
    function  GetEmpty: Boolean;
  public
    property  Width: Integer read FWidth;
    property  Height: Integer read FHeight;
    property  Stride: Integer read FStride;
    property  &Empty: Boolean read GetEmpty;
    property  BufferSize: Integer read FBytes;
    property  PixelFormat: TPixelFormat read FFormat;
    property  Buffer[const index: Integer]: Byte read (FPixels[&index]) write (FPixels[&index]);

    function  OffsetForPixel(const dx, dy: Integer): Integer;
    procedure Alloc(NewWidth, NewHeight: Integer; const PxFormat: TPixelFormat);
    procedure Release();

    function Read(Offset: Integer; ByteLength: Integer): array of Byte;
    procedure Write(Offset: Integer; const Data: array of Byte);

    constructor Create; virtual;

    finalizer;
    begin
      if not GetEmpty() then
        Release();
    end;
end;

TColorMixer = public class
end;

TPainter = public class
private
  FBuffer:    TPixelBuffer;
public
  property    PixelBuffer: TPixelBuffer read FBuffer;

  constructor Create(const PxBuffer: TPixelBuffer); virtual;
end;

implementation

//##################################################################################
// TPainter
//##################################################################################

constructor TPainter.Create(const PxBuffer: TPixelBuffer);
begin
  inherited Create();
  if PxBuffer  nil then
    FBuffer := PxBuffer
  else
    raise new Exception("Pixelbuffer cannot be NIL error");
end;

//##################################################################################
// TPixelBuffer
//##################################################################################

constructor TPixelBuffer.Create;
begin
  inherited Create();
  FDepthLUT := [1, 2, 2, 3, 4];
end;

function TPixelBuffer.GetEmpty: Boolean;
begin
  result := length(FPixels) = 0;
end;

function TPixelBuffer.OffsetForPixel(const dx, dy: integer): Integer;
begin
  if length(FPixels) > 0 then
  begin
    result := dy * FStride;
    inc(result, dx * FDepthLUT[FFormat]);
  end;
end;

procedure TPixelBuffer.Write(Offset: Integer; const Data: array of Byte);
begin
  for each el in Data do
  begin
    FPixels[Offset] := el;
    inc(Offset);
  end;
end;

function TPixelBuffer.Read(Offset: Integer; ByteLength: Integer): array of Byte;
begin
  result := new Byte[ByteLength];
  var xOff := 0;
  while ByteLength > 0 do
  begin
    result[xOff] := FPixels[Offset];
    dec(ByteLength);
    inc(Offset);
    inc(xOff);
  end;
end;

procedure TPixelBuffer.Alloc(NewWidth, NewHeight: Integer; const PxFormat: TPixelFormat);
begin
  if not GetEmpty() then
    Release();

  if NewWidth < 1 then
    raise new Exception("Invalid width error");

  if NewHeight  0 then
    result := ( (Result + AlignSize) - xFetch );
end;

end.

This code is just meant to give you a feel for the dialect. I have used a lot of “Delphi style” coding here, so chances are you will hardly see any difference bar namespaces and a funny looking property declaration.

Stay tuned for more posts as I explore the different aspects of Oxygene and webassembly in the days to come 🙂

Understanding a stack

May 9, 2019 Leave a comment

The concept of stacks is an old one, and together with linked-lists and queues – these form the most fundamental programming concepts a developer needs to master.

But, the stack most people use today in languages like object pascal and C++ are not actual stacks; they are more like “conveniently repurposed lists“. Not a huge issue I agree, but the misconception is enough to cause confusion when people dive into low-level programming.

Adventures in assembly-land

stackishIt might seem odd to focus on something as trivial as a stack, but I have my reasons. A friend of mine who is a brilliant coder with plenty of large projects behind him recently decided to have a go at assembly coding. He was doing fine and everything was great, until he started pushing and popping  things off the stack.

After a little chat I realized that the problem was not his code, but rather how he viewed the stack. He was used to high-level versions of stacks, which in most cases are just lists storing arbitrary sized data – so he was looking at the stack as a TList<item> expecting similar behavior. Superficially a real-stack and a list-stack work the same if all you do is clean push and pop operations, but the moment you start designing a stack-scheme and push more elaborate constructs (stack-frames), things can go wrong really fast.

The nature of a real stack

A “real” stack that is a part of a hardware SOC (system on a chip) has nothing to do with lists. It’s actually a solid chunk of memory with a register to keep track of an offset into this memory block.

Let’s for sake of argument say you have 4k of stack space right? It’s clean and contains nothing, so the SP (stack pointer, or offset) is zero. What happens when you push something to the stack? for example:

push EDX

The code above simply writes the content of the EDX register to whatever offset the SP contains. It then updates the SP with the size of the data (EDX is a 32bit register, so the SP is incremented by a longword or 4 bytes). In Delphi pseudocode what happens is something like:

var LAddr := FStackBuffer;
inc(LAddr, SP);
PLongword(LAddr)^ := EDX;
inc(SP, SizeOf(EDX));

The thing about a stack is that it doesn’t manage data-length for you. And that is a big difference to remember. It will push or pop data based on the size of the source (in this case the 32bit EDX register) you use.

If you push 1024 bytes of data to a list based stack, the list keeps track of the size and data for you. So when you pop the data from the stack, you get back that data regardless. But a “real” stack couldn’t care less — which is also why it’s so easy to screw up an entire program if you make a mistake.

In short: The length of what you push – must be matched when you pop the data back (!) If you push a longword, you MUST pop a longword later.

Benefits of a real stack

call stackThe benefit is that the cost of storing values on a stack is almost zero in terms of cpu operations. A list based stack is more expensive; it will allocate memory for a record-item to hold the information about the data, then it will allocate memory to hold the actual data (depends on the type naturally) and finally copy the data into the newly allocated buffer. Hundreds if not thousands of instructions can be involved here.

A real stack will just write whatever you pushed directly into the stack-memory at whatever offset SP is at. Once written it will add the length of the write to the SP – and that’s it! So it’s one of the oldest and fastest mechanisms for lining up data in a predictable way.

Again the rules are simple: when you pop something off the stack, the size must match whatever you used to push it there. So if you pushed a longword (EDX) you also have to make sure you use a 32-bit target when you pop the value back. If you use RDX, which is 64 bit then you will essentially steal 4 bytes from something else using that stack – and all hell will break loose down the line.

Stack schemes and frames

Im not going to dig too deeply into stack-frames here, but instead write a few words about stack-schemes and using the stack to persist data your code relies on. The lines blur between those two topics anyways.

The SP (stack pointer) is not just a simple offset you can read, you can also write and change it (it also serves as a pointer). You can also read from whatever memory the SP is pointing at without polling any data from the stack.

What language developers usually do, is that they design entire structures on the stack that are, when you get into the nitty-gritty, “offset based records”. For example, lets say you have a record that looks like this:

type
PMyRecord ) ^TMyRecord;
TMyRecord = record
  first: Pointer;
  second: integer;
  Third: array[0..255] of longword;
end;

Instead of allocating conventional ram to hold that record, people push it to the stack and then use offsets to read and update the values there. A bit like a super global variable if you like. This is why when you disassemble code, you find stuff like:

mov EDX, (SP)+4

If the above record was on the stack, that pseudo code would move the field “second” into the EDX register. Because that field is 4 bytes from the stack start (providing SP points to zero).

Every programming language has a stack scheme to keep track of things. Local variables, global variables, class instances, type RTTI — most of these things are allocated in conventional ram – but there is a “program record” on the stack that makes it easy to access that information quickly.

This “moving a whole record onto the stack” is basically what a stack-frame is all about. It used to be a very costly affair with a heavy cpu speed penalty. If you look in your Delphi compiler options you will see that there is a checkbox regarding this very topic. Delphi can be told to avoid stack-frames and do register allocation instead, which was super quick compared to stack-frames – but CPU’s today are largely optimized for stack-frame allocation as default, so I doubt there is much to gain by this in 2019.

Note: A stack frame is much more, but its out of scope for this post. Google it for more info.

To sum up

When doing high-level coding you don’t really need to bother with the nuances between a TStack<item> and a “real” stack. But if you plan on digging deeper and learning a few lines of assembly – learning the differences is imperative. Its boring stuff but as fundamental as wheels on a bicycle. There is no way to avoid it, so might as well jump in.

In its absolute raw form, here is roughly the same functionality for Delphi. This was written on the fly in 2 minutes while on the road, so its purely to give you a rough idea of behavior. I would add a secondary field to keep track of the end (next insertion point), that way SP can be changed without overwriting data on new pushes.

And yes, wrapping this in a TObject utterly defeats the purpose of low-level coding, but hopefully it gives you some idea of the differences 🙂

stack_01

stack_02

Building a Delphi Database engine, part four

March 23, 2019 Leave a comment

This article is over six months late (gasp!). Work at Embarcadero have been extremely time consuming, and my free time has been bound up in my ex-patreon project. So that’s why I was unable to finish in a more predictable fashion.

But better late than never — and we have finally reached one of the more exciting steps in the evolution of our database engine design, namely the place where we link our metadata to actual data.

So far we have been busy with the underlying mechanisms, how to split up larger pieces of data, how to collect these pieces and re-assemble them, how to grow and scale the database file and so on.

We ended our last article with a working persistence layer, meaning that the codebase is now able to write the metadata to itself, read it back when you open the database, persist sequences (records) – and our humble API is now rich enough to handle tasks like scaling. At the present we only support growth, but we can add file compacting later.

Tables and records

In our last article’s code, the metadata exposed a Table class. This table-class in turn exposed an interface to our field-definitions, so that we have a way to define how a table should look before we create the database.

You have probably taken a look at the code (I hope so, or much of this won’t make much sense) and noticed that the record class (TDbLibRecord) is used both as a blueprint for a table (field definitions), as well as the actual class that holds the values.

If you look at the class again (TDbLibRecord can be found in the file dblib.records.pas), you will notice that it has a series of interfaces attached to it:

  • IDbLibFields
  • IStreamPersist

The first one, which we expose in our Table as the FieldDefs property, simply exposes functions for adding and working with the fields. While somewhat different from Delphi’s traditional TFieldDefinition class, it’s familiar enough. I don’t think anyone who has used Delphi with databases would be confused around it’s members:

  IDbLibFields = interface
    ['{0D6A9FE2-24D2-42AE-A343-E65F18409FA2}']
    function    IndexOf(FieldName: string):  integer;
    function    ObjectOf(FieldName: string): TDbLibRecordField;

    function    Add(const FieldName: string; const FieldClass: TDbLibRecordFieldClass): TDbLibRecordField;
    function    Addinteger(const FieldName: string): TDbLibFieldInteger;
    function    AddStr(const FieldName: string): TDbLibFieldString;
    function    Addbyte(const FieldName: string): TDbLibFieldbyte;
    function    AddBool(const FieldName: string): TDbLibFieldboolean;
    function    AddCurrency(const FieldName: string): TDbLibFieldCurrency;
    function    AddData(const FieldName: string): TDbLibFieldData;
    function    AddDateTime(const FieldName: string): TDbLibFieldDateTime;
    function    AddDouble(const FieldName: string): TDbLibFieldDouble;
    function    AddGUID(const FieldName: string):  TDbLibFieldGUID;
    function    AddInt64(const FieldName: string): TDbLibFieldInt64;
    function    AddLong(const FieldName: string): TDbLibFieldLong;
  end;

But, as you can see, this interface is just a small part of what the class is actually about. The class can indeed hold a list of fields, each with its own datatype – but it can also persist these fields to a stream and read them back again. You can also read and write a value to each field. So it is, for all means and purposes, a single record in class form.

The term people use for this type of class is: property bag, and it was a part of the Microsoft standard components (Active X / COM) for ages. Its probably still there, but I prefer my own take on the system.

In this article we are going to finish that work, namely the ability to define a table, create a database based on the metadata, insert a new record, read records, and push the resulting binary data to the database file. And since the persistency is already in place, opening the database and reading the record back is pretty straight forward.

So this is where the metadata stops being just a blue-print, and becomes something tangible and real.

Who owns what?

Before we continue, we have to stop and think about ownership. Right now the database file persists a global list of sequences. The database class itself has no interest in who owns each sequence, if a sequence belongs to a table, if it contains a picture, a number or whatever the content might be — it simply keeps track of where each sequence begins.

So the first order of the day is to expand the metadata for tables to manage whatever records belongs to that table. In short, the database class will focus on data within its scope, and the table instances will maintain their own overview.

So the metadata suddenly need to save a list of longwords with each table. You might say that this is wasteful, that the list maintained by the database should be eliminated and that each table should keep track of it’s own data. And while that is tempting to do, there is also something to be said about maintenance. Being able to deal with persisted data without getting involved with the nitty-gritty of tables is going to be useful when things like database compacting enters at the end of our tutorial.

Locking mechanism

Delphi has a very user-friendly locking mechanism when it comes to databases. A table or dataset is either in read, edit or insert mode – and various functions are allowed or prohibited depending on that state. And it would probably be wise to merge the engine with Delphi’s own TDatabase and TTable at some point – but right now im more interested in keeping things clean and simple.

When I write “locking mechanism” I am not referring to a file-lock, or memory lock. Had we used memory-mapped files the locking mechanism would have been more elaborate. What I mean with a lock, is basically placing a table in one of the states I mentioned above. The table needs to know what exactly you want to do. Are you adding a record? Are you editing an existing record? The table code needs to know this to safely bring you from one mode to the next.

Suddenly, you realize why each table needs that extra list, because how is the table going to allow methods like first, next, last and previous? The record-list dealt with by the database is just a generic, non-ordered ledger of sequences (a global scope list if you will). Are you going to read all records back when you open the database to figure out who owns what?

A call to First() will mean a completely different offset for each table. And the logical way to handle this, is to give each table it’s own cursor. A class that keeps track of what records belongs to the table, and also keeps track of whatever states the table is in.

The database cursor

Since we are not up against Oracle or MSSQL here, but exploring database theory, I have kept the cursor as simple as I possibly could. It is a humble class that looks like this:

db_cursor

The idea of-course is that the table defaults to “read” mode, meaning that you can navigate around, record by record, or jump to a specific record using the traditional RecNo property.

The moment you want to insert or edit a record, you call the Lock() method, passing along the locking you need (edit or insert). You can then either cancel the operation or call post() to push the data down to the file.

The Lock() method is a function (bool), making it easier to write code, as such:

  if Database.Table.Cursor.Lock(cmInsert) then
  begin
    with Database.GetTableByName('access_log').cursor do
    begin
      Fields.WriteInt('id', FUserId);
      Fields.WriteStr('name', FuserName);
      Fields.WriteDateTime('access', Now);
      Post();
    end;
  end else
  raise exception.create('failed to insert record');

Im sure the are better designs, and the classes and layout can absolutely be made better; but for our purposes it should be more than adequate.

Reloading record data

In the previous articles we focused on writing data. Basically taking a stream or a buffer, breaking it into pages, and then storing the pages (or blocks) around the file where there was available space.

We cleverly crafted the blocks so that they would contain the offset to the next block in a sequence, making it possible to read back a whole sequence of blocks by just knowing the first one (!)

A part of what the cursor does is also to read data back. Whenever the RecNo field changes, meaning that you are moving around the table-records using the typical Next(), Previous(), First() etc functions — if the cursor is in read mode (meaning: you are not inserting data, nor are you editing an existing record), you have to read the record into memory. Otherwise the in-memory fields wont contain the data for that record.

Creating a cursor

One note before you dive into the code: You have to create a cursor before you can use it! So just creating a table etc wont be enough. Here is how you go about doing this:db_cursor_create

Creating the cursor will be neatly tucked into a function for the table instance, we still have other issues to deal with.

What to expect next?

Next time we will be looking at editing a record, commiting changes and deleting records. And with that in place we have finally reached the point where we can add more elaborate functionality, starting with expression parsing and filters!

You can check out the code here: https://bitbucket.org/cipher_diaz/dbproject/src/master/

Support the cause

If you like my articles and want to see more libraries and techniques, then consider donating to the project here: https://www.paypal.me/quartexNOR

paypal

Those that donate $50 or more automatically get access to the Quartex Web OS repositories, including full access to the QTX replacement RTL (for DWScript and Smart Mobile Studio).

Thank you for your support, projects like Amibian.js and the Quartex Web OS would not exist without my backers!

/Jon

Quartex: Mali GPU glitches

February 20, 2019 Leave a comment

EDIT: I did further testing after this article was written, and believe the source of this to be about heat. Even with extra fans, running games like Tyrian (asm.js) that are extremely demanding, plus resizing a graphics intensive windows constantly, the temperature reached 71 degrees C very quickly. And this was with two cabinet fans helping the built-in fan to cool the device. It is thus not unthinkable that when running solo (no extra fans) that the kernel shut the device down to not cook the chipset. Which also explains why the device wont boot properly afterwards (the device is still hot).

Glitches

Something really strange is happening on Chrome and Firefox for ARM. JavaScript is not supposed to be able to take down a system, and in this case it’s neither an attempt as such either — yet for some reason I have managed to take down the ODroid XU4 with both Chrome and Firefox lately.

ODroid XU4

I guess I should lead with that I’m not able to replicate this on x86. One of the things I really love about the ODroid XU4 is that it’s affordable, powerful and probably the only SBC I have used that runs stable on the mali GPU. As you probably know I tested at least 10 different SBC’s back in 2018, and whenever there was a mali GPU involved, the product was either haunted by instabilities or lacked drivers all together.

amibian

Since the codebase for Chrome (and I presume Firefox) is ultimately the same between platforms, it leaves a question-mark about the ODroid. It is by far the most stable SBC I have tested so far (except for the PI, which is sadly underpowered for this task), but stable doesn’t mean flawless. And to be honest, Amibian.js is pushing web tech to the very limits.

Not Mali again

The reason I suspect the mali to be the culprit behind all this, is because the “bug” if we can call it that, happens exclusively during resize. So if there is a lot going on inside a desktop-window, you can sometimes provoke the ODroid to cold-crash and reboot. You actually have to power the board down and switch it back on for it to boot properly.

50431451_10155954273110906_8776790185049325568_n

Cloudripper ~ 5x ODroid XU4 [40 cores] in a PICO 5h cube

The resize and moving of windows uses CSS transformation, which in modern browsers makes use of the GPU. Chrome talks directly with OpenGL (or glES), so the operations are proxied through that. And again, since OpenGL is pretty rock solid elsewhere, we are only left with one common denominator: the mali GPU.

The challenge is that there is no way to debug or catch this error, because when it occurs the whole system literally goes down. There is no exception thrown, nor is the browser process terminated (not even a log entry, so it’s a clean-cut) — the system reboots on the spot. Since it fails on reboot when opening X (setting a screen-mode) I again point the finger at the GPU. Somehow a flag or lock survives the cold-reboot and that’s why you have to manually switch it off and on again.

This is the exact problem that made the NanoPI Fire useless. It only shipped with Android embedded drivers. The X drivers could hardly open a display without crashing. Such a waste of a good cpu.

x86 as head

ODroid is perfect for a low-cost Amibian.js experience, but I was unsure if it would handle the payload. Interestingly it handles it just fine and even with a high-speed action game running + background tasks we are not using 50% of the CPU even.

Ram is holding up too, with memory consumption while running Tyrian + having a few graphics viewers open, is at a reasonable 700 mb (of 2 gigabyte in total).

51398321_10155998598505906_8984850199142727680_o

Tyrian jogs along at 45 fps ~ that is not bad for a $45 SBC

Right now this strange error is rare, but if it continues or grows into a problem (chrome is hardly useable at all, only firefox) then I have no option than to replace the master sbc in the cluster with something else. The x86 UP board is more than capable, but it would be a shame to break the price range because of that (excuse my language) crap mali GPU. I honestly don’t understand why board makers insist on using a mali. Every board that has a mali is haunted by problems and get poor reviews.

It will be exciting to check out the dragonboard, although I fear 1Gb memory will not be enough for smooth operation. Not without a sata interface and a good swap-file.

Android and Delphi

One alternative is to switch to Android and use Delphi to code a custom Chromium Embedded webview. I am hoping to avoid the overhead of Android, but Delphi would definitively be a bonus with Android embedded (“Android of things”).

We will see.

Five reasons to learn Delphi

February 8, 2019 5 comments

A couple of days ago I had a spectacular debate on Facebook. Like most individuals that are active in the IT community, my social media feed is loaded with advertisement for every trending IT concept you can imagine. Lately these adverts have been about machine learning and A.I. Or should I say, companies using those buzzwords to draw unwarranted attention to their products. I haven’t seen A.I used to sell shoes yet, but it’s only a matter of time before it happens.

Cloud Computing concept background with a lot of icons

Like any technology, Cloud is only as powerful as your insight

There is also this thing that: yes, a 14-year-old can put together an A.I chat robot in 15 minutes with product XYZ. But that doesn’t mean he or she understands what is happening beneath the user-interface. Surely the goal must be to teach those kids skills that will benefit them for a lifetime.

Those that know me also know that yes, I have this tendency to say what I mean, even when I really should keep my mouth shut. On the other hand that is also why companies and developers call me, because I will call bullshit and help them avoid it. That’s part of my job, to help individuals and companies that use Delphi to pick the right version for their need, get the components that’s right for their goals – and map out a strategy if they need some input on that.  I’ll even dive in and do some code conversion if they need it; goes with the territory.

Normally I just ignore advertizing that put “cloud” or “a.i” in their title, because it’s mostly click-bait designed for non-developers. But for some reason this one particular advert caught my eye. Perhaps it triggered the trauma of being subjected to early Java advertising during the late 90s’s, or maybe it released latent aggression from being psychologically waterboarded by Microsoft Silverlight. Who knows 🙂

The ad was about a Norwegian company that specialize in teaching young students how to become professional developers. You know the “become a guru in 3 weeks” type publisher? What baked my noodle was the fact that they didn’t offer a single course involving archetypical languages, and that they were spinning their material with promises that were simply not true. The only artificial intelligence involved was the advertizing engine at Facebook.

The thing is – the world has more than enough developers on desktop level. The desktop and web market is drowning in developers who has the capacity to download libraries, drop components on a form and hook up to a database. What the world really needs are more developers on archetypical languages. And if you don’t know what that is, then let me just do a quick summary before we carry on.

Archetypal languages

An archetypical programming language is one that is designed around how the computer actually works. As a consequence these languages and toolchains embody several of the following properties:

  • Pointers and raw memory access
  • Traditional memory management, no garbage collection
  • Procedural and object-orientation execution
  • Inline assembler
  • Little if no external dependencies
  • Static linking (embed pre-compiled code)
  • Compiled code can operate without an OS infrastructure
  • Suitable for kernel, driver, service, desktop, networking and cloud level development
  • Compiler that produce machine code for various chipsets

As of writing there are only two archetypical languages (actually 3, but assembly language is chipset specific so we will skip that here), namely C/C++ and Object Pascal. These are the languages you use to write all the other languages with. If you plan on writing your own operating-system from scratch, only C and Pascal is suitable. Which is why these are the only languages that have ever been used for making operating systems.

tiobi

Delphi is one of the 20 most used programming languages in the world. It ranked as #11 in 2017. Like all rankings it fluctuates depending on season and market changes.

Obviously i’m not suggesting that people learn Delphi or C++ builder to write their own OS – or that you must know assembly to make an invoice system; I’m simply stating that the insight and skill you get from learning Delphi and C/C++, even if all you do is write desktop applications – will make you a better developer on all levels.

Optimistic languages

Optimistic or humanized programming languages, have been around just as long as the archetypical ones. Basic is an optimistic language, C# and Java are optimistic languages, Go and Dart are equally optimistic languages. Script engines like node.js, python and Erlang (if you missed Scott Hanselman’s epic rant on the subject, you are in for a treat) are all optimistic. They are called optimistic because they trade security with functionality; sandboxing the developer from the harsh reality of hardware.

An optimistic language is typically designed to function according to “how human beings would like things to be” (hence the term optimistic). These languages rely heavily on existing infrastructure to even work, and each language tends to focus on specific tasks – only to branch out and become more general purpose over time.

There is nothing wrong with optimistic languages. Except when they are marketed to young students as being somehow superior or en-par with archetypical languages. That is a very dangerous thing to do – because teachers have a responsibility to prepare the students for real life. I can’t even count the number of times I have seen young developers fresh out of college get “that job”, only to realize that the heart of the business, the mission critical stuff, is written in Delphi or C/C++, which they never learned.

People have no idea just how much of the modern world rests on these languages.  It is almost alarming how it’s possible to be a developer in 2019 and have a blind spot with regards to these distinctions. Don’t get me wrong, it’s not the student’s fault, quite the opposite. And i’m happy that things are starting to change for the better (more about that further down).

The original full stack

So back to my little encounter; What happened was that I just commented something along the lines of “why not give the kids something that will benefit them for a lifetime”. It was just a drive-by comment on my part, and I should have just ignored it; And no sooner had I pressed enter, when a small army of internet warriors appeared to defend their interpretation of “full stack” in 2019. Oblivious to the fact that the exact same term was used around 1988-ish. I think it was Aztec or SAS-C that coined it. Doesn’t matter.

aztec-c

The original “full stack” holds a very different meaning in traditional development. While I don’t remember if it was Aztec-C or SAS-C, but the full stack was driver to desktop 🙂

Long story short, I ended up having a conversation with these teenagers about how technology has evolved over the past 35 years. Not in theory, but as one that has been a programmer since the C= 64 was released. I also introduced them to archetypal languages and pinpointed the distinction I made above. You cannot compare if you don’t know the difference.

I have no problems with other languages, I use several myself, and my point was simply that: if we are going to teach the next generation of programmers something, then let’s teach them the timeless principles and tools that our eco system rests on. We need to get Delphi and C/C++ back into the curriculum, because that in turn will help the students to become better developers. It doesn’t matter what they end up working with afterwards, because with the fundamental understanding in place they will be better suited. Period.

You will be a better Java developer if you first learn Delphi. You will be a better C# developer if you learn Delphi. Just like nature has layers of complexity, so does computing. And understanding how each layer works and what laws exist there – will have a huge impact on how you write high-level code.

All of this was good and well and the internet warriors seemed a bit confused. They weren’t prepared for an actual conversation. So what started a bit rough ended up as a meaningful, nice dialog.

And speaking of education: I’m happy to say that two universities in Norway now have students using Delphi again. Which is a step in the right direction! People are re-discovering how productive Object-Pascal is, and why the language remains the bread and butter of so many companies around the world.

Piracy, the hydra of problems

What affected me the most during my conversation with these young developers – was that they had almost no relationship to neither Delphi or C/C++. From an educational standpoint that is not just alarming, that is an intellectual emergency. The only knowledge they had of Delphi was hearsay and nonsense.

piracy

The source of the misrepresentation is piracy, openly so, of outdated versions that was never designed to run on modern operating systems. With the community edition people can enjoy a modern, high performance Delphi without resorting to illegal activities

But after a while I finally discovered where their information came from! Delphi 7 is being pirated en-mass even to this day. It’s for some strange reason very popular in Asia (most of the torrent IP’s ended up there when I followed up on this). So teenagers download Delphi 7 which is ancient by any standard, and the first thing they experience is incompatibility issues. Which is only to be expected because Delphi 7 was released a long, long time ago. But that’s the impression they are left with after downloading one of these cracked, illegal bundles.

I downloaded one of these “ready to use” bundles to have a closer look, and it contained at least 500 commercial components. You had the full TMS component collection, Developer Express, Remobjects SDK, ImageEN, FastReports, SecureBlackBox, Intraweb — tens of thousands of dollars worth of code. With one very obvious factor: both Delphi 7 and the components involved are severely outdated. Microsoft doesn’t even support Windows XP any more, it was written in the early bronze age.

So the reality of the situation was that these young developers had never seen a modern Delphi in their life. In their minds, Delphi meant Delphi 7 which they could download almost everywhere (which is illegal and riddled with viruses, stay well clear). No wonder there is confusion about the subject (!)

They were very happy to learn about the community edition, so in the end I at least got to wake them up to the awesome features that modern Delphi represents. The community edition has been a fantastic thing; the number of members joining Delphi-Developer on Facebook has nearly doubled since the community edition was released.

A few of the students went over to Embarcadero and downloaded the community edition, and their jaw dropped. They had never seen a development environment like this before!

Give me five good reasons to learn Delphi

delphi_boxIn light of this episode, thought I could share five reasons why Delphi and object-pascal remains my primary programming language.

I don’t have any problems dipping into JavaScript, Python or whatever the situation might call for – but when it comes to mission critical data processing and services that needs 24/7 up-time; or embedded solutions where CPU spikes simply cannot be tolerated. It’s Delphi I turn to.

These five reasons are also the same that I gave the teenagers. So here goes.


Great depth and wingspan

Object Pascal, from which Delphi is the trending dialect, is a fantastic language. At heart there is little difference between C/C++ and object pascal in terms of features, but the syntax of object pascal is more productive than C/C++ (IMHO).

Delphi and C++ builder actually share run-time libraries (there are two of them, the VCL which is Windows only, and Firemonkey which is platform independent). Developers often mix and match code between these languages, so components written in Delphi can be used in C++ builder, and libraries written in C can be consumed and linked into your Delphi executable.

One interesting factoid: people imagine Delphi to be old. But the C language is actually 3 years older than pascal. During their time these languages have evolved side by side, and Embarcadero (who makes Delphi and C++ builder) have brought all the interesting features you expect from a modern language into Delphi (things like generics, inline variables, anonymous procedures – it’s all in there). So this myth that Delphi is somehow outdated or unsuitable is just that – a myth.

foodchain

The eco-system of programming languages

And there is an added bonus! Just like C/C++, Delphi represents a curriculum and lineage that spans decades. Stop and think about that for a second. This is a language that has been evolved to solve technical challenges of every conceivable type for decades. This means that you can put some faith in what the language can deliver.

There are millions of Delphi developers in the world; an estimated 10 millions in fact. The language was ranked #11 on the TIOBI language index; it is under constant development with a clear roadmap and time-line – and is used by large and small companies as the foundation for their business. Even the Norwegian government rely on Delphi. The system that handles healthcare messages for the Norwegian population is pure Delphi.  That is data processing for 5.2 million individuals.

Object Pascal has not just stood the test of time, it has innovated it. Just like C/C++ object pascal has a wingspan and depth that reaches from assembler to system services, from database engines to visual desktop application – and from the desktop all the way to Cloud and essential web technology.

So the first good reason to learn Delphi is depth. Delphi covers the native stack, from kernel level drivers to high-speed database engines – to visual desktop applications. It’s also exceptionally well suited for cloud services (both Windows and Linux targets).


Easy to learn

I mention that Delphi is powerful and has the same depth as C/C++, but why then learn Delphi and not C++? Well, the language (object pascal) was especially tailored for readability. It was concluded that the human brain recognized words faster than symbols or glyphs – and thus it’s easier to read complex pascal code rather than complex C code. Individual taste notwithstanding.

Despite it's depth, Delphi is easy to learn and fun to master!

Despite its depth, Delphi is easy to learn and fun to master!

Object Pascal is also very declarative, with as little unknown factors as possible. This teaches people to write clean and orderly code.

And perhaps my favorite, a pascal code-file contains both interface and implementation. So you don’t have to write a second .h file which is common under C/C++.

If you already know OOP, be it Java, C#, Rust or whatever – learning Delphi will be a piece of cake. You already know about classes, interfaces, generics, operator overloading – and can pretty much skip forward to memory management, pointers and structures (records in pascal, struct in C).

Swing by Embarcadero Academy and take a course, or head over to Amazon and buy some good books on Delphi. Download the Community Edition of Delphi and you will be up and running in no-time.

Also remember to join Delphi Developer on Facebook, where thousands of active developers talk, help each other and share solutions 24/7.


Target multiple platforms

With Delphi and C++ builder it’s pretty easy to target multiple platforms these days. You can target Android, iOS, OS X, Windows and Linux from a single codebase.

One codebase, multiple targets

One codebase, multiple targets

I mean, are you going to write one version of your app in Java, a second one in C#, a third one in Objective C and a fourth in Dart? Because that’s the reality you face if plan on using the development tools provided by each operating-system manufacturer. That’s a lot of time, money and effort just to push your product out the door.

With Delphi you can hit all platforms at once, native code, reducing your time to market and ROI. People use Delphi for a reason.

You will also enjoy great performance from the LLVM optimized code Delphi emits on mobile platforms.


Rich codebase

The benefit of age is often said to be wisdom; I guess the computing equivalent is a large and rich collection of components, libraries and ad-hoc code that you can drop into your own projects or just study.

You can google just about any subject, and there will be code for Delphi. Github, BitBucket and Torry’s Delphi pages are packed with open-source frameworks covering everything from compiler cores, midi interfaces, game development to multi-threaded, machine clustered server solutions. Once you start looking, you will find it.

GitLab-vs-GitHub-vs-bitbucket-1

There is a rich constellation of code, components and libraries for Delphi and C++ builder around the internet.  Also remember dedicated sites like Torry’s

There is also a long list of technology partners that produce components and libraries for Delphi – and like mentioned earlier, you can link in C compiled code once you learn the ropes.

Oh, and when I mentioned databases earlier I wasnt just talking about the traditional databases. Delphi got you covered with those, no worries — im also talking about writing a database engine from scratch. There are several database engines that are implemented purely in Delphi. ElevateDB is one example.

Delphi also ships with Interbase and Interbase-light (embedded and mobile) so you have easy access to data storage solutions. There is also FireFAC that allows you to connect directly with established databases — and again, a wealth of free and commercial solutions.


Speed and technique

What I love about Delphi and C++ is that your code, or the way you write code, directly impacts your results. The art of optimization is rarely a factor in some of the new, optimistic languages. But in a native language you get to use traditional techniques that are time-less, or perhaps more interesting: explore ways of achieving the same with less.

As a native language Delphi and C/C++ produce fast executables. But I love how there is always room for your own techniques, your own components and your own libraries.

tomes

Techniques, like math, is timeless

Need to write a system driver? Well, suddenly speed becomes a very important factor. A garbage collector can be a disaster on that level, because it will kick-in on interval and cause CPU spikes. Perhaps you want to write a compiler, or need a solid scripting engine? How about linking the V8 JavaScript engine directly into your programs? All of this is quite simple with Delphi.

So with Delphi I get the best of both worlds, I get to use the scalpel when the needs are delicate, and I get the chain-saw to cut through tedious work. Things like property bindings are a god sent. This is a techniques where you can visually bind properties of any component together, almost like events, and create cause and effect chains. So if a value changes on a bound property, that triggers whatever is bound, and so on and so on — pretty awesome!

So you can create a complete database application, with grid and navigation, without writing a single line of code. That was just one simple example, you can do so much more out of the box – and it saves you so much time.

Yet when you really need to write high performance code, or build that killer framework that will set your company apart from the rest — you have that freedom!


So if you havent checked out RAD Studio, head over to Embarcadero and download a free trial. You will be amazed and realize just why Delphi and C++ builder are loved by so many.

Amibian.js under the hood

December 5, 2018 2 comments

Amibian.js is gaining momentum as more and more developers, embedded systems architects, gamers and retro computer enthusiasts discover the project. And I have to admit I’m pretty stoked about what we are building here myself!

intro

In a life-preserver no less 😀

But, with any new technology or invention there are two common traps that people can fall into: The first trap is to gravely underestimate a technology. JavaScript certainly invites this, because only a decade ago the language was little more than a toy. Since then JavaScript have evolved to become the most widely adopted programming language in the world, and runtime engines like Google’s V8 runs JavaScript almost as fast as compiled binary code (“native” means machine code, like that produced by a C/C++ compiler, Pascal compiler or anything else that produces programs that run under Linux or Windows).

It takes some adjustments, especially for traditional programmers that havent paid attention to where browsers have gone – but long gone are the days of interpreted JavaScript. Modern JavaScript is first parsed, tokenized and compiled to bytecodes. These bytecodes are then JIT compiled (“just in time”, which means the compilation takes place inside the browser) to real machine-code using state of the art techniques (LLVM). So the JavaScript of 2018 is by no means the JavaScript of 2008.

The second trap you can fall into – is to exaggerate what a new technology can do, and attach abilities and expectations to a product that simply cannot be delivered. It is very important to me that people don’t fall into either trap, and that everyone is informed about what Amibian.js actually is and can deliver – but also what it wont deliver. Rome was not built-in a day, and it’s wise to study all the factors before passing judgement.

I have been truly fortunate that people support the project financially via Patreon, and as such I feel it’s my duty to document and explain as much as possible. I am a programmer and I often forget that not everyone understands what I’m talking about. We are all human and make mistakes.

Hopefully this post will paint a clearer picture of Amibian.js and what we are building here. The project is divided into two phases: first to finish Amibian.js itself, and secondly to write a Visual Studio clone that runs purely in the browser. Since it’s easy to mix these things up, I’m underlining this easy – just in case.

What the heck is Amibian.js?

Amibian.js is a group of services and libraries that combined creates a portable operating-system that renders to HTML5. A system that was written using readily available web technology, and designed to deliver advanced desktop functionality to web applications.

The services that make up Amibian.js was designed to piggyback on a thin Linux crust, where Linux deals with the hardware, drivers and the nitty-gritty we take for granted. There is no point trying to write a better kernel in 2018, because you are never going to catch up with Linus Torvalds. It’s must more interesting to push modern web technology to the absolute limits, and build a system that is truly portable and distributed.

smart_ass

Above: Amibian.js is created in Smart Pascal and compiled to JavaScript

The service layer is written purely in node.js (JavaScript) which guarantees the same behavior regardless of host platform. One of the benefits of using off-the-shelves web technology is that you can physically copy the whole system from one machine to the other without any changes. So if you have a running Amibian.js system on your x86 PC, and copy all the files to an ARM computer – you dont even have to recompile the system. Just fire up the services and you are back in the game.

Now before you dismiss this as “yet another web mockup” please remember what I said about JavaScript: the JavaScript in 2018 is not the JavaScript of 2008. No other language on the planet has seen as much development as JavaScript, and it has evolved from a “browser toy” – into the most important programming language of our time.

So Amibian.js is not some skin-deep mockup of a desktop (lord knows there are plenty of those online). It implements advanced technologies such as remote filesystem mapping, an object-oriented message protocol (Ragnarok), RPCS (remote procedure call invocation stack), video codec capabilities and much more — all of it done with JavaScript.

In fact, one of the demos that Amibian.js ships with is Quake III recompiled to JavaScript. It delivers 120 fps flawlessly (browser is limited to 60 fps) and makes full use of standard browser technologies (WebGL).

utube

Click on picture above to watch Amibian.js in action on YouTube

So indeed, the JavaScript we are talking about here is cutting edge. Most of Amibian.js is compiled as “Asm.js” which means that the V8 runtime (the code that runs JavaScript inside the browser, or as a program under node.js) will JIT compile it to highly efficient machine-code.

Which is why Amibian.js is able to do things that people imagine impossible!

Ok, but what does Amibian.js consist of?

Amibian.js consists of many parts, but we can divide it into two categories:

  • A HTML5 desktop client
  • A system server and various child processes

These two categories have the exact same relationship as the X desktop and the Linux kernel. The client connects to the server, invokes procedures to do some work, and then visually represent the response This is identical to how the X desktop calls functions in the kernel or one of the Linux libraries. The difference between the traditional, machine code based OS and our web variation, is that our version doesn’t have to care about the hardware. We can also assign many different roles to Ambian.js (more about that later).

smartdesk

Enjoying other cloud applications is easy with Amibian.js, here is Plex, a system very much based on the same ideas as Amibian.js

And for the record: I’m trying to avoid a bare-metal OS, otherwise I would have written the system using a native programming language like C or Object-Pascal. So I am not using JavaScript because I lack skill in native languages, I am using JavaScript because native code is not relevant for the tasks Amibian.js solves. If I used a native back-end I could have finished this in a couple of months, but a native server would be unable to replicate itself between cloud instances because chipset and CPU would be determining factors.

The Amibian.js server is not a single program. The back-end for Amibian.js consists of several service applications (daemons on Linux) that each deliver specific features. The combined functionality of these services make up “the amibian kernel” in our analogy with Linux. You can think of these services as the library files in a traditional system, and programs that are written for Amibian.js can call on these to a wide range of tasks. It can be as simple as reading a file, or as complex as registering a new user or requesting admin rights.

The greatest strength of Amibian.js is that it’s designed to run clustered, using as many CPU cores as possible. It’s also designed to scale, meaning that it will replicate itself and divide the work between different instances. This is where things get’s interesting, because an Amibian.js cluster doesn’t need the latest and coolest hardware to deliver good performance. You can build a cluster of old PC’s in your office, or a handful of embedded boards (ODroid XU4, Raspberry PI’s and Tinkerboard are brilliant candidates).

But why Amibian.js? Why not just stick with Linux?

That is a fair question, and this is where the roles I mentioned above comes in.

As a software developer many of my customers work with embedded devices and kiosk systems. You have companies that produce routers and set-top boxes, NAS boxes of various complexity, ticket systems for trains and busses; and all of them end up having to solve the same needs.

What each of these manufacturers have in common, is the need for a web desktop system that can be adapted for a specific program. Any idiot can write a web application, but when you need safe access to the filesystem, unified API’s that can delegate signals to Amazon, Azure or your company server, things suddenly get’s more complicated. And even when you have all of that, you still need a rock solid application model suitable for distributed computing. You might have 1 ticket booth, or 10.000 nation wide. There are no systems available that is designed to deal with web-technology on that scale. Yet 😉

Let’s look at a couple of real-life scenarios that I have encountered, I’m confident you will recognize a common need. So here are some roles that Amibian.js can assume and help deliver a solution rapidly. It also gives you some ideas of the economic possibilities.

Updated: Please note that we are talking javascript here, not native code. There are a lot of native solutions out there, but the whole point here is to forget about CPU, chipset and target and have a system floating on top of whatever is beneath.

  • When you want to change some settings on your router – you login to your router. It contains a small apache server (or something similar) and you do all your maintenance via that web interface. This web interface is typically skin-deep, annoying to work with and a pain for developers to update since it’s connected to a native apache module which is 100% dependent on the firmware. Each vendor end up re-inventing the wheel over and over again.
  • When you visit a large museum notice the displays. A museum needs to display multimedia, preferably on touch capable devices, throughout the different exhibits. The cost of having a developer create native applications that displays the media, plays the movies and gives visual feedback is astronomical. Which is why most museums adopt web technology to handle media presentation and interaction. Again they re-invent the wheel with varying degree of success.
  • Hotels have more or less the exact same need but on a smaller scale, especially the larger hotels where the lobby have information booths, and each room displays a web interface via the TV.
  • Shopping malls face the same challenge, and depending on the size they can need anything from a single to a hundred nodes.
  • Schools and education spend millions on training software and programming languages every year. Amibian.js can deliver both and the schools would only pay for maintenance and adaptation – the product itself is free. Kids get the benefit of learning traditional languages and enjoying instant visual feedback! They can learn Basic, Pascal, JavaScript and C. I firmly believe that the classical languages will help make them better programmers as they evolve.

You are probably starting to see the common denominator here?

They all need a web-based desktop system, one that can run complex HTML5 based media applications and give them the same depth as a native operating-system; Which is pretty hard to achieve with JavaScript alone.

Amibian.js provides a rich foundation of more than 4000 classes that developers can use to write large, complex and media rich applications (see Smart Mobile Studio below). Just like Linux and Windows provides a wealth of libraries and features for native application development – Amibian.js aims to provide the same for cloud and embedded systems.

And as the name implies, it has roots in the past with the machine that defined multimedia, namely the Commodore Amiga. So the relation is more than just visually, Amibian.js uses the same system architecture – because we believe it’s one of the best systems ever designed.

If JavaScript is so poor, why should we trust you to deliver so much?

First of all I’m not selling anything. It’s not like this project is something that is going to make me a ton of cash. I ask for support during the development period because I want to allocate proper time for it, but when done Amibian.js will be free for everyone (LGPL). And I’m also writing it because it’s something that I need and that I havent seen anywhere else. I think you have to write software for yourself, otherwise the quality wont be there.

Secondly, writing Amibian.js in raw JavaScript with the same amount of functions and depth would take years. The reason I am able to deliver so much functionality quickly, is because I use a compiler system called Smart Mobile Studio. This saves months and years of development time, and I can use all the benefits of OOP.

Prior to starting the Amibian.js project, I spent roughly 9 years creating Smart Mobile Studio. Smart is not a solo project, many individuals have been involved – and the product provides a compiler, IDE (editor and tools), and a vast run-time library of pre-made classes (roughly 4000 ready to use classes, or building-blocks).

amibian_shell

Writing large-scale node.js services in Smart is easy, fun and powerful!

Unlike other development systems, Smart Mobile Studio compiles to JavaScript rather than machine-code. We have spent a great deal of time making sure we could use proper OOP (object-oriented programming), and we have spent more than 3 years perfecting a visual application framework with the same depth as the VCL or FMX (the core visual frameworks for C++ builder and Delphi).

The result is that I can knock out a large application that a normal JavaScript coder would spend weeks on – in a single day.

Smart Mobile Studio uses the object-pascal language, a dialect which is roughly 70% compatible with Delphi. Delphi is exceptionally well suited for writing large, data driven applications. It also thrives for embedded systems and low-level system services. In short: it’s a lot easier to maintain 50.000 lines of object pascal code, than 500.000 lines of JavaScript code.

Amibian.js, both the service layer and the visual HTML5 client application, is written completely using Smart Mobile Studio. This gives me as the core developer of both systems a huge advantage (who knows it better than the designer right?). I also get to write code that is truly OOP (classes, inheritance, interfaces, virtual and abstract methods, partial classes etc), because our compiler crafts something called a VMT (virtual method table) in JavaScript.

Traditional JavaScript doesn’t have OOP, it has something called prototypes. With Smart Pascal I get to bring in code from the object-pascal community, components and libraries written in Delphi or Freepascal – which range in the hundreds of thousands. Delphi alone has a massive library of code to pick from, it’s been a popular toolkit for ages (C is 3 years older than pascal).

But how would I use Amibian.js? Do I install it or what?

Amibian.js can be setup and used in 4 different ways:

  • As a true desktop, booting straight into Amibian.js in full-screen
  • As a cloud service, accessing it through any modern browser
  • As a NAS or Kiosk front-end
  • As a local system on your existing OS, a batch script will fire it up and you can use your browser to access it on https://127.0.0.1:8090

So the short answer is yes, you install it. But it’s the same as installing Chrome OS. It’s not like an application you just install on your Linux, Windows or OSX box. The whole point of Amibian.js is to have a platform independent, chipset agnostic system. Something that doesn’t care if you using ARM, x86, PPC or Mips as your CPU of preference. Developers will no doubt install it on their existing machines, Amibian.js is non-intrusive and does not affect or touch files outside its own eco-system.

But the average non-programmer will most likely setup a dedicated machine (or several) or just deploy it on their home NAS.

The first way of enjoying Amibian.js is to install it on a PC or ARM device. A disk image will be provided for supporters so they can get up and running ASAP. This disk image will be based on a thin Linux setup, just enough to get all the drivers going (but no X desktop!). It will start all the node.js services and finally enter a full-screen web display (based on Chromium Embedded) that renders the desktop. This is the method most users will prefer to work with Amibian.js.

The second way is to use it as a cloud service. You install Amibian.js like mentioned above, but you do so on Amazon or Azure. That way you can login to your desktop using nothing but a web browser. This is a very cost-effective way of enjoying Amibian.js since renting a virtual instance is affordable and storage is abundant.

The third option is for developers. Amibian.js is a desktop system, which means it’s designed to host more elaborate applications. Where you would normally just embed an external website into an IFrame, but Amibian.js is not that primitive. Hosting external applications requires you to write a security manifest file, but more importantly: the application must interface with the desktop through the window’s message-port. This is a special object that is sent to the application as a hand-shake, and the only way for the application to access things like the file-system and server-side functionality, is via this message-port.

Calling “kernel” level functions from a hosted application is done purely via the message-port mentioned above. The actual message data is JSON and must conform to the Ragnarok client protocol specification. This is not as difficult as it might sound, but Amibian.js takes security very seriously – so applications trying to cause damage will be promptly shut down.

You mention hosted applications, do you mean websites?

Both yes and no: Amibian.js supports 3 types of applications:

  • Ordinary HTML5/JS based applications, or “websites” as many would call them. But like I talked about above they have to establish a dialog with the desktop before they can do anything useful.
  • Hybrid applications where half is installed as a node.js service, and the other half is served as a normal HTML5 app. This is the coolest program model, and developers essentially write both a server and a client – and then deploy it as a single package.
  • LDEF compiled bytecode applications, a 68k inspired assembly language that is JIT compiled by the browser (commonly called “asm.js”) and runs extremely fast. The LDEF virtual machine is a sub-project in Amibian.js

The latter option, bytecodes, is a bit like Java. A part of the Amibian.js project is a compiler and runtime system called LDEF.

patron_asm2

Above: The Amibian.js LDEF assembler, here listing opcodes + disassembling a method

The first part of the Amibian.js project is to establish the desktop and back-end services. The second part of the project is to create the worlds first cloud based development platform. A full Visual Studio clone if you like, that allows anyone to write cloud, mobile and native applications directly via the browser (!)

Several languages are supported by LDEF, and you can write programs in Object Pascal, Basic and C. The Basic dialect is especially fun to work with, since it’s a re-implementation of BlitzBasic (with a lot of added extras). Amiga developers will no doubt remember BlitzBasic, it was used to create some great games back in the 80s and 90s. It’s well suited for games and multimedia programming and above all – very easy to learn.

More advanced developers can enjoy Object Pascal (read: Delphi) or a sub-set of C/C++.

And please note: This IDE is designed for large-scale applications, not simple snippets. The ultimate goal of Amibian.js is to move the entire development cycle to the cloud and away from the desktop. With Amibian.js you can write a cool “app” in BlitzBasic, run it right in the browser — or compile it server-side and deploy it to your Android Phone as a real, natively compiled application.

So any notion of a “mock desktop for HTML” should be firmly put to the side. I am not playing around with this product and the stakes are very real.

But why don’t you just use ChromeOS?

There are many reasons, but the most important one is chipset independence. Chrome OS is a native system, meaning that it’s core services are written in C/C++ and compiled to machine code. The fundamental principle of Amibian.js is to be 100% platform agnostic, and “no native code allowed”. This is why the entire back-end and service layer is targeting node.js. This ensures the same behavior regardless of processor or host system (Linux being the default host).

Node.js has the benefit of being 100% platform independent. You will find node.js for ARM, x86, Mips and PPC. This means you can take advantage of whatever hardware is available. You can even recycle older computers that have lost mainstream support, and use them to run Amibian.js.

A second reason is: Chrome OS might be free, but it’s only as open as Google want it to be. ChromeOS is not just something you pick up and start altering. It’s dependence on native programming languages, compiler toolchains and a huge set of libraries makes it extremely niche. It also shields you utterly from the interesting parts, namely the back-end services. It’s quite frankly boring and too boxed in for any practical use; except for Google and it’s technology partners that is.

I wanted a system that I could move around, that could run in the cloud, on cheap SBC’s. A system that could scale from handling 10 users to 1000 users – a system that supports clustering and can be installed on multiple machines in a swarm.

A system that anyone with JavaScript knowledge can use to create new and exciting systems, that can be easily expanded and serve as a foundation for rich media applications.

What is this Amiga stuff, isn’t that an ancient machine?

In computing terms yes, but so is Unix. Old doesn’t automatically mean bad, it actually means that it’s adapted and survived challenges beyond its initial design. While most of us remember the Amiga for its games, I remember it mainly for its elegant and powerful operating-system. A system so flexible that it’s still in use around the world – 33 years after the machine hit the market. That is quite an achievement.

image2

The original Amiga OS, not bad for a 33-year-old OS! It was and continues to be way ahead of everyone else. A testament to the creativity of its authors

Amibian.js as the name implies, borrows architectural elements en-mass from Amiga OS. Quite simply because the way Amiga OS is organized and the way you approach computing on the Amiga is brilliant. Amiga OS is much more intuitive and easier to understand than Linux and Windows. It’s a system that you could learn how to use fully with just a couple of days exploring; and no manuals.

But the similarities are not just visual or architectural. Remember I wrote that hosted applications can access and use the Amibian.js services? These services implement as much of the original ROM Kernel functions as possible. Naturally I can’t port all of it, because it’s not really relevant for Amibian.js. Things like device-drivers serve little purpose for Amibian.js, because Amibian.js talks to node.js, and node talks to the actual system, which in turn handles hardware devices. But the way you would create windows, visual controls, bind events and create a modern, event-driven application has been preserved to the best of my ability.

But how does this thing boot? I thought you said server?

If you have setup a dedicated machine with Amibian.js then the boot sequence is the same as Linux, except that the node.js services are executed as background processes (daemons or services as they are called), the core server is initialized, and then a full-screen HTML5 view is set up that shows the desktop.

But that is just for starting the system. Your personal boot sequence which deals with your account, your preferences and adaptations – that boots when you login to the system.

When you login to your Amibian.js account, no matter if it’s just locally on a single PC, a distributed cluster, or via the browser into your cloud account — several things happen:

  1. The client (web-page if you like) connects to the server using WebSocket
  2. Login is validated by the server
  3. The client starts loading preferences files via the mapped filesystem, and then applies these to the desktop.
  4. A startup-sequence script file is loaded from your account, and then executed. The shell-script runtime engine is built into the client, as is REXX execution.
  5. The startup-script will setup configurations, create symbolic links (assigns), mount external devices (dropbox, google drive, ftp locations and so on)
  6. When finished the programs in the ~/WbStartup folder are started. These can be both visual and non-visual.

As you can see Amibian.js is not a mockup or “fake” desktop. It implements all the advanced features you expect from a “real” desktop. The filesystem mapping is especially advanced, where file-data is loaded via special drivers; drivers that act as a bridge between a storage service (a harddisk, a network share, a FTP host, Dropbox or whatever) and the desktop. Developers can add as many of these drivers as they want. If they have their own homebrew storage system on their existing servers, they can implement a driver for it. This ensures that Amibian.js can access any storage device, as long as the driver conforms to the driver standard.

In short, you can create, delete, move and copy files between these devices just like you do on Windows, OSX or the Linux desktop. And hosted applications that run inside their own window can likewise request access to these drivers and work with the filesystem (and much more!).

Wow this is bigger than I thought, but what is this emulation I hear about? Can Amibian.js really run actual programs?

Amibian.js has a JavaScript port of UAE (Unix Amiga Emulator). This is a fork of SAE (scripted Amiga Emulator) that has been heavily optimized for web. Not only is it written in JavaScript, it performs brilliantly and thus allows us to boot into a real Amiga system. So if you have some floppy-images with a game you love, that will run just fine in the browser. I even booted a 2 gigabyte harddisk image 🙂

But Amiga emulation is just the beginning. More and more emulators are ported to JavaScript; you have Nes, SNes, N64, PSX I & II, Sega Megadrive and even a NEO GEO port. So playing your favorite console games right in the browser is pretty straight forward!

But the really interesting part is probably QEmu. This allows you to run x86 instances directly in the browser too. You can boot up in Windows 7 or Ubuntu inside an Amibian.js window if you like. Perhaps not practical (at this point) but it shows some of the potential of the system.

I have been experimenting with a distributed emulation system, where the emulation is executed server-side, and only the graphics and sound is streamed back to the Amibian.js client in real-time. This has been possible for years via Apache Guacamole, but doing it in raw JS is more fitting with our philosophy: no native code!

I heard something about clustering, what the heck is that?

Remember I wrote about the services that Amibian.js has? Those that act almost like libraries on a physical computer? Well, these services don’t have to be on the same machine — you can place them on separate machines and thus its able to work faster.

47470965_10155861938320906_4959664457727868928_n

Above: The official Amibian.js cluster, 4 x ODroid XU4s SBC’s in a micro-rack

A cluster is typically several computers connected together, with the sole purpose of having more CPU cores to divide the work on. The cool thing about Amibian.js is that it doesn’t care about the underlying CPU. As long as node.js is available it will happily run whatever service you like – with the same behavior and result.

The official Amibian.js cluster consists of 5 ODroid XU4/S SBC (single board computers). Four of these are so-called “headless” computers, meaning that they don’t have a HDMI port – and they are designed to be logged into and software setup via SSH or similar tools. The last machine is a ODroid XU4 with a HDMI out port, which serves as “the master”.

The architecture is quite simple: We allocate one whole SBC for a single service, and allow the service to copy itself to use all the CPU cores available (each SBC has 8 CPU cores). With this architecture the machine that deals with the desktop clients don’t have to do all the grunt work. It will accept tasks from the user and hosted applications, and then delegate the tasks between the 4 other machines.

Note: The number of SBC’s is not fixed. Depending on your use you might not need more than a single SBC in your home setup, or perhaps two. I have started with 5 because I want each part of the architecture to have as much CPU power as possible. So the first “official” Amibian.js setup is a 40 core monster shipping at around $250.

But like mentioned, you don’t have to buy this to use Amibian.js. You can install it on a single spare X86 PC you have, or daisy chain a couple of older PC’s on a switch for the same result.

Why Headless? Don’t you need a GPU?

The headless SBC’s in the initial design all have GPU (graphical processing unit) as well as audio capabilities. What they lack is GPIO pins and 3 additional USB ports. So each of the nodes on our cluster can handle graphics at blistering speed — but that is ultimately not their task. They serve more as compute modules that will be given tasks to finish quickly, while the main machine deals with users, sessions, traffic and security.

The 40 core cluster I use has more computing power than northern europe had in the early 80s, that’s something to think about. And the pricetag is under $300 (!). I dont know about you but I always wanted a proper mainframe, a distributed computing platform that you can login to and that can perform large tasks while I do something else. This is as close as I can get on a limited budget, yet I find the limitations thrilling and fun!

Part of the reason I have opted for a clustered design has to do with future development. While UAE.js is brilliant to emulate an Amiga directly in the browser – a more interesting design is to decouple the emulation from the output. In other words, run the emulation at full speed server-side, and just stream the display and sounds back to the Amibian.js display. This would ensure that emulation, of any platform, runs as fast as possible, makes use of multi-processing (read: multi threading) and fully utilize the network bandwidth within the design (the cluster runs on its own switch, separate from the outside world-wide-web).

I am also very interested in distributed computing, where we split up a program and run each part on different cores. This is a topic I want to investigate further when Amibian.js is completed. It would no doubt require a re-design of the LDEF bytecode system, but this something to research later.

Will Amibian.js replace my Windows box?

That depends completely on what you use Windows for. The goal is to create a self-sustaining system. For retro computing, emulation and writing cool applications Amibian.js will be awesome. But Rome was not built-in a day, so it’s wise to be patient and approach Amibian.js like you would Chrome OS. Some tasks are better suited for native systems like Linux, but more and more tasks will run just fine on a cloud desktop like Amibian.js.

Until the IDE and compilers are in place after phase two, the system will be more like an embedded OS. But when the LDEF compiler and IDE is in place, then people will start using it en-mass and produce applications for it. It’s always a bit of work to reach that point and create critical mass.

tomes

Object Pascal is awesome, but modern, native development systems are quite demanding

My personal need has to do with development. Some of the languages I use installs gigabytes onto my PC and you need a full laptop to access them. I love Amibian.js because I will be able to work anywhere in the world, as long as a browser and normal internet line is available. In my case I can install a native compiler on one of the nodes in the cluster, and have LDEF emit compatible code; voila, you can build app-store ready applications from within a browser environment.

 

I also love that I can set-up a dedicated platform that runs legacy applications, games – and that I can write new applications and services using modern, off the shelve languages. And should a node in the cluster break down, I can just copy the whole system over to a new, affordable SBC and keep going. No super expensive hardware to order, no absurd hosting fees, and finally a system that we all can shape and use in a plethora of systems. From a fully fledged desktop to a super advanced NAS or Router that use Amibian.js to give it’s customers a fantastic experience.

And yes, I get to re-create the wonderful reality of Amiga OS without the absurd egoism that dominates the Amiga owners to this day. I don’t even know where to begin with the present license holders – and I am so sick of the drama that rolling my own seemed the only reasonable path forward.

Well — I hope this helps clear up any misconceptions about Amibian.js, and that you find this as interesting as I do. As more and more services are pushed cloud-side, the more relevant Amibian.js will become. It is perfect as a foundation for large-scale applications, embedded systems — and indeed, as a solo platform running on embedded devices!

I cant wait to finish the services and cluster this sucker on the ODroid rack!

If you find this project interesting, head over to my Patreon website and get involved! I could really use your support, even if it’s just a $5 “high five”. Visit the project at: http://www.patreon.com/quartexNow

Leaving The Smart Company

October 30, 2018 Leave a comment

Effective immediately (30.10.2018) I am leaving The Smart Company AS and I have re-distributed my shares.

It’s almost unreal to think that it’s close to nine years since I started this project. Smart Mobile Studio continues to be a technology I am passionate about, and I must admit this is a tough call. It has taken me months to arrive at this decision, but I sadly see no other alternative given the circumstances.

In retrospect, we probably released the technology too early. I see more and more Delphi and C++ builder developers waking up to JavaScript and what web technology can do in the right hands. In other words, they are now where I was nine years ago.

When it comes to reasons there is not really much to say. There have been a few internal issues that were unfortunate, but for me this boils down to time, money and vision. Not really anything juicy to share, im simply not interested in being a partner under the terms the board currently operates with, not because I don’t believe in the product, but because I find the modus operandi counter productive.

Having said that, I am thankful for the journey and everything I have learned, and wish the team all the best for the future.

Smart Mobile Studio lives on

Even though I’m leaving the company and am re-distributed my stock, the product will continue without me. I still use and will continue to use Smart Mobile Studio in my work. But I no longer represent the company, nor will I be involved in further development. So my role as head of research and development is over.

smart_ass

My new compiler core and web IDE is written in Smart Pascal

There is a time and place for all things, and while it breaks my heart to hand Smart Mobile Studio over to a future without me; my time right now is better spent at Embarcadero – working to promote and deliver the language I love above all else; namely Delphi.

Besides Embarcadero I do consulting and occasional training sessions. I have also taken on responsibilities connected with my Patreon project. So I have more than enough to keep me occupied, both at Embarcadero and personally.

But this is not a clean cut. There is no animosity involved. I will continue to use Smart Mobile Studio to build cool stuff. I will publish articles on things I make and continue to evolve the QTX Framework (which has been dormant for two years now).

Sincerely

Jon L. Aasenden

Smart Mobile Studio presentation in Oslo

September 28, 2018 Leave a comment

Yesterday evening I traveled to Oslo and held a presentation on Smart Mobile Studio. The response was very positive and I hope that everyone who attended left with some new ideas regarding JavaScript, the direction the world of software is heading – and how Smart Mobile Studio can be of service to Delphi.

Smart Pascal is especially exciting in concert with Rad-Server, where it opens the doors to Node based, platform independent services and sub clustering. With relatively little effort Rad-Server can absorb the wealth that node has to offer through Smart – but on your terms, and under Delphi’s control. The best of both worlds.

You get the stability and structure that makes Delphi so productive, and then infuse that with the flamboyance, flair and async brilliance that JavaScript represents.

More important than technology is the community! It’s been a few years since I took part in the Oslo Delphi Club’s meetups, so it was great to chat with Halvard Vassbotten, Trond Grøntoft, Alf Christoffersen, Torgeir Amundsen and Robin Bakker face to face again. I also had the pleasure of meeting some new Delphi developers.

prespic

Presentation at ABG Sundal Collier’s offices in Oslo

Thankfully the number of attendees were a moderate 14, considering this was my first presentation ever. Last time I visited was when our late Paweł Głowacki presented FMX, and the turnout was in the ballpark of a hundred. So it was an easy-going, laid-back atmosphere throughout the evening.

Conflict of interest?

Some might wonder why a person working for Embarcadero will present Smart Mobile Studio, which some still regard as competition. Smart is not in competition with Delphi and never will be. It is written by Delphi developers for Delphi developers as a means to bridge two worlds. It’s a project of loyalty and passion. We continue because we love what it enables us to do.

The talks on Smart that I am holding now, including the november talk in London, were booked before I started at Embarcadero (so it’s not a case of me promoting Smart in leu of Embarcadero). I also made it perfectly clear when I accepted the job that my work on Smart will continue in my spare time. And Embarcadero is fine with that. So I am free to spend my after-work hours and weekend time as I see fit.

smart_desktop

The Smart Desktop, codename Amibian.js, is a solid foundation for building large-scale web front-ends. Importing Sencha’s JS API’s can be done via our TypeScript wizard

So, after my presentation in London in november Smart Mobile Studio presentations (at least hosted by me) can only take place during weekends. Which is fair and the way it should be.

Recording the English version

Since the presentation last evening was in Norwegian, there was little point in recording it. Norway have a healthy share of Delphi developers, but a programming language available internationally must be presented in English.

techA couple of months back, before I started working for Embarcadero I promised to do a video presentation that would be available on Delphi Developer and YouTube. I very much like to keep that promise. So I will re-do the presentation in English as soon as possible. I would have done it today after work, but buying tech from the US have changed quite dramatically in just a couple of years.

In short: I haven’t received the remaining equipment I ordered for professional video recording and audio podcasting (which is a part of my Patreon offering as well), as such there will be no live video-feed /slash/ webinar – and questions will be limited to either the comment-section on Delphi Developer; or perhaps more appropriate, the Smart Mobile Studio Forums.

I’m hoping to get the HD camera, mic-table-arm and various bits-and-bobs i ordered from the US sometime next week. I have no idea why FedEx have become so difficult lately, but the package is apparently at LaGuardia, and I have to send receipts that document that these items are paid for before they ship them abroad (so the package manifest listing me as the customer, my address, phone number and receipt from the seller is somehow not enough). This is a first for me.

Interestingly they also stopped a package from Embarcadero with giveaways for my upcoming Delphi presentation in Sweden – at which point I had to send them a copy of my work contract to prove that I indeed work for an American company.

But a promise is a promise, so come rain or shine it will be done. Worst case scenario we can put Samsung’s claims to the test and hook up a mic + photo lens and see if their commercials have any merit.

New article series on Delphi and C++ builder

August 7, 2018 4 comments

An army of Delphi developers

It’s been a while since I’ve done some hardcore Delphi articles, and since that is now my job I am happy that I can finally allocate a good chunk of time for that work. Dont worry, there will be plenty of Smart Pascal content too – but I think it’s time to clean up the blog situation a bit. This blog is personal and thus contains a pot-pourri of topics, from programming to 3d printing, embedded hardware to retro-gaming. It’s a fun blog, I enjoy being able to write about things I’m passionate about, but having one blog for each topic makes more sense.

So in the near future I think it’s good that I publish Smart Mobile Studio content (except random stuff and drive-by posts) to http://www.smartmobilestudio.com, and Delphi to Embarcadero’s blog server. If nothing else it will be easier for the readers to deal with. If you only want to read about my Delphi escapades then embedded and retro stuff is not always interesting.

Deep dive into Delphi and C++ builder

So what can be cool to write about? I spent the better part of last weekend pondering this. Delphi articles have a little blind spot between beginner and advanced that I would like to focus on. There are plenty of “learn Delphi” articles out there, and there are likewise a lot of very advanced topics. So hopefully my first series will hit where it should, and be interesting for those in between.

We need a light database

Let’s peek under the hood!

Right, so the last time I read about database coding, and I mean “making your own database engine” was at least 10 years ago. The Delphi community has always been blessed with a large group of insightful and productive people, people who share their knowledge and help others. But everyone is working on something and finding the time to deep dive into subjects like this is not always easy. So hopefully my series on this will at least inspire people to experiment, try new things and fall in love with Delphi like I did.

The second article series that I am working on right now, is getting to grips with C++ builder. This is actually a very fun experiment since it serves more than a single function; I mean, just how hard is it for a Delphi developer to learn C++ ? What can Embarcadero do to help developers feel comfortable on both platforms? What are the benefits for a Delphi developer to learn C/C++?

 

cppbuilder

C++ builder Community Edition rocks!

And yes I have had more than one episode where the new concepts drove me up the wall. It would be the world’s shortest article-series if Delphi Developer didn’t have my back and I didn’t buy books. Say what you will about modern programming, but sometimes you just need to sit down, turn off the computer, and read. Old school but effective.

Reflections

Embarcadero is very different from what I expected. Before I worked here (which is still a bit surrealistic) I envisioned a stereotypical american company, located in some tall office building; utterly remote from its users and the needs of the punters in the field. This past week has forced me to reflect more than I would have liked, and my armour of strong opinions (if not arrogance) has a very visible dent; because the company that has welcomed me with open arms is everything but that imaginary stereotype.

spartan warrior

Et in Borland ego sum

The core of Embarcadero turned out to be a team of dedicated developers that are literally bending backwards to help as many developers as possible. I left yesterdays meeting with a taste of shame in my mouth, because in my blog I have given at least two of the people who now welcomed me, a less than fortunate overhaul in the past. Yet they turned out to be human beings with the exact same interests, passions and goals as myself.

Building large-scale development tools is really hard work. Seriously. As a developer you forget things like marketing, the sales apparatus, the level of support a developer will need, documentation, tutorials. The amount of requests, conflicting requests that is, from users is overwhelming. You have users who focus on mobile who don’t care about legacy VCL support, then you have people who very much need VCL legacy support and dont care at all about mobile platforms; It’s a huge list of groups, topics and goals that is constantly shifting and needs prioritization.

But all in all the Delphi community and Embarcadero is in good shape. They have worked through a lot of old baggage that simply had to be transitioned, and the result is the change we see now: community editions and better dialog with the users. Compare that to the situation we had five years ago, or eight years ago for that matter. The changes have been many and the road long -but with a purpose: Delphi is growing at a healthy rate again.

What will you need and what will we do?

The goal of the Delphi article is to implement the underlying mechanics of a database. I’m not talking about a “file of record” here or something like that, but a page and sequence based filestream and it’s support apparatus for managing blocks and available resources. This forms the basis of all databases, large or small. So we will be coding the nitty-gritty that has to be in place before you venture into expression parsing.

510242661If time allows I will implement support for filters, but naturally a full SQL parser would be over the top. The techniques demonstrated should be more than enough for a budding young developer to take the ball and run with it. The filter function is somewhat close to a “select” statement – and the essence of expression parsing will be in the filter code.

Note: I will skip memory mapping techniques, for one reason only: it can get in the way of understanding the core principles. Once you have the principles under wraps – memory mapping is the natural next step and evolution of the thoughts involved, so it will fall into place in due time.

You wont need anything special, just Delphi. Most of the code will be classical object pascal, but the parser will throw in some generics and operators, so this is a good time to download the community edition or upgrade to a compiler from this century.

The C/C++ articles will likewise have zero dependencies except the community edition of C++ builder. I went out and bought two books, C++ Primer fifth edition and The C++ programming language by Bjarne Stroustrup himself. Which should be on presciption because i fell at sleep

My frontal lobe is already reduced to jello at the sight of these books, but let’s jump in with both feet and see what we make of it from a Delphi developers point of view. I can’t imagine it can be more of a mess than raw webassembly, but C/C++ has a wingspan that rivals even Delphi so it’s wise not to underestimate the curriculum.

OK, let’s get cracking! I will see you all shortly and post the first Delphi article.

Graphics essentials in Smart Mobile Studio 3

August 5, 2018 Leave a comment

JavaScript and the DOM has a few quirks that can be a bit tricky for Delphi developers to instinctively understand. And while our RTL covers more or less everything, I would be an idiot if I said we havent missed a spot here and there. A codebase as large as Smart is like a living canvas; And with each revision we cover more and more of our blind-spots.

Where did TW3Image.SaveToStream vanish?

We used to have a SaveToStream method in TW3Image that took the raw DIB data (raw RGBA pixel data) and emitted that to a stream. That method was never really meant to save a picture in a compliant format, but to make it easy for game developers to cache images in a buffer and quickly draw the pixel-data to a canvas (or push it to localstorage, good if you are making a paint program). This should have been made more clear in the RTL unit, but sadly it escaped me. I apologize for that.

But in this blog-post we are going to make a proper Save() function, one that saves to a proper format like PNG or JPG. It should be an interesting read for everyone.

Resources are global in scope

Before we dig in, a few words about how the browser treats resources. This is essential because the browser is a resource oriented system. Just think about it: HTML loads everything it needs separately, things like pictures, sounds, music, css styles — all these resources are loaded as the browser finds them in the code – and each have a distinct URI (uniform resource identifier) to represent them.

So no matter where in your code you are (even a different form), if you have the URI for a resource – it can be accessed. It’s important to not mix terminology here because URI is not the same as a URL. URI is a unique identifier, an URL (uniform resource location) defines “where” the browser can find something (it can also contain the actual data).

If you look at the C/C++ specs, the URL class inherits from URI. Which makes sense.

Once a resource is loaded and is assigned an URI, it can be accessed from anywhere in your code. It is global in scope and things like forms or parent controls in the RTL means nothing to the underlying DOM.

Making new resources

When you are creating new resources, like generating a picture via the canvas, that resource doesn’t have an URI. Thankfully, generating and assigning an URI so it can be accessed is very simple — and once we have that URI the user can download it via normal mechanisms.

But the really cool part is that this system isn’t just for images. It’s also for raw data! You can actually assign a URI to a buffer and make that available for download. The browsers wont care about the content.

If you open the RTL unit SmartCL.System.pas and scroll down to line 107 (or there about), you will find the following classes defined:


  (* Helper class for streams, adds data encapsulation *)
  TAllocationHelper = class helper for TAllocation
    function  GetObjectURL: string;
    procedure RevokeObjectURL(const ObjectUrl: string);
  end;

  TW3URLObject = static class
  public
    class function  GetObjectURL(const Text, Encoding, ContentType, Charset: string): string; overload;
    class function  GetObjectURL(const Text: string): string; overload;
    class function  GetObjectURL(const Stream: TStream): string; overload;
    class function  GetObjectURL(const Data: TAllocation): string; overload;
    class procedure RevokeObjectURL(const ObjectUrl: string);

    // This cause a download in the browser of an object-url
    class procedure Download(const ObjectURL: string; Filename: string); overload;
    class procedure Download(const ObjectURL: string; Filename: string;
          const OnStarted: TProcedureRefS); overload;
  end;

The first class, TAllocationHelper, is just a helper for a class called TAllocation. TAllocation is the base-class for objects that allocate raw memory, and can be found in the unit System.Memory.Allocation.pas.
TAllocation is really central and more familiar classes like TMemoryStream expose this as a property. The idea here being that if you have a memory stream with something, making the data downloadable is a snap.

Hopefully you have gotten to know the central buffer class, TBinaryData, which is defined in System.Memory.Buffer. This is just as important as TMemoryStream and will make your life a lot easier when talking to JS libraries that expects an untyped buffer handle (for example) or a blob (more on that later).

The next class, TW3URLObject, is the one that is of most interest here. You have probably guessed that TAllocationHelper makes it a snap to generate URI’s for any class that inherits from or expose a TAllocation instance (read: really handy for TMemoryStream). But TW3URLObject is the class you want.

The class contains 3 methods with various overloading:

  • GetObjectURL
  • RevokeObjectURL
  • Download

I think these are self explanatory, but in short they deliver the following:

  • GetObjectURL creates an URI for a resource
  • RevokeObjectURL removes a previously made URI from a resource
  • Download triggers the “SaveAs” dialog so users can, well, save the data to their local disk

The good news for graphics is that the canvas object contains a neat method that does this automatically, namely the ToDataUrl() function, which is a wrapper for the raw JS canvas method with the same name. Not only will it encode your picture in a normal picture format (defaults to png but supports all known web formats), it will also return the entire image as a URI encoded string.

This saves us the work of having to manually call GetObjectURL() and then invoke the save dialog.

Making some offscreen graphics

TW3Image is not meant for drawing, it’s like Delphi’s TImage and is a graphics container. So before we put a TW3Image on our form we are going to create the actual graphics to display. And we do this by creating an off-screen graphics context, assign a canvas to it, draw the graphics, and then encode the data via ToDataUrl().

To make things easier, lets use the Delphi compatible TBitmap and TCanvas classes. These can be found in SmartCL.Legacy. They are as compatible as I could make them.

  • Browsers only support 32 bit graphics, so only pf32bit is allowed
  • I havent implemented checkered, diagonal or other patterns – so bsSolid and bsClear are the only brush modes for canvas (and pen style as well).
  • Brush doesn’t have a picture property (yet), but this will be added later at some point. I have to replace the built-in linedraw() method with the Bresham algorithm for that to happen (and all the other primitives).
  • When drawing lines you have to call Stroke() to render. The canvas buffers up all the drawing operations and removes overlapping pixels to speed up the final drawing process — this is demanded by the browser sadly.

Right, with that behind us, lets create an off-screen bitmap, fill the background red and assign it to a TW3Image control.

To replicate this example please use the following recipy:

  1. Start a new “visual components project”
  2. Add the following units to the uses clause:
    1. System.Colors
    2. System.Types.Graphics
    3. SmartCL.Legacy
  3. Add a TW3Button to the form
  4. add a TW3Image to the form
  5. Save your project
  6. Double-Click on the button. This creates a code entry point for the default event, which for a button is OnClick.

Let’s populate the entry point with the following:

procedure TForm1.W3Button1Click(Sender: TObject);
var
  LBitmap:  TBitmap;
  LRect:    TRect;
begin
  LBitmap := TBitmap.Create;
  try
    LBitmap.Allocate(640, 480);
    LRect := TRect.Create(0, 0, LBitmap.width-1, LBitmap.Height-1);
    LBitmap.Canvas.Brush.Color := clRed;
    LBitmap.Canvas.FillRect(LRect);

    w3image1.LoadFromUrl( LBitmap.Canvas.ToDataURL('image/png') );

  finally
    LBitmap.free;
  end;
end;

The code above creates a bitmap, which is an off-screen (not visible) graphics context. We then set a background color to use (red) and fill the bitmap with that color. When this is done we load the picture-data directly into our TW3Image control so we can see it.

Triggering a download

With the code for creating graphics done, we now move on to the save mechanism. We want to download the picture when the user clicks the button.

offscreen

Offscreen graphics is quite fun once you know how it works

Since the image already have an URI, which it get’s when you call the ToDataURL() method, we don’t need to mess around with blob buffers and generating the URI manually. So forcing a download could not be simpler:

procedure TForm1.W3Button1Click(Sender: TObject);
var
  LBitmap:  TBitmap;
  LRect:    TRect;
begin
  LBitmap := TBitmap.Create;
  try
    LBitmap.Allocate(640, 480);
    LRect := TRect.Create(0, 0, LBitmap.width-1, LBitmap.Height-1);
    LBitmap.Canvas.Brush.Color := clRed;
    LBitmap.Canvas.FillRect(LRect);

    var LEncodedData:= LBitmap.Canvas.ToDataURL('image/png');
    w3image1.LoadFromUrl(LEncodedData);

    TW3URLObject.Download( LEncodedData, 'picture.png');

  finally
    LBitmap.free;
  end;
end;

Note: The built-in browser in Smart doesn’t allow save dialogs, so when you run this example remember to click the “open in browser” button on the execute window. Then click the button and voila — the image is downloaded directly.

Well, I hope this has helped! I will do a couple of more posts on graphics shortly because there really is a ton of cool features here. We picked heavily from various libraries when we implemented TW3Canvas and TCanvas, so if you like making games or display data – then you are in for a treat!

Delphi community edition, learn real coding

August 2, 2018 8 comments

Update: I updated the text to better point out writing in past-tense at one point. I apologize for not catching the formulation quicker, but I have edited the text to better reflect this now.

embheader

With the release of the community edition of Delphi and C++ builder, Embarcadero is finally making Delphi accessible to anyone who wants to enjoy the rich flavour of object-pascal that Delphi represents. In my 30+ years of coding I have yet to find a language or development toolkit as creative as object pascal, with Delphi being the flagship compiler and toolkit. Java and C# might appeal to some, but for developers solving real problems out there, the stability of Delphi is hard to match (more about that later).

Besides, object-pascal is fun, highly creative and easy to learn! Just imagine the wealth of knowledge a language that has stood the test of time has to offer!

Finally a straight up license

The license Embarcadero has landed on is easy to understand and straight to the point: the community edition is free for open source projects, and you can use it for commercial products until your sales reach a certain sum – and then you are expected to buy it. I wish I had this back in the day, I bought my first Delphi with money from my student loan.

maxresdefault

Unreal engine operates with a similar license

This license is incidentally the same used by market leading game and multimedia companies. Both Crytech (CryoEngine) and Epic Games (Unreal engine) operate with the same concept. Instead of charging you a sum up-front, you can create your product and pay when your earnings justify it. Unreal-engine has a fixed percentage if memory serves me correct. So the Embarcadero license is more than fair.

What the community edition of Delphi and C++ builder means in practical terms, is that you get to learn, build and bring your idea to market without that initial investment. When you boat floats and you make money, then you pay for the toolbox that helped you be successful.

If you are a startup company with investors and limited funds, you get to adjust your license fee to your runway after the fact rather than before. So if your product tanks and you never make the expected sum; well that’s one bill less to worry about.

The myth of free Microsoft products

visual_studioOne of the things I often hear when talking to developers, is the “visual studio myth”. The notion that Visual Studio is free and there are no strings attached. And this is a myth, just to be clear. Microsoft have ridicules amounts of money so they could afford to lend you Visual Studio for five years (which was how they operated until very recently). If you checked the license for Visual Studio that’s what it said: you get to use it for five years, then you better pony up the cash. And by that time you have no doubt advanced to Enterprise level, which means that check will be signed in blood.

So this illusion that Visual Studio is free, is just that. Young developers are just as likely to use a pirated copy as violating a community agreement – so even today with the subscription model and ordinary trial they don’t notice the devil lurking in the details.

But for entrepreneurs that are starting from scratch, that need to set up a budget for their product that a board or single investor can trust, well it’s hard work because development is never an exact science. The coding part is, but it’s the human factor that is challenged, not the technology. It’s the spirit and individuals ability to see solutions where others see only walls that is tested; especially when you are making something truly unique. Something that doesn’t exist yet.

And once you are in that basket and have your entire product, perhaps even your career, riding on the investors being happy (which are rarely developers I might add) – the temptation of going “all in” is very real and very tangible. Let’s use MSSQL since we already have a VS license. Let’s use IIS instead and get rid of Apache. Lets use Sharepoint since we get a nice discount. Complete dependency doesn’t take long.

Now it’s no secret that my brief affair with C# makes me biased, and I am biased. Proudly biased. Bias on tap even. This is an object pascal blog where all things object pascal is loved and valued. So read my articles while imagining me with a cheeky smile in the corner of my mouth, and a slight sparkle in my eyes.

But in all fairness, the new Delphi community license is up-front, no hidden fees, honest and direct. If it wasnt.. well, I have a history of shooting myself in both feet by being painfully honest. I cant find anything wrong with the license and believe me I have tried. This is christmas and my birthday all rolled into one! Embarcadero has put their ears to the ground and listened to their customers.

With this in place Embarcadero is cementing a foundation of growth for our community, the languages they deliver and our future.

Rock solid

Delphi has always been known for producing rock solid, reliable database solutions. Delphi is awesome because it covers the whole spectrum of coding, from low level procedural dll files to system services, industrial scale servers, desktop and mobile applications – the list goes on. There are 3 million active Delphi developers around the world. Not to mention the millions more relying on older versions of Delphi or alternative compilers to power their businesses.

livebindings

Visually bind database fields to containers, its details like this that saves time

If you mentally jump into a time machine and travel back to the 1980s, then slowly walk along the timeline and look at the changes in computing. Look at all the challenges before Delphi and how in 1995 Delphi took the world by storm. Into Delphi, Anders Hejlsberg and his team invested all their knowledge and everything they had learned from previous compilers and run-time libraries. This investment never stopped. There have been many architects involved over the years, each adding their contribution.

The amount of skill, insight, technique and dedication is breathtaking.

C# might be the cool kid on the block right now, but it’s painfully unsuitable for a wide range of tasks. Tasks that require a programming language with more depth. There is also something to be said about the test of time. Delphi and C++ builder have decades of evolution behind them. Many of the core principles were inherited from Turbo Pascal which dominated the 1980s and early 1990s.

And let me back that up with an example:

I used to do some work for a Norwegian company that delivers POS terminals to most of northern europe. When I got there they had a C# department and a Delphi department. Obviously I thought they wanted me to work on the Delphi codebase, but to my surprise they threw me into C#.

While I was there I noticed that Delphi was used on the hardware, the actual terminals themselves and the data transmissions. POS terminals is a potentially fragile but important instrument for any store; it has to operate 24/7 and a single mistake can be a financial disaster. I doubt more needs to be said here.

terminal

A POS terminal consists of many parts, here showing the card reader. Instability in the terminal can lead to loss of data, corrupted backups and network problems

The irony in all this was – that two years earlier they had tried to replace Delphi on the terminals with C#. They invested millions into rewriting the whole thing from scratch. But the rollout of this monstrosity was a total fiasco.

The bro-grammer’s forgot that some things are there for a reason. They neglected the subtle nuances of how each language works and how code behaves under extreme conditions; conditions where ram, storage space and cpu power are severely limited. On cheap, low-powered embedded boards even the slightest fluctuation in CPU activity can tank the whole system.

C# and Java were unfit because the GC (garbage collector) would kick in on random intervals to clean up the heap, this caused CPU spikes. The spikes were enough to freeze the terminal for a brief second, disturbing network activity, disk operations and database stability. It was the first time since the early 90s that I actually saw “Disk C: has a read-write error” dialog. I had to bite my lip to not laugh out loud. I tried so hard, honestly.

The glorified update was haunted by broken transmissions, un-responsive UIs and ruined backups (the device backs up its receipt database both locally and remotely many times a day). After a couple of weeks they rolled back the whole thing. Customers demanded their old system back. The system written in Delphi (and if you think the C# “native image compiler” from Microsoft made things better, think again).

So Delphi and object pascal still powers a large amount of financial transactions in northern europe. You will find Delphi used by the government, security companies, oil companies, POS brokers, ATM’s, missile guidance systems – anywhere where a high level of reliability is essential.

Getting started

Jumping into a new programming language or learning your first one can be daunting. Thankfully Delphi has been around almost as long as C/C++ (3 years younger) so there is plenty of knowledge online, most of it free (always google something before asking on forums, make that a habit).

tomes

Study the classics that teaches you how and things work

But to really save you time I urge you to buy a couple of books on Delphi. Now before you run off to Amazon or google around, there are two types of books you want.

You want a book that teaches you Delphi in general, a modern book that shows you OOP, generics and all the features that were added to Delphi after the XE version naming. So make sure you buy a book that teaches you Delphi from (at the very least) XE6 and upwards. Delphi “Berlin” or “Tokyo” is perfect.

The next book has to do with technique. What makes Delphi so incredibly powerful is this awesome depth. You can write libraries in hand optimized assembly code if you want – or you can write object-oriented, generics driven high-level mobile apps. Between those two extremes is a wealth of topics, including system services, your own servers, every database engine known to mankind and much, much more.

But you want a good book that teaches you techniques, techniques that underpin all the cool high-level features people take for granted. The most cherished book you will ever own for Delphi, is The tomes of Delphi: Algorithms and data structures (catchy title, but this is a book you can come back to over many years).

Now go download that thing and enjoy! Welcome to the coolest language in the world!

Happy coding!

What is new in Smart Mobile Studio 3.0

July 16, 2018 1 comment

Trying to sum up the literally thousands of changes we have done in Smart Mobile Studio the past 12 months is quite a challenge. Instead of just blindly rambling on about every little detail – I’ll try to focus on the most valuable changes; changes that you can immediately pick up and experience for yourself.

Scriptable css themes

theme_structure

A visual control now has its border and background styled from our pre-defined styles. The styles serve the same function in all themes even though they look different.

This might not feel like news since we introduced this around xmas, but like all features it has matured through the beta phases. The benefits of the new system might not be immediately obvious.

So what is so fantastic about the new theme files compared to the old css styling?

We have naturally gone over every visual control to make them look better, but more importantly – we have defined a standard for how visual controls are styled. This is important because without a theme system in place, making application “theme aware” would be impossible.

  • Each theme file is constructed according to a standard
  • A visual control is no longer styled using a single css-rule (like we did before), but rather a combination of several styles:
    • There are 15 background styles, each with a designated role
    • There are 14 borders, each designed to work with specific backgrounds
    • We have 4 font sizes to simplify what small, normal, medium and large means for a particular theme.
  • A theme file contains both CSS and Smart pascal code
  • The code is sandboxed and has no access to the filesystem or RTL
  • The code is executed at compile time, not runtime (!). So the code is only used to generate things like gradients based on constants; “scaffolding” code if you will that makes it easier to maintain and create new themes.

Optimized and re-written visual controls

Almost all our visual controls have been re-written or heavily adjusted to meet the demands of our users. The initial visual controls were originally designed as examples, following in the footsteps of mono where users are expected to work more closely with the code.

To remedy this we have gone through each control and added features you would expect to be present. In most cases the controls are clean re-writes, taking better advantage of HTML5 features such as flex-boxing and relative positions (you can now change layout mode via the PositionMode property. Displaymode is likewise a read-write property).

flexing

Flex boxing relieves controls of otherwise expensive layout chores and evenly distributes elements

Flex-boxing is a layout technique where the browser will automatically stretch or equally distribute screen real estate for child elements. Visual controls like TW3Toolbar and TW3ListMenu makes full use of this – and as a result they are more lightweight, requires no resize code and behave like native controls.

Momentum scrolling as standard

Apple have changed the rules for scrolling 3 times in the past eight years, and it’s driving HTML/JS developers nuts every time. We decided years ago that we had enough and implemented momentum scrolling ourselves written in Smart Pascal. So no matter if Apple or anyone else decides to make life difficult for developers – it wont bother us.

momentum

Momentum scrolling with indicator (or scrollbars) are now standard for all container controls and lists.

Our new TW3Scrollbox and (non visual) TW3ScrollController means that all our container and list controls supports GPU powered momentum scrolling by default. You can also disable this and use whatever default method the underlying web-view or browser has to offer.

Bi-directional Tab control

A good tab control is essential when making mobile and web applications, but making one that behaves like native controls do is quite a challenge. We see a lot of frameworks that have problems doing the bi-directional scrolling that mobile tabs do, where the headers scroll in-place as you click or touch them – and the content of the tab scroll in from either side (at the same time).

tabcontrol

Thankfully this was not that hard to implement for us, since we have proper inheritance to fall back on. JS developers tend to be limited to prototype cloning, which makes it difficult to build up more and more complex behavior. Smart enjoys the same inheritance system that Delphi and C++ uses, and this makes life a lot easier.

Google Maps control

Not exactly hard to make but a fun addition to our RTL. Very useful in applications where you want to pinpoint office locations.

google-maps-android-100664872-orig

Updated ACE coding editor

ACE is by many regarded as the de-facto standard text and code editor for JavaScript. It is a highly capable editor en-par with SynEdit in the Delphi and C++ world. This is actually the only visual control that we did not implement ourselves, although our wrapper code is substantial.

ace

Ace comes with a wealth of styles (color themes) and support for different programming languages. It can also take on the behavior of other editors like emacs (an editor as old as Unix).

We have updated Ace to the latest revision and tuned the wrapper code for speed. There was a small problem with padding that caused Ace to misbehave earlier, this has now been fixed.

The Smart Desktop, windowing framework

People have asked us for more substantial demos of what Smart Mobile Studio can do. Well this certainly qualifies. It is probably the biggest product demo ever made and represents a complete visual web desktop with an accompanying server (the Ragnarok Websocket protocol).

37013365_10155541149775906_3122577366065348608_o

The Smart Desktop showcases some of the power Smart Mobile Studio can muster

It involves quite a bit of technology, including a filesystem that uses the underlying protocol to browse and access files on the server as if they were local. It can also execute shell applications remotely and pipe the results back.

A shell window and command-line system is also included, where commands like “dir” yields the actual directory of whatever path you explore on the server.

Since the browser has no concept of “window” (except a browser window) this is fully implemented as Smart classes. Moving windows, maximizing them (and other common operations) are all included.

The Smart desktop is a good foundation for making large-scale, enterprise level web applications. Applications the size of Photoshop could be made with our desktop framework, and it makes an excellent starting-point for developers involved in router, set-top-boxes and kiosk systems.

Node.JS and server-side technology

While we have only begun to expand our node.js namespace, it is by far one of the most interesting aspects of Smart Mobile Studio 3.0. Where we only used to have rudimentary support (or very low-level) for things like http – the SmartNJ namespace represents high-level classes that can be compared to Indy under Delphi.

As of writing the following servers can be created:

  • HTTP and HTTPS
  • WebSocket and WebSocket-Secure
  • UDP Server
  • Raw TCP server

The cool thing is that the entire system namespace with all our foundation code, is fully compatible and can be used under node. This means streams, buffers, JSON, our codec classes and much, much more.

I will cover the node.js namespace in more detail soon enough.

Unified filesystem

The browser allows some access to files, within a sandboxed and safe environment. The problem is that this system is completely different from what you find under phonegap, which in turn is wildly different from what node.js operates with.

In order for us to make it easy to store information in a unified way, which also includes online services such as Azure, Amazon and Dropbox — we decided to make a standard.

filesys

The Smart Desktop shows the filesystem and device classes in action. Here accessing the user-account files on the server both visually and through our command-line (shell) application.

So in Smart Mobile Studio we introduce two new concepts:

  • Storage device classes (or “drivers”)
  • Path parsers

The idea is that if you want to save a stream to a file, there should be a standard mechanism for doing so. A mechanism that also works under node, phonegap and whatever else is out there.

For the browser we went as far as implementing our own filesystem, based on a fast B-Tree class that can be serialized to both binary and JSON. For Node.js we map to the existing filesystem methods – and we will continue to expand the RTL with new and exciting storage devices as we move along.

Path parsers deals with how operative-systems name and deal with folders and files. Microsoft Windows has a very different system from Unix, which again can have one or two subtle differences from Linux. When a Smart application boots it will investigate what platform it’s running on, and create + install an appropriate path parser.

You will also be happy to learn that the unit System.IOUtils, which is a standard object pascal unit, is now a part of our RTL. It contains the class TPath which gives you standard methods for working with paths and filenames.

New text parser

Being able to parse text is important. We ported our TextCraft parser (initially written for Delphi) to Smart, which is a good framework for making both small and complex parsers. And we also threw in a bytecode assembler and virtual-cpu demo just for fun.

Note: The assembler and virtual cpu is meant purely as a demonstration of the low-level routines our RTL has to offer. Most JS based systems run away from raw data manipulation, well that is not the case here.

asmparse

Time to get excited!

I hope you have enjoyed this little walk-through. There are hundreds of other items we have added, fixed and expanded (we have also given the form-designer and property inspector some much needed love) – but some of the biggest changes are shown here.

For more information stay tuned and visit www.smartmobilestudio.com

Smart Mobile Studio: Q&A about v3.0 and beyond

July 1, 2018 4 comments

A couple of days back I posted a sneak-peek of our upcoming Smart Mobile Studio 3.0 web desktop framework; as a consequence my Facebook messenger app has practically exploded with questions.

smart_desktop

The desktop client / server framework is an example of what you can do in Smart

As you can imagine, the questions people ask are often very similar; so similar in fact that I will answer the hottest topics here. Hopefully that will make it easier for everyone.

If you have further questions then either ask them on our product forums or the Delphi Developer group on Facebook.

 

Generics

Yes indeed we have generics running in the labs. We havent set a date on when we will merge the new compiler-core, but it’s not going to happen until (at the earliest) v3.4. So it’s very much a part of Smart’s future but we have a couple of steps left on our time-line for v3.0 through v3.4.

RTTI access

RTTI is actually in the current version, but sadly there is a bug there that causes the code generator to throw a fit. The fix for this depends on a lot of the sub-strata in the new compiler-core, so it will be available when generics is available.

Associative arrays

This is ready and waiting in the new core, so it will appear together with generics and RTTI.

Databases

We have supported databases since day 1, but the challenge with JavaScript is that there are no “standards” like we are used to from established systems like Delphi or Lazarus.

Under the browser we support WebSQL and our own TW3Dataset. We also compiled SQLite from native C to JavaScript so we can provide a fast, lightweight SQL engine for the browser regardless of what the W3C might do (WebSQL has been deprecated but will be around for many years still).

Server side it’s a whole different ballgame. There you have drivers (or modules) for every possible database you can think of, even Firebird. But each module is implemented as the authors see fit. This is where our Database framework comes in, sets a standard, and we then inherit out classes and implement the engines we want.

This framework and standard is being written now, but it wont be introduced until v3.1 and v3.2. In the meantime you have sqlite both server-side and client-side, WebSQL and TW3Dataset.

Attributes

This question is often asked separately from RTTI, but it’s ultimately an essential part of what RTTI delivers.

So same answer: it will arrive with the new compiler-core / infrastructure.

Server-side scripting

22555292_1630289757034374_6911478701417326545_n

The new theme system in action

While we do see how this could be useful, it requires a substantial body of work to make a reality. Not only would we have to implement the whole “system” namespace from scratch since JavaScript would not be present, but we would also have to introduce a a secondary namespace; one that would be incompatible with the whole RTL at a fundamental level. Instead of going down this route we opted for Node.js where creating the server itself is the norm.

 

If we ever add server-side scripting it would be JavaScript support under node.js by compiling the V8 engine from C to asm.js. But right now our focus is not on server-side-scripting, but on cloud building-blocks.

Bytecode compilation

I implemented the assembler and runtime for our bytecode system (LDef) this winter / early spring; So we actually have the means to create a pure bytecode compiler and runtime.

But this is not a priority for us at this time. Smart Mobile Studio was made for JavaScript and while it would be cool to compile Delphi sourcecode to portable bytecodes, such a project would require not just a couple of namespaces – but a complete rewrite of the RTL. The assembler source-code and parser can be found in the “Next Generation Demos” folder (Smart Mobile Studio 3.0 demos). Feel free to build on the codebase if you fancy creating your own language;Get creative and enjoy! **Note: Using the assembler in your own projects requires a valid Smart Mobile license.

Native Apps

It’s interesting that people still ask this, since its one of our central advantages. We already generate native apps via the Phonegap post-processor.

phonegap

Phonegap turns your JS apps into native apps

Phonegap takes your compiled Smart (visual projects only) compiled code, processes it, and spits out native apps for every mobile OS on the market (and more). So you don’t have to compile especially for iOS, Android, Tizen or FireOS — Phonegap generates one for each system you need, ready for AppStore.

So we have native covered by proxy. And with millions of users Phonegap is not going anywhere.

Release date

We are going over the last beta as I type this, and Smart Mobile Studio 3.0 should be available next week. Which day is not easy to say, but at least before next weekend if all goes accoring to plan.

Make sure you visit www.smartmobilestudio.com and buy your license!

Using Smart Mobile Studio under Linux

April 22, 2018 Leave a comment

Every now and then when I post something about Smart Mobile Studio, an individual or two wants to inform me how they cannot use Smart because it’s not available for Linux. While more rare, the same experience happens now and then with OS X.

linux

The Smart desktop demo running in Firefox on Ubuntu, with Quake 3 at 60 fps

While the request for Linux or OS X support is both valid and understandable (and something we take seriously), more often than not these questions can be a pre-cursor to a larger picture; one that touches on open-source, pricing and personal philosophical points of view; often with remarks on pricing.

Truth be told, the price we ask for Smart Mobile Studio is close to symbolic. Especially if you take the time to look at the body of work Smart delivers. We are talking hundreds of hand written units with thousands of classes, each spesifically adapted for HTML5, Phonegap and Node.js. Not to mention ports of popular JavaScript frameworks.

If you compare this to native object pascal development tools with similar functionality, they can set you back thousands of dollars. Our asking price of $149 for the pro edition, $399 for the enterprise edition, and a symbolic $42 for the basic edition, that is an affordable solution. Also keep in mind that this gives you access to updates for a duration of 12 months. When was the last time you bought a full development suite that allows you to write mobile applications, platform independent servers, platform independent system services and HTML5 web applications for less that $400 ?

prices

Our price model is more than reasonable considering what you get

With platform independent we mean that in the true sense of the word: once compiled, no changes are required. You can write a system service on Windows and it will run just fine under Linux or OS X. No re-compile needed. You can take a server and copy it to Amazon or Azure, run it in a cluster or scale it from a single instance to 100 instances without any change. That is unheard of for object pascal until now.

Smart Mobile Studio is the only object pascal development system that delivers a stand-alone IDE, stand-alone compiler, a wast object-oriented run-time library written from scratch to cover HTML5, Node.js and embedded systems that run JavaScript.

And yes, we know there are other systems in circulation, but none of them are even close to the functionality that we deliver. Functionality that has been polished for seven years now. And our RTL is growing every day to capture and expose more and more advanced functionality that you can use to enrich your products.

21272810_1585822718147745_4358597225084661216_o

The RTL class browser shows the depth of our RTL

As of writing we have a team of six people working on Smart Mobile Studio. We have things in our labs that is going to change the way people build applications forever. We were the first to venture into this new landscape. There were nobody we could imitate, draw inspiration from or learn from. We had to literally make the path as we moved forward.

And our vision and goal remains the same today as it was seven years ago: to empower object pascal developers – and to secure their investment in the language and methodology that is object pascal.

Discipline and purpose

There is so much I would like to work on right now. Things I want to add to Smart Mobile Studio because I find them interesting, powerful and I know people are going to love them. But that style of development, the “Garage days” as people call it, is over. It does wonders in the beginning of a project maybe, but eventually you reach a stage where a formal timeline and business plan must be carved in stone.

And once defined, you have to stick to it. It would be an insult to our customers if we pivoted left and right on a whim. Like every company we have room for research, even a couple of skunkwork projects, but our primary focus is to make our foundation rock solid before further growth.

j5

By tapping into established JavaScript frameworks you can cover more than 40+ embedded systems and micro-controllers. More and more hardware supports JS for automation

Our “garage days” ended around three years ago, and through hard work we defined our timeline, business model and investor program. In 2017 we secured enough capital to continue full-time development.

Our timeline has been published earlier, but we can re-visit some core points here:

The visual components that shipped with Smart Mobile Studio in the beginning, were meant more as examples than actual ready-to-use modules. This was common for other development platforms of the day, such as Xamarin’s C# / Mono toolchain, where you were expected to inherit from and implement aspects of a “partial class”. This is also why Smart Pascal has support for partial classes (neither Delphi or Freepascal supports this wonderful feature).

native

One of our skunkwork projects is a custom linux distro that runs your Smart applications directly in the Linux framebuffer. No X or desktop, just your code. Here running “the smart desktop” as the only visual front-end under x86 vmware

Since developers coming from Delphi had different expectations, there was only one thing to do. To completely re-write every single visual control (and add a few new controls) so that they matched our customers expectations. So the first stretch of our timeline has been 100% dedicated to the visual aspects of our RTL. While doing so we have made the RTL faster, more efficient, and added some powerful sub-systems:

  • A dedicated theme engine
  • Unified event delegation
  • Storage device classes
  • Focus and control tracking
  • Support for relative position modes
  • Support for all boxing models
  • Inline linking ( {$R “file.js”} will now statically link an external library)
  • And much, much more

So the past eight months has been all about visual components.

22814515_1630289797034370_9138255627706616601_n

Theming is important

The second stretch, which we are in right now, is dedicated to the non-visual infrastructure. This means in particular Node.js but also touches on non-visual components, TAction support and things that will appear in updates this year.

Node.js is especially important since it allows you to write platform and chipset independent web servers, system services and command-line applications. This is pioneering work and we are the first to take this road. We have successfully tamed the alien landscape of JavaScript, both for client, mobile and server – and terraformed it into a familiar, safe and productive environment for object pascal developers.

I feel the results speak for themselves, and our next update brings Smart Mobile Studio to the next level: full stack cloud and web development. We now cover back-end, middle-ware and front-end technologies. And our RTL now stretches from micro-controllers to mobile application to clustered cloud services.

This is the same technology used by Netflix to process terabytes of data every second on a global scale. Which should tell you something about the potential involved.

Working on Linux

Since Smart Mobile Studio was designed to be a swiss army knife for Delphi and Lazarus developers, capable to reaching segments of the market where native code is unsuitable – our primary focus is Microsoft Windows. At least for now.

Delphi itself is a Windows-based development system, and even though it supports multiple targets, Windows is still the bread and butter of commercial software development.

Like I mentioned above, we have a timeline to follow, and until we have reached the end of that line, we are not prepared to refactor our IDE for Linux or OS X. This might change sooner than people think, but until our timeline for the RTL is concluded, we will not allocate time for making the IDE platform independent.

But, you can actually run Smart Mobile Studio on both Linux and OS X today.

Linux has a system called Wine. This is a system that implements the Windows API, but delegates all the calls to Linux. So when a Windows program calls a WinAPI through Wine, its delegated to Linux variation of the same call. This is a massive undertaking, but it has years of work behind it and functions extremely well.

So on linux you can install it by opening up a shell and typing:

sudo apt-get install wine

I take for granted here that your Linux flavour has aperture installed (I’m using Ubuntu since that is easy to work with), which is the package manager that gives you the “apt-get” command. If you have some other system then just google how to install a package.

With Wine and it’s dependencies installed, you can run the Smart Mobile Studio installer. Wine will create a virtual, sandboxed disk for you – so that all the files end up where they should. Once finished you punch in the license serial number as normal, and voila – you can now use Smart Mobile Studio on Linux.

Note: in some cases you have to right-click the SmartMS.exe and select “run with -> Wine”, but usually you can just double-click the exe file and it runs.

Smart Mobile Studio on OSX

Wine has been ported to OS X, but it’s more user friendly. You download a program called wine-bottler, which takes Smart and bundles it with wine + any dependencies it needs. You can then start Smart Mobile Studio like it was a normal OS X application.

Themes and look

The only problem with Wine is that it doesnt support Windows themes out of the box. It would be illegal for them to ship those files. But you can manually copy over the windows theme files and install them via the Wine config application. Once installed, Smart will look as it should.

By default the old Windows 95 look & feel is used by Wine. Personally I dont care too much about this, it’s being able to code, compile and run the applications that matters to me – but if you want a more modern look then just copy over the Windows theme files and you are all set.

 

 

Multiplication decomposer

April 22, 2018 Leave a comment

Here is a fun little number decomposer I made a while back. As you might know, the ancient egyptians were no strangers to binary. They also didnt use multiplication tables like we do – but instead used a method called “double down”.

To demonstrate this technique I wrote a simple little program that takes a multiplication and then breaks it down to the numbers an egyptian would look-up in his table to find the right answere.

egypt

It’s actually pretty cool. They did not apply multiplication like we do at all, but adding

You will need to drop two textboxes, 3 labels, one button and a memo control on your form (see layout in picture).

 

unit Form1;

interface

uses
  System.Types,
  System.Types.Convert,
  System.Objects,
  System.Time,
  SmartCL.System,
  SmartCL.Time,
  SmartCL.Graphics,
  SmartCL.Components,
  SmartCL.FileUtils,
  SmartCL.Forms,
  SmartCL.Fonts,
  SmartCL.Theme,
  SmartCL.Borders,
  SmartCL.Application,
  SmartCL.Controls.Button,
  SmartCL.Controls.Label,
  SmartCL.Controls.EditBox,
  SmartCL.Controls.Memo;

type

  TDecomposer = Class(TObject)
  private
    FBitValues:   TIntArray;
    FLeftPillar:  TIntArray;
    FRightPillar: TIntArray;
    FSumPillar:   TIntArray;
  protected
    procedure BuildLeftPillar(Number:Integer);
    procedure BuildRightPillar(Multiplier:Integer);
  public
    Property  LeftPillar:TIntArray read FLeftPillar;
    Property  RightPillar:TIntArray read FRightPillar;
    Property  SumPillar:TIntArray read FSumPillar;

    function  DecomposeNumber(aNumber:Integer):String;
    function  DecomposeMultiplication(aNumber,aMultiplier:Integer):String;

    Constructor Create;Virtual;
  end;

  TForm1 = class(TW3Form)
    procedure W3Button1Click(Sender: TObject);
  private
    {$I 'Form1:intf'}
  protected
    procedure InitializeForm; override;
    procedure InitializeObject; override;
    procedure Resize; override;
  end;

implementation

{ TForm1 }

procedure TForm1.W3Button1Click(Sender: TObject);
begin
  var LObj := TDecomposer.Create;
  try
    var LText := LObj.DecomposeMultiplication(StrToInt(w3editbox1.Text),StrToInt(w3editbox2.text));
    w3memo1.text := LText;
    writeln(LText);
  finally
    LObj.free;
  end;
end;

procedure TForm1.InitializeForm;
begin
  inherited;
  // this is a good place to initialize components
end;

procedure TForm1.InitializeObject;
begin
  inherited;
  {$I 'Form1:impl'}
end;

procedure TForm1.Resize;
begin
  inherited;
end;

//#############################################################################
// TDecomposer
//#############################################################################

constructor TDecomposer.Create;
begin
  inherited Create;

  (* Build table of bitvalues *)

  var mValue := 1;
  for var x :=1 to 32 do
  begin
    FBitValues.add(mValue);
    mValue:=mValue * 2;
  end;
end;

procedure TDecomposer.BuildLeftPillar(Number:Integer);
begin
  FLeftPillar.clear;
  if FBitValues.length>0 then
  begin
    var mValue := 1;
    for var x := FBitValues.low to FBitValues.high do
    begin
      if FBitValues[x] <= Number then         FLeftPillar.add(FBitValues[x])       else         break;     end;   end; end; procedure TDecomposer.BuildRightPillar(Multiplier:Integer); begin   FRightPillar.clear;   if FLeftPillar.length>0 then
  begin
    for var x := FLeftPillar.low to FLeftPillar.high do
    begin
      FRightPillar.add(Multiplier);
      Multiplier:=Multiplier * 2;
    end;
  end;
end;

function TDecomposer.DecomposeMultiplication
         (aNumber,aMultiplier:Integer):String;
begin
  var mSum := aNumber * aMultiplier;
  BuildRightPillar(aMultiplier);

  result := aNumber.toString + ' x '
     + aMultiplier.toString
     + ' = '
     + DecomposeNumber(mSum);
end;

function TDecomposer.DecomposeNumber(aNumber:Integer):String;
begin
  FSumpillar.clear;
  FLeftPillar.clear;
  FRightPillar.clear;

  BuildLeftPillar(aNumber);

  for var x := FLeftPillar.low to FLeftPillar.High do
  begin
    if TInteger.getBit(x,aNumber) then
    FSumPillar.add(FBitValues[x]);
  end;

  if FSumPillar.length>0 then
  Begin
    result:=aNumber.ToString + ' = ';
    for var x:=FSumPillar.low to FSumpillar.high do
    begin
      if x=FSumPillar.low then
      result += FSumPillar[x].toString else
      result += ' + ' + FSumPillar[x].toString;
    end;
  end;
end;

function QTX_GetNumberProducer(aNumber:Integer):String;
const
  QTX_BITS:  Array[0..31] of Integer =
  ( 1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024, 2048,
    4096, 8192, 16384, 32768, 65536, 131072, 262144, 524288,
    1048576, 2097152, 4194304, 8388608, 16777216, 33554432,
    67108864, 134217728, 268435456, 536870912, 1073741824,
    2147483648);
Begin
  var LStack: array of string;
  if aNumber > 0 then
  begin
    for var x := QTX_BITS.low to QTX_BITS.high do
    begin
      if TInteger.getBit(x,aNumber) then
        LStack.add(QTX_BITS[x].toString);
      if QTX_BITS[x] >= aNumber then
        break;
    end;

    if LStack.length>0 then
    begin
      result := aNumber.toString + ' = ';
      for var x := LStack.low to LStack.high do
      begin
        if x > LStack.low then
          result += ' + ' + LStack[x]
        else
          result += LStack[x];
      end;
    end;
  end else
    result := "0";
end;

function QTX_GetNumberMultiplier(aFirst,aSecond:Integer):String;
Begin
  var LSum := aFirst * aSecond;
  result := aFirst.ToString() + ' x ' + aSecond.ToString();
  result += ' = ';
  result += QTX_GetNumberProducer(LSum);
end;

initialization
  Forms.RegisterForm({$I %FILE%}, TForm1);
end.

Smart Mobile Studio 3.0 and beyond

March 20, 2018 Leave a comment

cascade_03With Smart Mobile Studio 3.0 entering its second beta, Smart Pascal developers are set for a boost in quality, creativity and power. We have worked extremely hard on the product this past year, including a complete rewrite of all our visual controls (and I mean all). We also introduced a completely new theme engine, one that completely de-couples visual appearance from structural architecture (it also allows scripting inside the CSS theme files).

All of that could be enough for a version bump, but we didn’t stop there. Much of the sub-strata in Smart has been re-implemented. Focus has been on stability, speed and future growth. The system is now divided into a set of name-spaces (System, SmartCL, SmartNJ, Phonegap, and Espruino), making it easier to navigate between the units as well as expanding the codebase in the future.

To better understand the namespaces and why this is a good idea, let’s go through how our units are organized.

smart_namespace

The RTL is made to expand easily and preserve as much functionality as possible

  • The System namespace is the foundation. It contains clean, platform independent code. Meaning code that doesn’t rely on the DOM (browser) or runtime (node). Focus here is on universal code, and to establish common object-pascal classes.
  • Our SmartCL namespace contains visual code, meaning code and controls that targets the browser and the DOM. SmartCL rests on the System namespace and draws functionality from it. Through partial classes we also expand classes introduced in the system namespace. A good example is System.Time.pas and SmartCL.Time.pas. The latter expands the class TW3Dispatch with functionality that will only work in the DOM.
  • SmartNJ is our high-level nodejs namespace. Here you find classes with fairly complex behavior such as servers, memory buffers, processes and auxillary classes. SmartNJ draws from the system namespace just like SmartCL. This was done to avoid multiple implementations of streams, utility classes and common functions. Being able to enjoy the same functionality under all platforms is a very powerful thing.
  • Phonegap is our namespace for mobile devices. A mobile application is basically a normal visual application using SmartCL, but where you access extra functionality through phonegap. Things like access to a device’s photos, filesystem, dialogs and so on is all delegated via phonegap.
  • Espruino is a namespace for working with Espruino micro-controllers. This has been a very low-level affair so far, due to size limitation on these devices. But with our recent changes you can now, when you need to, tap into the system namespace for more demanding behavior.

As you can see there is a lot of cool stuff in Smart Mobile Studio, and our codebase is maturing nicely. With out new organization we are able to expand both horizontally and vertically without turning the codebase into a gigantic mess (the VCL being a prime example of how not to implement a multi-platform framework).

Common behavior

One of the coolest things we have added has to be the new storage device classes. As you probably know the browser has a somewhat “limited” storage mechanism. You are stuck with name-value pairs in the cache, or a filesystem that is profoundly frustrating to work with. To remedy this we took the time to implement a virtual filesystem (in memory filesystem) that emits data to the cache; we also implemented a virtual storage device stack on top of it, one for each target (!).

In short, if a target has IO capability, we have implemented a storage “driver” for it. So instead of you having to write 4-5 different storage mechanisms – you can now write the storage code once, and it works everywhere.

This is a pretty cool system because it doesn’t limit us to local device storage. We can have device classes that talk to Google-Storage, One-Drive, Dropbox and so on. It also opens up for custom storage solutions should you already have this pre-made on your server.

Database support, a quick overview

Databases have always been available in Smart Mobile Studio. We have units for WebSQL, IndexDB and SQLite. In fact, we even compiled SQLite3 from native C code to asm.js, meaning that the whole database engine is now pure JavaScript and no-longer dependant on W3C standards.

smart_db

Each DB engine is implemented according to a framework

Besides these we also have TW3Dataset which is a clean, Smart Pascal implementation of a single table dataset (somewhat inspired by Delphi’s TClientDataset). In our previous beta we upgraded TW3Dataset with a robust expression parser, meaning that you can now set filters just like Delphi does. And its all written in Smart Mobile Studio which means there are no dependencies.

 

And ofcourse, there is also direct connections to Embarcadero Datasnap servers, and Remobjects SDK servers. This is excellent if you have an already existing Delphi infrastructure.

A unified DB framework

If you were hoping for a universal DB framework in beta-2 of v3.0, sadly that will not be the case. The good news is that databases should make it into v3.2 at the latest.

Databases looks simple: table, rows and columns right? But since each database engine known to JavaScript is written different from the next, our model has to take height for these and be dynamic enough to deal with them.

The model we used with WebSQL is turning out to be the best way forward I feel, but its important to leave room for reflection and improvements.

So getting our DB framework established is a priority for us, and we have placed it on our timeline for (at the latest) v3.2. But im hoping to have it done by v3.1. So it’s a little ahead of us, but we need that time to properly evolve the framework.

Smart Desktop [a.k.a Amibian.js]

The feedback we have received on our Smart Desktop demos have been pretty overwhelming. It is also nice to know that our prototype is being used to deliver software to schools and educational centers. So our desktop is not going away!

smart_desktop

Fancy a game of Quake at 60+ fps? Web assembly rocks!

But we are not rushing into this without some thought first. The desktop will become a project type like I have written about many times before. So you will be able to create both the desktop and client applications for it. The desktop is suitable for software that requires a windowing environment (a bit like Sencha or similar frameworks). It is also brilliant for kiosk displays and as a remote application hub.

Our new storage device system came largely from Amibian, and with these now a part of our RTL we can clean up the prototype considerably!

Smart assembler

It may sound like an oxymoron, but a lab project we created while testing our parser framework (system.text.parser unit) turned into an exercise in compiler / assembler making. We implemented a virtual machine that runs instructions represented by bytecodes (fairly straight ahead stuff). It supports the most common assembler methods, vaguely inspired by the Motorolla 68k processor with a good dose of ARM thrown in for good measure.

smart_assembler

Yes that is a full parser, assembler and runtime model

If you ponder why on earth this would be interesting, consider the following: most web platforms allow for scripting by third-party developers. And by opening up for that these, the websites themselves become prone to attacks and security breaches. There is no denying that any JS based framework is very fragile when potentially hundreds of unknown developers are hacking away at it.

But what if you could offer third parties to write plugins using more traditional languages? Perhaps a dialect of pascal, a subset of basic or perhaps C#? Wouldnt that be much better? A language and (more importantly) runtime that you have 100% control over.

While our assembler, disassembler and runtime is still in its infancy (and meant as a demo and excercise), it has future potential. We also made the instructions in such a way that JIT compiling large chunks of it is possible – and the output (or codegen) can be replaced by for example web assembly.

Right now it’s just a curiosity that people can play with. But when we have more time I will implement high-level parsers and codegens that emit code via this assembler. Suddenly we have a language that runs under node.js, in the browser or any modern JS runtime engine – and its all done using nothing but Smart Mobile Studio.

Well, stay tuned for more!

Alternative pointers in Smart Mobile Studio

February 27, 2018 Leave a comment

Smart Mobile Studio already enjoy a rich and powerful set of memory handling classes and methods. If you have a quick look in the memory units (see below) you will find that Smart Mobile Studio really makes JavaScript sing and dance like no other.

As of writing in version 3.0 BETA the following units are dedicated to raw memory manipulation:

  • System.Memory
  • System.Memory.Allocation
  • System.Memory.Buffer
  • System.Memory.Views

Besides these, the unit System.Types.Convert represents the missing link. It contains the class TDataType which converts data between intrinsic (language level) data types and byte arrays.

Alternative pointers

While Smart has probably one of the best frameworks (if not THE best) for memory handling out there, including the standard library that ships with Node.js, the way it works is slightly different from Delphi’s and Freepascal’s approach.

Since JavaScript is reference based rather than pointer based, a marshaling offset mechanism is more efficient in terms of performance; So we modeled this aspect of Smart on how C# in particular organized its memory stuff.

But, is it possible to implement more Delphi like pointers? To some degree yes. The best would be to do this on compiler level, but even without such deep changes to the system you can actually implement a more Delphi-ish interface.

Here is an example of just such a system. It is small and efficient, but compared to the memory units in the RTL it’s much slower. This is also why we abandoned this way of handling memory in the first place. But perhaps someone will find it interesting, or it can help you port over code from Delphi to HTML5.

unit altpointers;

interface

uses
  W3C.TypedArray,
  System.Types,
  System.Types.Convert,
  System.Memory,
  system.memory.allocation,
  System.Memory.Buffer,
  System.Memory.Views;

type

  Pointer = variant;

  TPointerData = record
    Offset: integer;
    Buffer: JArrayBuffer;
    View:   JUint8Array;
  end;

function IncPointer(Src: Pointer; AddValue: integer): Pointer;
function DecPointer(Src: Pointer; DecValue: integer): Pointer;
function EquPointer(src, dst : Pointer): boolean;

// a := a + bytes
operator + (Pointer,   integer): Pointer uses IncPointer;

// a := a - bytes
operator - (Pointer,   integer): Pointer uses DecPointer;

// if a = b then
operator = (Pointer,   Pointer): boolean uses EquPointer;

function  Allocmem(const Size: integer): Pointer;
function  Addr(const Source: Pointer; const Offset: integer): Pointer;
procedure FreeMem(const Source: Pointer);
procedure MemSet(const Target: pointer; const Value: byte); overload;
procedure MemSet(const Target: pointer; const Values: array of byte); overload;
function  MemGet(const Source: pointer): byte; overload;
function  MemGet(const Source: pointer; ReadLength: integer): TByteArray; overload;

implementation

function MemGet(const Source: pointer): byte;
begin
  if (Source) then
  begin
    var SrcData: TPointerData;
    asm @SrcData = @Source; end;
    result := SrcData.View.items[SrcData.Offset];
  end else
  raise Exception.Create('MemGet failed, invalid pointer error');
end;

function MemGet(const Source: pointer; ReadLength: integer): TByteArray;
begin
  if (Source) then
  begin
    var SrcData: TPointerData;
    asm @SrcData = @Source; end;

    var Offset := SrcData.Offset;

    while ReadLength > 0 do
    begin
      result.add( SrcData.View.items[Offset] );
      inc(Offset);
      dec(ReadLength);

      if offset >= SrcData.View.byteLength then
        raise Exception.Create('MemGet failed, offset exceeds memory');
    end;
  end else
  raise Exception.Create('MemGet failed, invalid pointer error');
end;

procedure MemSet(const Target: pointer; const Value: byte);
begin
  var DstData: TPointerData;
  asm @DstData = @Target; end;
  dstData.View.items[DstData.Offset] := value;
end;

procedure MemSet(const Target: pointer; const Values: array of byte);
begin
  if Values.length > 0 then
  begin
    var DstData: TPointerData;
    asm @DstData = @Target; end;

    var offset := DstData.Offset;
    for var x := low(Values) to high(Values) do
    begin
      dstData.View.items[offset] := Values[x];
      inc(offset);
      if offset >= DstData.View.byteLength then
        raise Exception.Create('MemSet failed, offset exceeds memory');
    end;
  end;
end;

function EquPointer(src, dst : Pointer): boolean;
begin
  if (src) then
  begin
    if (dst) then
    begin
      var SrcData: TPointerData;
      var DstData: TPointerData;
      asm @SrcData = @Src; end;
      asm @DstData = @dst; end;
      result := SrcData.buffer = dstData.buffer;
    end;
  end;
end;

function IncPointer(Src: Pointer; AddValue: integer): Pointer;
begin
  if (Src) then
  begin
    // Check that there is an actual change.
    // If not, just return the same pointer
    if AddValue > 0 then
    begin
      // Map source data
      var SrcData: TPointerData;
      asm @SrcData = @Src; end;

      // Calculate new offset, using the current view
      // position as the present location.
      var NewOffset := srcData.Offset;
      inc(NewOffset, AddValue);

      // Make sure the new offset is within the range of the
      // memory buffer. Picky yes, but this is not native land
      if  (NewOffset >=0)
      and (NewOffset  0 then
    begin
      // Map source data
      var SrcData: TPointerData;
      asm @SrcData = @Src; end;

      // Calculate new offset, using the current view
      // position as the present location.
      var NewOffset := srcData.Offset;
      dec(NewOffset, DecValue);

      // Make sure the new offset is within the range of the
      // memory buffer. Picky yes, but this is not native land
      if  (NewOffset >=0)
      and (NewOffset  0 then
  begin
    var Data: TPointerData;
    Data.Offset := 0;
    Data.Buffer := JArrayBuffer.Create(Size);
    Data.View := JUint8Array.Create(Data.Buffer, 0, Size);
    asm
      @result = @data;
    end;
  end else
  raise Exception.Create('Allocmem failed, invalid size error');
end;

function Addr(const Source: Pointer; const Offset: integer): Pointer;
begin
  if (Source) then
  begin
    if offset > 0 then
    begin
      // Map source data
      var SrcData: TPointerData;
      asm @SrcData = @Source; end;

      // Check that offset is valid
      if (Offset >=0) and (offset < srcData.buffer.byteLength) then
      begin
        // Setup new Pointer data
        var Data: TPointerData;
        Data.Buffer := SrcData.Buffer;
        Data.View := SrcData.View;
        Data.Offset := Offset;
        asm
          @result = @data;
        end;
      end else
      raise Exception.Create('Addr failed, offset exceeds memory');
    end else
    raise Exception.Create('Addr failed, invalid offset error');
  end else
  raise Exception.Create('Addr failed, invalid pointer error');
end;

procedure FreeMem(const Source: Pointer);
begin
  if (source) then
  begin
    // Map source data
    var SrcData: TPointerData;
    asm @SrcData = @Source; end;

    // Flush reference and let the GC take care of it
    SrcData.Buffer := nil;
    SrcData.View := nil;
    SrcData.Offset := 0;
    asm
      srcData = {}
    end;
  end else
  raise Exception.Create('FreeMem failed, invalid pointer error');
end;

end.

Using the pointers

As you can probably see from the code there is no such thing as PByte, PWord or PLongword here. We use a clean uint8 typed array that we link to a memory buffer, so “pointer” here is fully byte based despite it’s untyped origins. In reality it just holds a TPointerData structure, but since this is done via asm sections, the compiler cant see it and treats it as a variant.

The operators add support for code like:

var buffer := allocmem(1024);
memset(buffer, $ff);
buffer := buffer + 1;
memset(buffer, $FA)

But using the overloaded memset procedure is a bit more efficient:

var buffer := allocmem(1024);
var bytes := TDataType.StringToBytes('this is awesome!');
memset(buffer, bytes);
buffer := buffer + bytes.length;
// write more data here

While fun to play with and perhaps useful in porting over older code, I highly recommend that you familiarize yourself with classes like TBinaryData that represents a fully managed buffer with a rich number of methods to use.

And ofcourse let us not forget TMemoryStream combined with TStreamWriter and TStreamReader. These will no doubt feel more at home both under HTML5 and Node.js

Note: WordPress formatting of pascal code is not the best. Click here to view the code as PDF.