Next update, a closer look behind the scenes

With the next update right around the corner, what can you expect to find? Well, I am going to give you some insight and preview into the update right here. For some of you it will be like Christmas and new-years eve all rolled up into one. Especially custom control writers who need fine-grained accuracy for their HTML5 components. So let’s visit the secret lab of object pascal fundamentalists and have a peek 🙂

The secret lab that shall not be named

Functional handles

If you know your way around the VJL (visual JavaScript library) then you will notice a few minor changes here and there, especially that we now have more handle types. So far all browser specific types have been references by the THandle type, which is of type variant and thus capable of abstracting everything.

Starting with this update we begin the journey to a fully typed RTL; This means that the ultimate goal is to get rid of THandle all together and replace it with a typed reference to the HTML5 element itself. So instead of a “Handle:THandle” property in TW3EditBox you would actually get “Handle:JInputElement” which gives you direct access to the element from your pascal code.

But we are not there yet. So for now we have begun separating different handle types, which means that user-controls are no longer packing THandle as their handle-property, but rather TControlHandle.

Why did we do this? Well it’s simple. By separating handles into distinct types we can attach helper classes to them, simplifying how you work with and access handles. This allows us to replace one handle type at a time (without breaking backwards compatibility) and do so in a timely manner.

TControlHandle introduces the following helper methods:

  TControlHandleHelper = helper for TControlHandle
    function  Valid:Boolean;
    function  Ready:Boolean;
    function  Parent:TControlHandle;
    function  Root:TControlHandle;
    procedure ReadyExecute(OnReady:TProcedureRef);
    procedure ReadyExecuteAnimFrame(OnRead:TProcedureRef);
    function  Equals(const Source:TControlHandle):Boolean;
  end;

The ReadyExecute() method requires special attention. In short, whenever you create a visual control it can take a few milliseconds before it actually appears. This means that your code both inside the control and outside stand the risk of accessing a control which is not yet attached to the DOM (document object model).

ReadyExecute solves this by attaching a testing routine to this job, namely to continuously check if the handle is ready for use (read: injected into the DOM and is thus capable of being displayed). When the control is ready it will execute the anonymous procedure you provided.

This allows us to do things like this inside our constructor:

Constructor TMyListControl.InitializeControl;
Begin
  inherited;
  Handle.ReadyExecute( Procedure ()
    begin
      BrowserAPI.RequestAnimationFrame( procedure ()
      Begin
        ReSize;
        LayoutChildren;
      end;
    end);
end;

What the above code does, is to initialize the control as usual, but instruct the system to call-back immediately when the handle is inserted into the DOM. At which point we ask the browser to redraw AFTER we have resized and done layout of child elements.

This results in your control smoothly appearing inside it’s container. You don’t need to issue a resize or “nudge” to force a the layout manually, like some controls require right now. All of that is gone and we will revamp our entire control library with new and updated code.

We also have a method which includes the RequestAnimationFrame call automatically, cleverly named ReadyExecuteAnimFrame(). This is a handy method to make use of:

Constructor TMyListControl.InitializeControl;
Begin
  inherited;
  Handle.ReadyExecuteAnimFrame( Procedure ()
    begin
        ReSize;
        LayoutChildren;
    end);
end;

The Root Method is a great way to check if a control is actually attached to the DOM. If it’s attached to the DOM then the root element will be HTMLBodyElement. So this is a handy way of testing for this particular state.

Valid is as the name implies a method for validating that a TControlHandle is, well.. Valid! A valid reference means that it’s assigned a value and that it’s not null. Which would be the case for an invalid reference. It also makes your code look more human:

if Handle.Valid then
Begin
  if not Parent.Handle.Equals(w3form1.handle) then
  Raise Exception.Create("This control cannot be dropped directly on a form");
end;

The parent method will return the immediate parent, it will not traverse the DOM to locate the first owner, but rather just whatever element you have created an element under (form, panel, custom control).

The Ready method does the exact same thing as ReadyExecute, except that it simply performs the test. It doesnt create a time object which keeps checking for a state change, nor does it call back. This is an excellent way to find out of the current control is attached and ready to use.

Design considerations

I have written by font measuring before. I realize that to some people this may not seem very important, but you have to remember that HTML and JavaScript doesn’t have any functions for measuring font metrics (!). Thats right, these things doesn’t exist in the browser. So being able to measure the width and height of a piece of text, taking font-face and style into consideration — is no small feat. In fact you can’t even enumerate what fonts the browser supports, which is another aspect of accuracy we now provide as standard.

Starting with the next update, TW3MovableControl (base class) introduces these methods:

    function getFontInfo:TW3FontInfo;overload;
    function MeasureText(aContent:String):TW3TextMetric;overload;
    function MeasureTextFixed(aContent:String):TW3TextMetric;overload;

    class function MeasureTextSize(const aHandle:THandle;
          const aContent:String):TW3TextMetric;

    class function MeasureTextSizeF(const aHandle:THandle;
          const aWidth:Integer;const aContent:String):TW3TextMetric;

    class function getFontInfo(const aHandle:THandle):TW3FontInfo;overload;

Attributes

This is a slightly more complex feature which may not seem immediately valuable to some, but believe me, a lot of software could not be written without them.

A couple of years ago browsers began to support “user defined attributes”. In short it means that you can now attach your own attributes to HTML elements. The rules are simple: always pre-fix your attributes with “data-” and the browser wont ignore them. If you don’t follow that rule the browser just ignores them (or deletes them).

Let me explain just how important these are:
This update introduces the ability to apply effects with a single call to any custom-control. You can also chain effects together and they will execute in-turn, one after another. Cool right?

Absolutely, but how do you think this was done? The problem is this: Effect objects are created when you call them, but there has to be a list or queue type of mechanism which prevents them from starting all at once. There must be some way of telling effects waiting in escrow – to wait until the control is ready for more effects.
Since effect objects cannot be kept track of easily, not without an expensive list of stack table – the best way to inform the waiting effects that the object is busy, is to write the value to the element itself!

And this is exactly how we solves having multiple effects waiting their turn. When an effect is active, it writes a busy-flag to the HTML element. The other effects keep checking this flag and only execute when the object is not-busy.

This would not be possible without attribute access. Well, at least not without a rather expensive lookup table and cleanup procedure.

Other uses of attributes

Loads. You can now “tag” your own elements with whatever values you like. Perhaps you want to link a visual control to some data? Well, then you would define table and field as attributes. And when the data is ready, query the DOM through a selector to return those elements marked with your “tags”‘s. Voila you can now freely target and populate visual controls without even registering them. A sort of weak-binding type solution.

Visual effects

I mentioned above that the QTX effects library has been built into the RTL. And that is true. This makes it child’s play to make your UI designs become more responsive and alive by responding with effects to touches and callbacks. But don’t over-do it! To much can render your application useless, so use with care!

All TW3CustomControl decendants now supports the following effects:

1. fxFadeOut
2. fxFadeIn
3. fxWarpOut
4. fxWarpIn
5. fxZoomIn
6. fxZoomOut
7. fxScaleTo
8. fxMoveTo
9. fxMoveBy
10. fxMoveUp
11. fxMoveDown
12. fxSizeTo
13. fxScaleDown
14. fxScaleUp

These effects can be daisy-chained to execute after each other, like so:

  w3Button1
    .fxFadeIn(0.2)
    .FxMoveDown(1.2)
    .fxFadeOut(0.3);

Legacy

For all your FreePascal and Delphi coders we have a little treat, namely some classes which will simplify your HTML5 graphics programming life. Like most Delphi programmers you are probably used to good old TCanvas and TBitmap to do your work. Perhaps you want to port over some older Delphi code and cringe at using HTML5’s canvas to do the work?

Well we have re-implemented both TBitmap, TCanvas, TBrush and TPen for you. So go straight ahead and do that port, also be amazed at the wonderful speed enhancement HTML5 offers over native Delphi.

Streams, buffers and memory

I have covered these features before, you can read more about these here:

• Head to Head: Smart Mobile Studio vs. Embarcadero Delphi
• System.Interop, How to for Smart Mobile Studio

Dataset support

Yes. Finally. I know you guys have been asking about this since the beginning. Well starting with the next update we ship TW3CustomDataset and TW3Dataset, which is an in-memory dataset solution.

NodeJS builder will make the current architecture make more sense

It does not connect to anything and is more like TClientDataset in Delphi, but you can use it to cache records you want to send via DataSnap, or via RemObjects, and naturally it’s a perfect match for nodeJS services. It will all make more sense when we ship the nodeJS service designer (ops! Perhaps I should not have mentioned that part).

You will find that in system.dataset, which means you can also reference it from your nodeJS projects. Communication between server and client will be 100% compatible since dataset can be deployed on both sides.

Later I will introduce binary records (everything is now JSon) and in-memory storage which will be super-fast and super efficient. So we havent even begun scratching the surface of databases under HTML5!

And much, much more

These were just some of the features I am adding to SMS this time. We are a team of 4 developers so there will be plenty of bug-fixes and other features, both for the IDE, the compiler and the RTL.

I cant wait to get to work on the next update! There are so much cool stuff in the pipeline I hardly know where to begin!

Head to Head: Smart Mobile Studio vs. Embarcadero Delphi

I know I have been writing a lot about Smart Mobile Studio and memory access lately. But I’m sure you all understand that it’s just as vital in HTML5 applications as it is under native Delphi or C# applications.

V8 JIT powered JavaScript is awesome, even on embedded platforms

Having spent the better part of this weekend optimizing and re-factoring the codebase, I am now able to write 1.000.000 (one million) records to memory in 417 milliseconds. At first I did not believe that it could be possible, there had to be some error somewhere or perhaps the data was not actually written. But using hexdump and inspecting the buffer avails that yes – there is no error, it’s that bloody fast.

    mChunk:=0;
    mMake.Allocate(21000021);
    mStart:=Now;
    for x:=0 to 1000000 do
    begin
      mMake.Write(mChunk + 0,$AAAA0000);
      mMake.WriteFloat32(mChunk + 4,19.24);
      mMake.WriteFloat64(mChunk + 8,24.18);
      mMake.Write(mChunk + 16,false);
      mMake.write(mChunk + 17,$0000AAAA);
      inc(mChunk, 21);
    end;
    mStop:=Now;

This means that JavaScript, in particular Webkit V8 (JIT) code is now faster than native Delphi for standard memory operations (move, fill, read, write).

Graphics humiliation

Some five years back when I proposed on my earlier blog that a compiler producing JavaScript from Object Pascal could be constructed, people laughed. One of the biggest challenges when creating something new is that you face the general consensus of people stuck in the past, fixed to rigid ideas which may or may no longer apply to reality.

This was also the case with Delphi and our community back then. To the majority of programmers at that point in time, JavaScript was deemed a toy; it was to slow, unserious and unstable for anything real and tangible. A proposal that JavaScript could in fact be used as a universal platform for mobile and desktop applications was laughed out of the chat rooms.

Introduction of new ideas is often met by heavy resistance

Thankfully a genius french developer, Eric Grange, which is the maintainer of Delphi Web Script (DWS) came to my aid. What he did was create a HTML5 canvas demo which outperformed Delphi by magnitudes. First using FireFox which is as we all know slightly slower than Webkit, but later he executed the same code in Webkit’s V8, leaving native Delphi in the dust.

So the laughter I had faced was quickly silenced. I had been vindicated and Eric and myself went on to write what to day is Smart Mobile Studio. Eric adapted Delphi Web Script to serve as the compiler framework, adding a JavaScript code-generator which not only realized my ideas; It also went far beyond anything I had envisioned, with a VMT (virtual method table), class inheritance, interfaces, operator overloading, partial classes and the works.

The moral of the story here is that we were never out to humiliate Delphi. Quite the opposite, Smart Mobile Studio was created as a toolkit for Delphi developers. In fact it was created to help Embarcadero venture into new markets. I would never suspected to be treated like we have been by Embarcadero, which has blacklisted our entire team from public events, threatening to poll funding if we are allowed to speak (!)

But it has made one thing clear. I no longer care if Embarcadero is damaged by my products.  With the exception of two individuals working for EMB, they have not given me (or my team) the time of day. Which is strange, because our technology would revolutionize and to some degree re-vitalize the Delphi brand. Something it sorely needs.

Memory matters

Well, today I am pleased to announce that not only does Smart Mobile Studio humiliate Delphi in every graphics operation known to mankind, including 3D rotation of user-controls; Smart Mobile Studio is now faster in every aspect of memory manipulation. This includes:

• Writing un-typed array to memory
• Writing typed arrays to memory
• Moving data between two memory segments
• Moving data within the same memory segment
• Reversing the data inside a memory segment
• Fill a memory segment with char data
• Fill a memory segment with un-typed data
• Fill a memory segment with typed-data

The tests were all done on VMWare Fusion running on an 2.7 Ghz Intel Core i5 iMac with 8 GB memory (1600 Mhz DDR3).
1 Gigabyte of memory was allocated for the VMWare machine, which is running Windows 7 Ultimate Service Pack 2. The tests were performed using Chrome (Smart Mobile Studio ships with Chromium Embedded) and Chrome for OS X.

Webkit has excellent profiling capabilities, here showing profile for 1.000.000 writes

 

Inspection of memory and CPU timeline

The following Delphi versions were used in my tests:

• Delphi 7
• Delphi 2005
• Delphi 2006
• Delphi XE
• Delphi XE3
• Delphi XE6
• Delphi XE7

I also performed the tests using the latest FreePascal compiler, and I am happy to report that FreePascal is the only compiler capable of delivering roughly the same amount of power as V8 (the JavaScript runtime and JIT engine under webkit is called V8).

To be perfectly honest, I have begun to re-compile more and more Delphi projects using FreePascal and Lazarus these days. And we are moving Smart Mobile Studio to FreePascal as well. It gives us the benefit of compiling our product for OS X, Linux and Win32.

It also puts to rest any clause in the Embarcadero License which prevents users from producing executables from a Delphi application (yeah, it’s that screwed up!). So yes, a FreePascal powered version of Smart Mobile Studio will appear sooner or later.

Test yourself

The memory management unit, support for streams and marshaled pointers will be available in the next update of Smart Mobile Studio, so you can download a trial version if you dont own a license and run the tests yourself.

The full source-code will be posted here shortly, it’s identical to the SMS version in both payload and infrastructure.