Archive

Archive for January, 2015

Message handling under HTML5 with Smart Mobile Studio

January 31, 2015 5 comments

In my previous post I informed you that my latest addition to the QTX library is WinAPI like messages. It’s important to underline that this has nothing to do with SMS text-messages, push messages or anything like that; rather it deals with sending messages between forms, controls and routines on an application-wide scale.

This is a lot harder to achieve under JavaScript than you imagine. In a pointer based language you have the freedom of criss-crossing between memory boundaries inside your own program, but JavaScript has no such concept. It doesn’t even know what an object is (!) So being able to pass messages from Object A to Object B where none of them knows about each other is quite a task.

Thankfully HTML5 comes with a message stack built in, which we have made full use of in our API.

Why messages?

The Windows programming API is based on messages. In fact message-handling is the building block of any Microsoft Windows application. If you come from a .net or Visual Basic background you have been shielded from this – but C++ and Delphi programmers will know what I’m talking about. Native Windows programming is all about message dispatching and consumption.

Messages represents a very powerful technology. Actually there are some chores that can be said to be impossible to achieve without them. Being able to broadcast messages which every single class or component in an application, regardless of their relationship or knowledge of each other, are capable of receiving and processing.. well, it goes without saying that messages solves a lot of tough problems.

Note: Messages are great but they are also very easy to abuse! Always make sure you use messages with care, or you may risk turning your mobile app into a proverbial debugging nightmare.

Using messages

Before we look at how to use messages, let’s take a look at what exactly a message is. The message unit is names qtx.msgport.pas and can be found in the quartex library folder. Simply check this out from SVN and copy it to your libraries folder. There is a shortcut to this folder under the start-menu item for Smart Mobile Studio.

If you open up this file you will find the message defined as such:

  (* The TQTXMessageData represents the actual message-data which is sent
     internally by the system. Unlike Delphi or FPC, it is a class rather
     than a record. Also, it does not derive from TObject - and as such
     is suitable for 1:1 JSON mapping.

     Note:  You dont need to free TQTXMessageData objects, Javascript is
            garbage-collected. *)

  TQTXMessageData = class(JObject)
  public
    property    ID: Integer;
    property    Source: String;
    property    Data: String;

    function    Deserialize:String;
    procedure   Serialize(const value:String);

    Constructor Create;
  end;

Right, let’s step through the class field by field..

The ID field

The ID field of the message is just that, the message-id. This can be anything you want. I suggest you define your message ID’s using constants – and also that you start at some +1000 number. Please note that ID 0 through 999 is reserved for Smart Mobile Studio (!).

In the unit there is a constant called CNT_QTX_MESSAGES_BASEID which you can use to define the lower limit of your application specific messages. Below is how I typically define my own messages. Starting CNT (constant), then the name of my application – followed by the name of the functionality. And I always start from CNT_QTX_MESSAGES_BASEID;

const
  CNT_MYAPP_DATALOADING = CNT_QTX_MESSAGES_BASEID + 1;
  CNT_MYAPP_DATALOADED  = CNT_QTX_MESSAGES_BASEID + 2;
  CNT_MYAPP_CALCSTARTED = CNT_QTX_MESSAGES_BASEID + 3;

The source field

The source field is special. HTML5 messages require that you include the source-url from where the message came. This may seem odd, but it makes perfect sense when you realize that messages can be sent between open browser windows. The application running in one window may not come from the same server as the second window. Hence you have to populate this with the current URI.

If you dont care about checking the URI you can simply use “*”, this allows your message to pass though anyway. The security of URI checking is for your benefit, it’s not included against you. Keep that in mind.

The data field

This is where the fun stuff begins, because this string is all yours. This is where you place your message data. Now you may be tempted to think “what? Just a string? Thats not enough!” – but before you jump into that line of thinking, remember that all JavaScript objects serialize to JSON. Which means you can stuff any non TObject derived object into this field.

The non TObject derived object is important. The reason Smart Mobile Studio is better than any other JavaScript compiler out there, is because our compiler generates a VMT, which is short for “virtual method table”. It’s the VMT that gives you true OOP, inheritance, virtual and abstract procedures, interface support etc. etc.

But this also means that our generated code functions more or less like Delphi assembly. Every method in a TObject derived class expects to have a hidden parameter, namely the “self” parameter. Which is a reference to a structure in memory which represents the current object instance. This is not problematic for JSON, but the RTTI info which accompany each method is (!) because RTTI identifiers, which are unique each time you execute an application, will also be duplicated.

So the rule is, that whatever you want to serialize should inherit from JObject, which is a parent-less object without the OOP mechanisms we know and love from Delphi and C++ builder (read: without all that extra data).

But you dont need to know about all that technical mumbo-jumbo. Let’s define a hypotethical message you want to use in your application. It goes like this:

  TMyComplexMessage = class(JObject)
  public
    Property  FirstName:String;
    Property  LastName:String;
    Property  Age:Float;
    Property  Birth:TDateTime;

    function  Deserialize:String;
    procedure Serialize(value:String);

    procedure FromMessage(const msg:TQTXMessageData);
    function  ToMessage:TQTXMessageData;
  end;

The art of turning an object into a JSON structure (or XML for that matter) is called “deserialization”, and not unexpectedly the reverse, turning a JSON string into an object is cleverly called “serialization”. So for simplicity I decided to call our methods just that. What these methods do is take the current object and either turn that into JSON text or, take a JSON text and populate our object with it’s values.

You may think this is overkill, but I tell you, writing good software is all about structure, order and maintainability. The more time you spend on the infrastructure the better your software will be. That is a fact of life.

Next we have the FromMessage and ToMessage, these are just helper methods that allows us to skip the nitty-gritty in our program code, and enables us to send and handle messages with a single line.

The code for these methods looks like this:

procedure TMyComplexMessage.FromMessage(const msg:TQTXMessageData);
begin
  serialize(msg.Data)
end;

function  TMyComplexMessage.ToMessage:TQTXMessageData;
Begin
  result:=new TQTXMessageData();
  result.ID:=CNT_MYAPP_MyComplexMessage;
  result.source:="*";
  result.data:=Deserialize;
end;

function TMyComplexMessage.Deserialize:String;
begin
  result:=JSON.stringify(self);
end;

procedure TMyComplexMessage.Serialize(value:String);
var
  mData:  variant;
begin
  mData:=JSON.Parse(Value);
  self.FirstName:=TMyComplexMessage(mData).FirstName;
  self.LastName:=TMyComplexMessage(mData).LastName;
  self.Age:=TMyComplexMessage(mData).Age;
  self.Birth:=TMyComplexMessage(mData).Birth;
end;

As you can see, it’s all fairly straight forward code. You can even isolate these methods in a base-class and just override Serialize() in each derived type, since that’s where the differences will be.

Sending our message

We are now ready to send our message, but let’s start in reverse – by implementing the receiver first!

  FEventHandler:=TQTXMessageSubscription.Create;
  FEventHandler.SubScribe(CNT_MYAPP_MyComplexMessage,procedure (Message:TQTXMessageData)
    var
      mObj: TMyComplexMessage;
    begin
      mObj:=new TMyComplexMessage();
      mObj.serialize(message.data);
      // "mObj" is now ready to be used!
    end);

Sending a message is simplicity itself, especially with our serialization methods firmly implemented:

  var mMessage:TMyComplexMessage = new TMyComplexMessage();

  mMessage.firstname:="jon";
  mMessage.lastname:="aasenden";
  mMessage.age:="40.9";

  QTX_PostMessage(mMessage.toMessage);

That’s basically how easy it is to send complex messages. You can ofcourse send smaller stuff, if all you want to do is transport a text-string between two components, then creating a complex datatype (message class) is overkill. But if you are writing a serious application you really want to isolate everything in proper classes. It’s so much easier to maintain and extend later.

Messages to solve problems

If you have spent a couple of years in Delphi you know perfectly well that the VCL (Visual Component Library) which has been Delphi’s bread and butter for nearly two decades, is littered with message-use.

Everything from database states to refreshing a GUI elements is handled by messages. In fact, if you take a closer look at TCustomControl – which is the primary control that most visual controls inherit from, you will see that it functions, in company with it’s ancestor TWinControl, as the proverbial switch-box for messages. It catches all the relevant messages from the OS, like key-presses, redraw requests, movement – you name it, and isolates these in event procedures.

I have no desire for Smart Mobile studio to go the same way. It could, ofcourse, but there is little benefit in doing so since the difference between the browser environment and the WinAPI environment is like night and day.

But what the JVL (JavaScript Visual component Library) could be extended with, in order to solve some notification challenges, is messages. I am going to propose that we use messages, especially in the upcoming database component layer. Using messages to notify data-subscribers of updates (and even database aware controls in the future) is a cost effective, safe and elegant solution to an otherwise daunting task.

For example: You have 3 TW3Edit boxes connected to a database-field. Whenever the data in the table changes, all the visual controls should reflect that by changing as well.

Without messages all controls would have to register with the data-source and all types of updates would be represented by a call mechanism. This is perfectly valid, but it represents a lot more code (and potential errors) than using messages to solve the problem.

All connected controls could be notified with something as simple as:

mMsg.db := FDB.name;
mMsg.Table := FDB.Table.Name;
mMsg.FieldName := FDB.Table.Field[mChanged].Name;
mMsg.Value := FDB.Table.Field[mChanged].asVariant;
QTX_BroadcastMessage(mMsg.serialize);

The messages are automatically filtered out by it’s ID, and later by the content of the message. Each message handler only consume one particular type of message – hence the cost of broadcasting is minimal. The “sender” field in this case (as explained at the beginning of this article) would serve as the namespace. Hence different forms with the same name on their controls and listening to the same field would be clearly separated.

So if Form1 is set to receive updates on 3 controls, and Form2 is inactive but set to receive the same updates — then only Form1 will get the updates. This is controlled by the application dispatcher.

In short: You can expect some pretty cool controls in the future 🙂

Well, hope you found this little introduction helpful — now go download QTX 🙂

Messages in Smart Mobile Studio

January 29, 2015 Leave a comment

If you have missed message coding from Delphi under Smart Pascal then you are in for a treat. The QTX library now supports application-wide messages in the browser; allowing you to send and subscribe to as many messages as you like. More or less duplicating the stuff you are used to under WinAPI.

In fact, you can subscribe to the same message in the same object as many times as you want — and also instantiate as many subscription objects as you like.

Receiving messages

Here is how you would create and setup a message-subscription for a form:

procedure TForm1.InitializeObject;
begin
  inherited;
  {$I 'Form1:impl'}

  //create our message subscriber object
  FMsgPort:=TQTXMessageSubscriber.Create;

  // subscribe to message-type $100
  FMsgPort.Subscribe($100,procedure (Message:TQTXMessageData)
    begin
      writeln("Data received:" + message.data);
    end);

  // subscribe to message-type $200
  FMsgPort.Subscribe($200,procedure (Message:TQTXMessageData)
    begin
      writeln("Data received:" + message.data);
    end);
end;

To send real objects through messages, simply use JSON to serialize it, then de-serialize it in your message handler.

Sending messages

The simplest way to send messages are through the two API methods:

  • QTX_SendMessage(Message:TQTXMessageData);
  • QTX_BroadcastMessage(Message:TQTXMessageData);

Alternatively, although highly unpractical, you could create your own message-port. The whole message system uses just a single message-port so you would never really create these yourself. If all you want to do is send and receive messages – you have more or less the same support as Delphi as for WinAPI messages — but simpler and easier to use!

Practical uses

Messages have been a part of Windows since the beginning, and it’s also hard-coded into Delphi. Their uses are almost limitless and you can use then to notify changes across controls and forms, transport data from one place to another — it’s really only creativity that sets the bounds.

What is cool about Smart Mobile Studio – is that you now have the same system under JavaScript. It works in every browser and allows your HTML5 forms and controls to communicate indirectly.

The technology is based on the HTML5 message system, which is a part of modern browsers. It’s extremely efficient and very reliable.

QTX Library for Smart Mobile Studio updated

January 27, 2015 4 comments

QTX (Quartex) is a library written for Smart Mobile Studio users. It extends the run-time library with a wide range of capabilities, including database (dataset) support, custom effects, tag-attribute storage and much, much more.

If you are a serious HTML5 developer using Smart Mobile Studio, then this library is a must. The font and content measurement alone is worth the download – and did I mention that it’s free?

Facebook clone, coded in a couple of hours using QTX for Smart Mobile Studio

Facebook clone, coded in a couple of hours using QTX for Smart Mobile Studio

Library overview

QTX is packed with classes and methods, and I have done my best to make it as easy as possible to understand. I take it for granted that you understand the concept of HTML-Tags, and that you understand that Smart Mobile Studio code creates TAGS when it’s constructor fires, and removes the tag when the destructor executes. As such, we have no need for libraries like JQuery – because the handle (reference) to the element is always known. JQuery is ultimately about extracting tags within a criteria from the DOM (hence “Query” in the title, as in SQL for querying a database).

What we do like however, is plenty of effects – and dead simple syntax for using them. Well it doesnt get any easier than with QTX. The QTX library extends TW3CustomControl with it’s own class helper, meaning that once you have included the qtx.effects unit – all your controls supports the whole range of effects.

Also, effects are executed in sequence. You can trigger 10 effects and they will execute one by one, which is no small feat under HTML5 (but easy with QTX tag attribute support).

Here is a brief overview of the classes and methods you get when installing QTX:

Database

  • TQTXDataset
  • TQTXDatasetField
  • TQTXBooleanField
  • TQTXIntegerField
  • TQTXFloatField
  • TQTXStringField
  • TQTXDatasetFields
  • TQTXFieldDef
  • TQTXFieldDefs

Tag-Attribute Storage

  • TQTXAttrAccess

CSS3 GPU powered animations

  • TQTXMoveAnimation
  • TQTXFadeAnimation
  • TQTXSizeAnimation
  • TQTXAnimationHelper

Effect management

  • TQTXEffectsHelper
    • fxSetBusy
    • fxBusy:boolean
    • fxScaleUp
    • fxScaleDown
    • fxSizeTo
    • fxMoveDown
    • fxMoveUp
    • fxMoveBy
    • fxMoveTo
    • fxScaleTo
    • fxZoomOut
    • fxZoomIn
    • fxWarpIn
    • fxWarpOut
    • fxFadeIn
    • fxFadeOut

Font and content measurement

  • TQTXTextMetric
  • TQTXFontInfo
  • TQTXFontDetector

Helper classes

  • TQTXHandleHelper
  • TQTXIntegerHelper
  • TQTXStringHelper

Media IO management

  • TQTXIOAccess
    • LoadXML
    • LoadFile
    • LoadCSS
    • LoadScript
    • LoadImage
    • PreloadImages

Delayed execution

  • TQTXRuntime
    • DelayedDispatch
    • CancelDelayedDispatch
    • Execute
    • Ready
    • ExecuteDocumentReady

Dynamic stylesheets

  • TQTXStyleSheet

Flicker free, smooth momentum scroll baseclasses

  • TQTXScrollOptions,
  • TQTXScrollController
  • TQTXScrollWindow

3D for any HTML elements

  • TQTXSprite3DController

Custom controls

  • TQTXHeaderButton
  • TQTXBackButton
  • TQTXNextButton
  • TQTXHeaderTitle
  • TQTXHeaderBar
  • TQTXLabel
  • TQTXScrollText

Downloading

Simply point your SVN client to: svn checkout http://qtxlibrary.googlecode.com/svn/trunk/ and grab the units.

Installing

Copy the target folder into your Smart Mobile Studio -> Libraries folder and re-start the IDE. That’s it! Now include the units in your uses-clause and kick some serious HTML5 butt!

Heres a little parser I wrote

January 22, 2015 2 comments

If you plan on parsing source-code in some form or another, you need a proper text-buffer that let’s you parse the text as simple as possible.

Here is a cool one I wrote during my lunch break. It’s written in Smart Pascal (Smart Mobile Studio) but should be easy enough to port back to Delphi or Lazarus.

Here is how you use it to dump out word-by-word from a snippet:

procedure TForm1.testNewBuffer;
var
  mbuffer:  TPascalTextBuffer;
  mText:  String;
begin
  mBuffer:=TPascalTextBuffer.Create;
  try
    mBuffer.Buffer:=#"procedure TForm1.test(first,second,third);
      begin
        // this is cool
      end;";

    mBuffer.first;

    while not mBuffer.EOF do
    begin
      mtext:=mBuffer.ReadWord;
      if mtext.length>0 then
      writeln(mText) else
      break;
    end;

  finally
    mBuffer.free;
  end;
end;

Which gives you the following output:

Much easier to handle tokens and symbol names this way

Much easier to handle tokens and symbol names this way

 

unit uniparse;

interface

uses 
  SmartCL.System;


  type

  (*  About:    Generic string buffer class
      Comments: This class allows you to traverse a text-buffer using
                next/back/first/last methods. The current character is
                always exposed in the "current" property.

                It also contains simple bookmark features, allowing you do
                recurse into the buffer for validation, and jump back
                with a single line.
  *)
  TCustomTextBuffer = Class(TObject)
  private
    FBuffer:    string;
    Fpos:       Integer;
    FBookmarks: Array of Integer;
  protected
    procedure setBuffer(Value:String);
    procedure setPosition(Value:Integer);
    function  getLength:Integer;
    function  getEmpty:Boolean;
    function  getBOF:Boolean;
    function  getEOF:Boolean;
    function  getCurrent:String;
  public
    property  Position:Integer read Fpos write setPosition;
    property  Buffer:String read FBuffer write setBuffer;
    Property  BufferLength:Integer read getLength;
    Property  Empty:Boolean read getEmpty;
    property  BOF:Boolean read getBOF;
    property  EOF:Boolean read getEOF;
    Property  Current:String read getCurrent;

    procedure Skip(Value:Array of String);

    function  PeekAhead(count:Integer):String;
    Function  CompareAhead(value:String):Boolean;

    function  Peek(count:Integer):String;
    function  Compare(Value:String):Boolean;

    procedure Bookmark;
    procedure UnBookmark;

    procedure Next;
    procedure Back;
    Procedure First;
    procedure Last;

    procedure Clear;virtual;
  end;

  (*  About:    word based buffer class
      Comments: This class extents TCustomTextBuffer with word-reading
                capabilities. This means you can traverse a text buffer
                word-by-word rather than character-by-character.

      Note:     The parser automatically skips un-readable characters like
                space, linefeed, cariage return and tab.

      Note:     "Breaker" characters will count as a single word.
                For instance, a text like this:

                "Procedure test(sender:TObject)"

                will break down into:
                  - Procedure
                  - test
                  - (
                  - sender
                  - :
                  - TObject
                  - )

                This makes it much easier to write parsers which deals with
                expected sequences and text structures. It also makes it much
                easier to deal with character-combinations, such as "(*" and
                "/*" combinations which are typically used for remarks.

      Note:     Expected sequences can be validated easily with the Ensure()
                method.
  *)
  TWordTextBuffer = Class(TCustomTextBuffer)
  public
    function  ReadTo(aBreakChars:Array of String):String;
    function  ReadWord:String;
    function  &Ensure(Sequence:Array of string):Boolean;
  end;

  TLanguageBuffer = Class(TWordTextBuffer)
  public
    function  Remark:Boolean;virtual;abstract;
    procedure SkipRemark;virtual;abstract;
  end;

  TNPPTextBuffer = Class(TWordTextBuffer)
  end;

  TPascalTextBuffer = Class(TLanguageBuffer)
  public
    function  Remark:Boolean;override;
    procedure SkipRemark;override;
  end;

implementation

//#############################################################################
//  TPascalTextBuffer
//#############################################################################

function  TPascalTextBuffer.Remark:Boolean;
Begin
  result:=((Current ='/') and (PeekAhead(1)='/'))
    or    ((current ='(') and (PeekAhead(1)='*'))
    or    ((current ='/') and (PeekAhead(1)='*'));
end;

procedure TPascalTextBuffer.SkipRemark;
Begin
  if not Empty
  and not EOF then
  begin
    if ((Current ='/') and (PeekAhead(1)='/')) then
    Begin
      self.ReadTo([#13]);
      next;
    end else

    if ((current ='(') and (PeekAhead(1)='*')) then
    Begin
      next;
      next;
      self.ReadTo(['*']);
      if PeekAhead(1)=')' then
      next;
    end else

    if ((current ='/') and (PeekAhead(1)='*')) then
    begin
      next;
      next;
      ReadTo(['*']);
      if PeekAhead(1)='/' then
      next;
    end;

  end;
end;

//#############################################################################
//  TWordTextBuffer
//#############################################################################

function TWordTextBuffer.ReadTo(aBreakChars:Array of String):String;
Begin
  setLength(result,0);
  if not Empty then
  Begin
    while not EOF do
    begin
      if not (current in aBreakChars) then
      result+=Current else
      break;
      next;
    end;
  end;
end;

function TWordTextBuffer.&Ensure(Sequence:Array of string):Boolean;
var
  x:      Integer;
  mRead:  String;
Begin
  result:=False;
  if not Empty
  and not EOF then
  begin
    if sequence.length>0 then
    begin
      for x:=sequence.low to sequence.high do
      begin
        mRead:=readWord;
        result:=sameText(mRead,Sequence[x]);
        if not result then
        break;
      end;
    end;
  end;
end;

function TWordTextBuffer.ReadWord:String;
Begin
  if not Empty
  and not EOF then
  begin
    Skip([' ',#9,#10,#13]);

    while not EOF do
    begin
      if (current in ['A'..'Z','a'..'z','0'..'9','_']) then
      result += current else
      break;
      next;
    end;

    if (result.length=0)
    and not EOF then
    Begin
      result:=Current;
      next;
    end;
  end;
end;

//#############################################################################
//  TCustomTextBuffer
//#############################################################################

Function TCustomTextBuffer.CompareAhead(value:String):Boolean;
var
  mText:  String;
begin
  if not empty
  and not EOF then
  Begin
    mText:=peekAhead(Length(Value));
    result:=Sametext(mText,Value);
  end else
  result:=False;
end;

function TCustomTextBuffer.Compare(Value:String):Boolean;
var
  mText:  String;
  mlen:   Integer;
begin
  if not empty
  and not EOF then
  Begin
    mLen:=Length(Value);
    mText:=Current + peekAhead(mLen-1);
    result:=Sametext(mText,Value);
  end else
  result:=False;
end;

function TCustomTextBuffer.Peek(count:Integer):String;
begin
  if Count>0 then
  begin
    bookmark;
    try
      setLength(result,0);
      if not EOF then
      Begin
        while length(result)<Count do
        Begin
          result:=result + current;
          if not EOF then
          next else
          break;
        end;
      end;
    finally
      unBookmark;
    end;
  end;
end;

function TCustomTextBuffer.PeekAhead(count:Integer):String;
Begin
  if Count>0 then
  begin
    bookmark;
    try
      setLength(result,0);
      next;
      if not EOF then
      Begin
        while length(result)<Count do
        Begin
          result:=result + current;
          if not EOF then
          next else
          break;
        end;
      end;
    finally
      unBookmark;
    end;
  end;
end;

procedure TCustomTextBuffer.Skip(Value:Array of String);
Begin
  if not empty
  and not EOF then
  Begin
    repeat
      if (current in Value) then
      next else
      break;
    until EOF;
  end;
end;

procedure TCustomTextBuffer.Bookmark;
Begin
  if not Empty then
  FBookmarks.add(FPos) else
  raise exception.create('Failed to add bookmark, buffer is empty error');
end;

procedure TCustomTextBuffer.UnBookmark;
Begin
  if FBookmarks.Length>0 then
  begin
    FPos:=FBookmarks[FBookmarks.high];
    FBookmarks.delete(FBookmarks.high,1);
  end else
  raise Exception.Create
  ('Failed to revert to bookmark, no bookmarks found error');
end;

function TCustomTextBuffer.getCurrent:String;
begin
  if not Empty
  and not EOF
  and not BOF then
  result:=FBuffer[FPos] else
  raise exception.Create('Read failed, invalid position');
end;

Procedure TCustomTextBuffer.First;
Begin
  if not empty then
  Fpos:=1;
end;

procedure TCustomTextBuffer.Last;
Begin
  if not empty then
  Fpos:=Length(FBuffer);
end;

procedure TCustomTextBuffer.Next;
Begin
  if not Empty
  and not EOF then
  inc(FPos);
end;

procedure TCustomTextBuffer.Back;
Begin
  if not empty
  and not BOF then
  dec(Fpos);
end;

function TCustomTextBuffer.getBOF:Boolean;
Begin
  result:=FPos<1;
end;

function TCustomTextBuffer.getEOF:Boolean;
Begin
  result:=FPos>Length(FBuffer);
end;

function TCustomTextBuffer.getEmpty:Boolean;
Begin
  result:=Length(FBuffer)<1;
end;

procedure TCustomTextBuffer.Clear;
begin
  Fbookmarks.Clear;
  FBuffer:='';
  FPos:=0;
end;

procedure TCustomTextBuffer.setBuffer(Value: String);
begin
  FBuffer:=Value;
  FBookmarks.Clear;
  FPos:=0;
end;

function TCustomTextBuffer.getLength: Integer;
begin
  result:=Length(FBuffer);
end;

procedure TCustomTextBuffer.setPosition(Value: Integer);
begin
  if (value>0) and (value<length(FBuffer)) then
  FPos:=value;
end;

end.

Inversion of control, dependency Injection, service oriented programming?

January 13, 2015 4 comments

If you google this you and trying to make sense of it is a bit like jumping head first into the rabbit hole. There are so many explanations, so many odd ways of explaining this that you could mistake it as something extremely difficult, attainable only by the elite (that super secret liege of developers that run the world behind closed doors and invent all acronyms just to confuse you).

In short, DI (dependency injection), SO (Service orientation) and IOC (inversion of control) have become hype words. Us old-timers are used to this, namely that in order to justify a complete-rewrite of a product, developers and team leaders must regularly invent buzzwords to sell an idea to shareholders. The buzzwords must make just enough sense to feel familiar, but also be odd enough to be shrouded in mystery. Boys will be boys, and hype is a part of the game.

The terms Dependency Injection (DI) and Inversion of Control (IoC) are generally used interchangeably to describe the same design stuff. Hence some people says IoC Container and some people says DI container but both terms indicate the same thing.

I tend to look at IOC as the technique for writing your code, and dependency injection as the method for managing your implementation.

Just jump in with both feet, it’s fairly simple stuff once you get rid of the hype.

Jumping in

Right, dependency Injection, what is it? First of all, all these topics are techniques. There is no software to buy and no language is better or worse. So if you think this is just for the .net boys or Java, and that in order to deliver this you must somehow throw away your 20 year investment in Delphi – think again.

In fact, one of the finest and easiest explanations to DI in my view was penned by our very own Nick Hodges in his latest book “Coding in Delphi“. I’m not going to steal or re-publish his work here, but simply break it down to “what it is”. Get Nick’s book – it’s a good read and simplifies a lot of topics, way beyond IOC/DI.

A dependency in DI has nothing to do with dependencies in the common term, like a DLL or some library dependency (although it can be that, but not without getting inversion of control first). A dependency in DI terms is rather defined as “what does your class need in order to function?”. Most object hierarchies are hopelessly bound to its members, like TCollectionItem is to TCollection in Delphi. TCollectionItem depends on TCollection in order to compile, and TCollection is useless if you delete TCollectionItem from the unit.

If you analyze a large codebase with these glasses on, you will suddenly realize that your code is full of such invisible “dependencies”. Some have to be there of course, otherwise your application will become to abstract to work, but a lot of the code functionality of an application can be better made by de-coupling class A from class B.

So how do you do this? Well, this is where interfaces come in. And incidentally covers the basic idea behind “inversion of control”.

If all you had to do was to support ICollectionItem and your object could be used by TCollection out of the box – that would be a lot more elegant and maintainable than the present hard-coded model. Also, when interfaces controls access – the implementation can be shielded. Meaning that for the consumer code, meaning the code you write that accesses TCollection in my example here, doesn’t even have to reside on the same machine. As long as you provide an object which supports the correct interface, the meat of the code can be isolated in an RPC (remote procedure call) service or exported to a DLL file or whatever tickles your fancy.

IOC/DI example

IOC/DI example

That is an extremely simple explanation of inversion of control. To get the full low-down I urge you to get a good book on the subject, like the one I already mentioned 🙂

So if you think “inversion of control” has anything to do with classical dependencies, you can put that idea to rest. In many ways IOC/DI is the same as a plugin system — but for your whole application. And once you have architected your applications according to these principles – maintenance, isolation of code in your program, a DLL’s or as services is how you deal with your code in general.

DI containers

Since we now know that Inversion of control is just a fancy way of abstracting your code, and that “dependency injection” has to do with limiting class and entity dependency on each other (or co-dependency), what is the most efficient way to handling variations of code?

Imagine you have a picture program which should read files from various sources. It should read from the file-system, it should read from FTP, it should read images from ZIP files and perhaps even directly from a website.

How would you solve this?

The classical approach is to create a custom-class, a base-class which you then derive local, ftp, zip and http “drivers” from. You would then have a second class, a controller, which would create the correct instance for you.

Injection isolation

Injection isolation

This means that the controller depends on the baseclass (or better explained: the baseclass is known to the controller) and visa-versa. Since we are talking about file-access over various protocols here, it’s unlikely that the various classes doesn’t know about the controller either – so voila, we have the same old co-dependency occurring. The whole infrastructure is hopelessly dependent on everything in it, you can’t de-couple anything or it breaks apart. You have a rigid, static structure. And the more complex the structure, the more dependencies occur and the harder it is to re-model the architecture.

A DI container (this is where the “injection” part comes in) is a special object into which you register interfaces and the classes that provide them. This means that all the filesystem classes doesn’t need to know about the controller, and the controller doesn’t need to know about the various file systems. Both sides of the coin only needs to know one thing: how to access the container to get the interfaces they need – and the interface declaration. The whole event-sink topology we are used to can be distributed elsewhere – and you can even drop handling events all together if you so chose.

This is incidentally why it’s called inversion of control, because your main program will also export interfaces for whatever child classes you have made, meaning that your child classes will now call into your main program – rather than your main program just creating instances and calling that. So suddenly your program becomes a mesh of non-dependent, self oriented services, breaking somewhat with the linear parent/child model of classical OOP programs.

Service oriented programming

When you reduce dependencies and rely solely on interfaces, as is the point of IOC/DI, your programming changes. Whenever you need to do something you get the interface, use it, then forget it. You begin to look at your program more as an environment of services more than a traditional cluster of units, classes and lose procedures.

Each interface (read: feature set) is more as a service which provides X number of features. Where these features are implemented, be it inside your program, a win32 service running in the background or some REST or RPC service half-way across the globe is insignificant.

Service oriented programming is more or less just that. You write your code as a series of services, accessing them in the same way as you would Amazon or Azure. It is of no concern if the features you use (or need) resides in your process, some other process or remotely. You focus on the features you deliver and utterly de-couple it from location.

When writing code meant to run on high-end servers you naturally want to spread your code over a series of services. Your database work would be isolated as a single service, exporting X number of interfaces for reading, writing, creating and updating data. A second service deals with staging data (preparing data for the database). Perhaps you have a zipcode service which deals exclusively with that as well — and so on.

That in effect is what SOP is about. It’s a lot more complex of course and you should get a book about it, but in essence it deals with software architecture.

CodeTyphoon, GPL and the whole money thing

January 4, 2015 4 comments

After my initial post regarding “Free Sparta” and “CodeTyphoon” by PilotLogic, both separate products which are in violation of the GPL license and (not withstanding) morally bankrupt I received a lot of comments. Both here and on the Delphi Developer group where the post was issued first.

the majority of the human population are socially programmed to think exclusively in terms of gain and loss, checks and bills, making a quick buck and taking advantage of a situation.

First, turns out that the Free Sparta project died out a few weeks back, so we can at least scratch that off the list. apparently the package introduced no changes to the original codebase, something I find very hard to believe since the one of its core selling points was its ease of use and better IDE. Either way, it’s out of circulation so no point wasting time on that.

The second product, CodeTyphoon, is in direct violation of the GPL. They do not provide source-access to their product or alterations to the original code, which is the fundamental point of GPL. Namely to allow others to evolve a piece of software, improving and building on the initial system.

Since there seem to a lot of confusion, let’s go through a few facts

Selling GPL software

Stallman

Stallman

First off, yes – you are allowed to sell GPL software. This was initially included to cover the cost of distribution back in the days of floppy-disks and modems; back when a 50 megabyte hard disk would cost you an arm and a leg. Be that as it may – yes, you are allowed to sell GPL based software —- but once again, a part of that states that you must also provide source-code access to the sold GPL based software. So once again, PilotLogic is off the mark.

Secondly, regarding the “open source” model. A lot of people seem to be under the impression that OSS is money agnostic or “open for personal interpretation”. But that could not be further from the truth. OSS as a concept is very much against profit based on software. The entire point of OSS is to ensure that no corporation or financial entity is able to control the market through closed source. Closed source is regarded essentially as robbing the individual of the right to choose, hindering scientific research through patents and establishing information based monopolies. Source: Revolution OS, documentary on Linux

But OSS is not against making money, but rather it pushes potential for income into services around the product. A prime example is Open-Office which itself is free. It’s used by probably hundreds of thousands of schools around the world and is responsible for breaking Microsoft’s monopoly in the office space.

So where does the money come from? From tutoring, from presentations and from sponsorship. What typically happens is that a company hires you to do custom adaptations of a product. They pay the bills and finances your work, and in return they get to promote the work (and use the tools).

Besides from that you are expected to sell your expertise, using the product to create or achieve something. And you can also scrape in some cash for distribution, both binaries and source-code.

This way of working may seem alien to most programmers out there, but it’s actually how researchers work around the world. You live off grants, sponsorships and custom work for various institutions. When operating with open-source software, especially under Apache or GPL, you have to remember that these both originate at universities and scientific establishments. They dont originate from a software house or “club”, but are designed to cater for scientists and researchers first of all.

Back to the future

Richard Stallman, the founder of the GNU foundation and the author of the GPL license – and pretty much the grandfather of the open-source movement, holds a completely different philosophy than most people. And this has to be underlined because the majority of the human population are socially programmed to think exclusively in terms of gain and loss, checks and bills, making a quick buck and taking advantage of a situation. This is the thicket in which the goat of reason easily get’s stuck.

Stallman said his primary goal is to live off the prize money so that he can devote his time to continue his not-for-profit work leading the Free Software Foundation, developing the GNU operating system, and campaigning for social change in the way software is written and distributed. Source: Stallman to Recieve Price

The entire point of the GNU operative system (what people today call Linux, which is actually just the kernel. The operative system, libraries, desktop and software is actually called GNU) and it’s license was to ensure that a free alternative existed. An operative system that went completely against everything Microsoft, Apple, IBM and all the other big-wigs from way-back times stood for: namely economic power based on digital secrecy.

The reason a license exists in the first place, is to define a moral framework in which to work, which ensures that people use and behave according to the intention of the original authors

Stallman comes from a very different place than most people; His roots go back to the late hippie movement of the 60’s and 70’s. You could say that he is the drug-free, non flower power, scientific version of a hippie. He believes in total freedom within a moral framework (as opposed to limited freedom within a larger framework, which is the ideology the western world presently lives under) which has resulted in one of the biggest software archives in the world. All of it written to avoid and bypass the whole corporate culture which Stallman and his posse hates.

Stallman with his new Software Gospel

Stallman preaching his gospel of free software

Stallman is an ideologist which has more than often spoken against concepts like patents, because they cripple technological achievements (and I have to agree on that point) when allowed to be held for more than the initial 6 years (which was the original time-frame you could hold a patent back in the late 1800’s). So the entire GPL culture is all about free research, free evolution of software and free access to software.

What people have to understand is that this line of thinking does not mean you can do whatever you want. That’s the difference between the hippie-movement and anarchists. You are still expected to show ordinary social respect for other people’s work, to ask and to behave as an intelligent, responsible human being. The reason a license exists in the first place, is to define a moral framework in which to work, which ensures that people use and behave according to the intention of the original authors. This has to do with respect.

Freedom under responsibility

Freedom without boundaries is not freedom, that would be chaos. Without boundaries it’s impossible to set goals and perform co-operation. As such Stallman and his crew made damn sure that no-one would be able to do exactly what PilotLogic is doing. Namely to grab an open-source project, evolve it and then keep the code behind closed doors.

That is, when you think about it, the very definition of greed. It is equal to being poor, getting a truck with food to give out to others — only to keep the whole thing for yourself. You were given access to the code under the premise that you, just like the original authors, demonstrate solidarity and transparency.

It doesn’t really get much clearer than this.

Easy to point the finger

So there!

So there!

I am not perfect, and surely I have grabbed some piece of code I found online over the years, fixed it up and stuffed it in a library. But I have always tried to add the name of the original author or source to the unit. I mean, sending an e-mail to someone asking if you can re-name a unit and classes — it wont really cost you anything.

I just want to underline that im not doing a witch hunt here. I am not “perfect” in any way, and I have for a long time given PilotLogic the benefit of the doubt. Maybe they just forgot about it? Maybe they did not have the resources? But I think it’s become quite evident that PilotLogic simply don’t care.

They post updates and keep evolving their product, and last time I checked a sourceforge or GIT account is free. So there really is no excuse that can validate ignoring emails from users, ignoring complaints by programmers who suddenly find their own work available inside CodeTyphoon.

Summing up

Since PilotLogic is in direct violation, it should not be necessary to preach morality. But I know that I would personally be quite offended if I suddenly discovered my code in some RTL out there, completely stripped of my name or any form or info regarding origin. This is in fact the reason I publish code under “COPYRIGHT”, even on this blog.

People are allowed to read, study and use my code – but if you want to use it in a commercial product, then just ask me. I’m a fairly mellow individual and not a hard-ass at all, so if you need a unit, just send me an email.

Most programmers are fairly laid back, but unless otherwise stated, it’s good to send an email before you grab a huge piece of code. I do that myself.

That’s not too much to ask for.

But when it comes to open source, the rules are clear. I have two open-source projects myself, with one Lazarus fork of ByteRage out in the wild. And as expected the converter contacted me via Facebook asking if it was ok. Of course it’s ok.

And he also had the ordinary decency to leave the header alone, adding a secondary header on top of the unit with his information. No problem at all, use and have fun.

See how easy that was?

Free sparta and CodeTyphoon violates GPL

January 3, 2015 21 comments

Look around the net for alternatives to Delphi is like looking for water in the desert. To date there are really just 3 real options: Smart Mobile Studio, Freepascal & Lazarus– or Oxygene Pascal from Remobjects.

But FPC and Lazarus are open-source products, meaning that anyone can freely fork and create their own versions within certain limits. There is however one significant rule to this: you must publish whatever changes you have made, or at least provide a working download link so that people have free access to your variation of the root product. That is the deal you signed when you forked the source code and you have an obligation by law to honor that agreement.

Violating individual rights is not a good idea, you will get a response

Violating individual rights is not a good idea, you will get a response

What really pisses me off is that the two forks of Lazarus out there, being sold as commercial solutions, completely ignores the license. Recently I came across a fork of Lazarus called “free sparta” – which is, as it stands now, is a complete insult to the people who have created FPC and Lazarus, dedicating years of hard work to build and maintain both product and community.

First of all the product is commercial (GPL to commercial? Wow! That’s a new twist), meaning that you are basically asked to pay for a product which is bound by international law to be free. Secondly, and this just infuriates me so much, they do not provide any links or SVN/GIT access to the sources. Which they are bound to do by law since the original codebase is GPL.

A second product, called CodeTyphoon takes the whole thing even further; besides being “selective” about the GPL – they demonstrate no scruples re-naming components and units, removing author information and origin of code just to suit their own needs.

Clean up your act

If you are going to compete with commercial products then at least have the balls to follow the rules. Stop stealing code which is illegal, and stop re-branding other people’s work as your own to make a profit (which is not only illegal and a violation of the license but also immoral and intellectually bankrupt).

And what are the rules? You should think they were oh-so complex, but in both these cases they can be summed up as:

  • Ask the original authors about re-branding their units
  • Do not remove author notes and/or comments from the code
  • If they decline your question, show some bloody respect and either exclude it from your product, or use the names provided by the authors.
  • Stop violating the GPL license because that will have consequences
  • Stop trying to make money on other people’s work
  • Provide public read access to either SVN or GIT where people can fork your changes, that is how you got the code in the first place – denying others the same access tells volumes about what type of person you are

Ignoring these simple steps is not just a violation of the GPL license, it’s also a sure way to fuck-up the resolve and determination of the original authors. Why the hell should the FPC and Lazarus community keep working for free while you earn money on their work? Bet your mother must be real proud of you.

But yes, doing what you do is against the law and it is a violation of the GPL license. There is no middle ground here, you either follow the rules or you fucking leave it alone.

As you probably know the GPL license is connected to some of the largest companies in the world – back up by a bloody army of lawyers. As such, anyone violating the GPL can quickly find themselves in a shit-load of trouble, because the GPL is protected by the same organization which protects Linux. This means that if anyone reports your sorry-ass to the GNU foundation – you will be held financially accountable. No matter where you live.

Dont think it wont happen, I’ll happily make the phone-call myself if I have to. And believe you me, the GNU foundation have made examples out of smaller fish that you.

Stealing code and re-branding it is something that makes me furious. Sharing code, learning from each other is one things – but trying to gain financially on what other people have made? Thats just fucking rotten.

If you want to play soccer, then you play by the rules or you don’t play at all. It’s not a debate or open for your personal interpretations of the law. You publish the code ASAP or you will be reported to the GNU foundation.

How to make money on GPL

The open-source model was initially created “TO NOT MAKE MONEY”. So just wrap your mind around that straight away. No middle ground, no “but I could” — no. Forget it. It doesnt work that way. It’s founded on a 100% no-money philosophy.

Open Source pushes the potential of income away from the product, and places the potential in the use of the product and services around the product. In other words you dont have to publish your DOC’s as free, they can be sold separately. Also, you dont need to provide an installer – that can also be sold separately.

But the product itself, no matter if you spend 10 years evolving the code — if it’s GPL it’s free. And if you persist in forking FPC and Lazarus you must provide a working, open for all, SVN, GIT or otherwise functioning download URL.

End of debate.

I rarely get involved in stuff like this, but this time — failure to follow this and respect the original authors, will result in a formal report to the GNU foundation.

Boycott these products

Support FPC, say no to thieves!

Support FPC, say no to thieves!

I urge everyone to boycot these illegal products. Several individuals have already sent protest emails to pilotLogic, the company behind CodeTyphoon, as well as the “free sparta” cash-in project; They refuse to respond (as expected).

It is an insult to the programmers behind FPC and Lazarus and to show your support of FPC and Lazarus, I hope you agree and stay away or stop using these products.

You may also want to send an email requesting SVN/GIT access to both — reminding them that they are obliged by international law to provide this, regardless of their own personal “interpretation” of the law.