Smart Raspberry PI project

Raspberry PI is a great little $35 mini computer the size of a credit card. The latest version (2.0) comes with 1 gigabyte of ram and a mid-range powerful ARM processor. Raspberry PI is used by hobbyists, schools and IOT companies to create clever consumer gadgets. The sky is the limit and what you can do with your PI is defined by imagination only (and I admit, some knowledge of Linux).

What our custom Linux distro brings, is a ready-to-use kiosk system

Smart Mobile Studio full control Raspberry PI

My good friend Glenn, a citizen of Denmark, loves all things Linux – so when I asked him if we could hijack the boot sequence, drop the desktop and boot straight into full-screen Chrome instead.. well, it didn’t take long for him to figure out a way to do that. I have to little experience with the mysteries of Linux to make that happen (it would take weeks of fiddling), so it’s good to have friends who know’s what they are doing.

A fully working desktop environment, took Glenn and myself 15 minutes to make

In essence we are creating a custom Linux distro which is designed to run your Smart Mobile Studio projects exclusively. This means:

• full access to the filesystem
• no content domain restrictions (CORS)
• no blocking-operation restraints
• No restriction on database or storage sizes
• Your Smart program completely runs the show

As a gateway to the operative system and interesting functionality, I have gone for a nodeJS server running in the background (booted before the browser display) which means the desktop (or your project) can use RPC calls for advanced functions.

This is pretty cool because it means that the UI will be completely abstracted from the service functionality. The service can be local and run on the same installation – or it can run on Azure or Amazon cloud servers on the other side of the world.

Think of it this way: You dont have to run it on the same device. You can upload your desktop (or kiosk) to a normal web-host, disable CORS on the host, and then use websockets and connect to the NodeJS layer; which can be hosted on another computer or domain.

But all of that is already possible today. This is one of the simplest things to make with Smart Mobile Studio actually. A much cooler project is what we are doing now with the Raspberry PI — giving your creations the ability to live on and control a full Linux system.

So what are you guys up to?

The first project is a NetFlix like media system (or a XBMC clone) with full support for USB wireless remote controls. To make it short: a $35 home theatre that plugs into your television, which will scan your movies folder (or external drive) and present your movies like NetFlix does it. It will download info from MovieDB.com and other media services for identification. It sounds complicated but it’s actually very straight forward. One of the simplest solutions I’ve done with Smart Mobile Studio to be honest.

Using a standard USB remote control is excellent because it registers as a touch-device. So the buttons you press registers as touch-events in a predictable order (read: it works on all controls of this type).

What our custom Linux distro brings, is a ready-to-use kiosk system. You can use it to display advertizing in a shop window if you like, or add a touch screen and build your own ticket ordering station. Again, it’s only limited by your own imagination.

A JavaScript cloud desktop

Depending on the success of the project we may go full-out and create the world’s first JavaScript desktop. It is essentially Debian + Chrome + Smart Mobile Studio. This would require a bit more NodeJS magic, where each exposed node service (RPC) represents a distinct part of the “operative system”. A virtual operative system running on top of a Linux stub. Pretty darn cool if you ask me. Who knows, maybe we can define POSIX for the cloud?

A fun hobby to say the least 🙂

Copas, Delphi script for Shell

Have you ever wished that there was some way to script the shell without resorting to archaic batch files, python or any of the other weird languages out there? Well, a while back I started to implement a special version of DWScript (Delphi web script) which does exactly that. It allows you to execute DWScript files directly from the shell powered by a rich set of OS level functionality.

I havent come up with a cool name for it yet so i dubbed it copas, short for “command-line object pascal”. Since DWScript is the parser engine we use with Smart Mobile Studio, the dialect is identical – and you can even use some of the Smart Pascal classes with it (no web-engine stuff naturally).

It’s a start

If someone updates the DWScript repository for FreePascal to the latest, we can kiss python, node.js and all the other odd-ball scripting engines for shell/bash goodbye!

Why is this cool?

Being able to execute pascal scripts is not the hard part (since that is already covered by DWScript), the hard part is providing classes that allows you to interact with the operative system. Things like file creation, pipes, executing applications, service rights — these are not complex but rather time-consuming tasks.

Since time is not something I have these days, it will take some time before I release any source-code for this, but perhaps over Xmas I’ll have time to finish it. I started on this months ago but havent had time to do much work on it. But streams and a pipe client (for windows) is already in place. The plan is to compile with FireDAC (or morpheus) so you can connect directly to all major DB engines, create databases on the fly, copy data between engines, perform backup tasks with ease — well, the possibilities are endless and only limited by time and imagination.

But the benefits should be obvious:

• creating scripts that deals with files and folders can now be done in a language you know and love
• Full OOP support, including partial classes, interfaces and all the cool stuff Smart Pascal is currently offering
• Script servers (REST, WebSocket etc)
• Add smart mobile studio and you have a fully object pascal driven infrastructure.
• Perfect for automation tasks

Well, no time to complete this now – but after Xmas I’ll share out the SVN repo.

In REST we trust

Delphi is a bit odd these days. For instance, Embarcadero ships some fairly good REST client components. Bloated beyond belief, but fairly good. Yet where exactly is the REST server? I could hardly believe my ears when I was told that DataSnap was the only option supported by Embarcadero. It should be considered a crime against humanity to push possibly the most sluggish DB solution in existence as the basis for REST based networking protocols.

It completely undermines the purpose of REST, namely to mine the power of fast HTTP servers to create quick, small and responsive services. REST is essentially a technology which piggyback’s on the established HTTP standard. So the explanation Embarcadero comes up with to justify mixing this with Datasnap should be absolutely spectacular.

It’s like “fixed” order in XML, it utterly undermines the point of a flexible, text based format.

Sweet Jesus no! Arrrg… Datasnap!

Well, guess I better build a REST server myself then..

Indy still rocks

The basis for my server is good old Indy. Some might say that you get 0.2 microseconds better speed using Synapse or whatever other library out there, but for me Indy is king when it comes to server solutions. Indy is rock solid, stable as a mountain, not to mention tried, tested and developed over 10+ years, it has been used successfully in thousands of products, its platform independent and the architecture is one of the best you’ll ever see; regardless of programming language or platform.

Adapting an Indy HTTP server (TIdCustomHTTPServer) so it supports the REST paradigm is really not that hard. In essence you override 3 protected methods and with a few lines of code you can support both REST and normal HTTP behavior.

  TIdRestServer = Class(TIdCustomHTTPServer,
                        IIdRestServer,
                        IIdRestElementSchema)
strict protected
    procedure DoCommandError(AContext: TIdContext; ARequestInfo: TIdHTTPRequestInfo;
              AResponseInfo: TIdHTTPResponseInfo; AException: Exception); override;

    procedure DoCommandGet(AContext: TIdContext; ARequestInfo: TIdHTTPRequestInfo;
              AResponseInfo: TIdHTTPResponseInfo); override;

    procedure DoCommandOther(AContext: TIdContext; ARequestInfo: TIdHTTPRequestInfo;
              AResponseInfo: TIdHTTPResponseInfo); override;

    procedure DoRestCommand(Uri: TIdURI; AContext: TIdContext;
              ARequestInfo: TIdHTTPRequestInfo; AResponseInfo: TIdHTTPResponseInfo);
end;

REST Services

In my system I have decided to “emulate” what I normally use for tasks like this, namely Remobjects SDK. Remobjects SDK makes it a snap to create RPC (remote procedure call) servers. It even ships with a service designer that helps you define X number of services, which in turn has X number of invokable methods. Each method has X number of parameters and once defined they behave just like local procedures and functions.

The power of products like Remobjects SDK is that they are (like I mentioned above) RPC systems. This means that you can isolate classes and tasks on a server half way around the world, and call them in your programs just like they were linked and compiled as usual.

Logical and straight forward namespacing

When you call a RPC service there is quite a lot taking place: Your parameters are serialized, a connection is made to the server, data is transferred; on the other end the data is de-serialized into an object, validated against a schema – and finally the implementation of your code fires. The result of the operation is again serialized and sent back to your program.

With REST as the basis, accessing services becomes even easier than with Remobjects SDK (or at least en-par). Why? Because the service-with-method architecture lends itself easily to REST methodology. The URL scheme is so simple and easy to use – and comes with a pleasant benefit; namely using your browser to access schemas! Heck, you can even call functions directly from the browser (if you omit any form of security and authentication that is).

Schema generation

The server generates JSON based schemas for the services it exposes

Making heads or tails of an alien REST service can be hard. Trust me, there are some seriously disturbed services out there. And if you think there is a fixed REST standard that everyone follows, think again.

Errors for instance is a hot topic. Should you re-label some HTTP errors and then respond to that on the client? Well, some do that and others don’t. I firmly believe your REST based server should not interfere with the ordinary HTTP behavior, but rather kick in only if the referenced URI points to the /REST/ namespace. And since the API will be used by a client who reads and understands JSON, there is no point in throwing an error back at all.

Now viewing schemas is easy: In essence, when you visit a valid REST URL on the server – but you don’t provide any parameters, the server generates the schema for that element. There is no such thing as a REST method without a parameter (POST attachment also counts as a parameter).

So if you are wondering about, say, the syntax for a particular method, you can visit the method URL directly with your browser and view the schema, This works on all levels (root, service and method) so you can get the full API for the whole server by visiting the root /rest/ URL.

Writing server-side methods

Having to dabble in tons of events or (perhaps even worse) nested TCollection items is something we want to avoid. First of all because synchronization with the main VCL thread is easily abused (with catastrophic performance results), and secondly we want to de-couple as much as we can to make the best of a multi-threaded environment. So for our API I decided to make full use of anonymous methods and weak references. It’s now so much easier to write services, even easier than the Remobjects SDK service designer even.

In the example below we first create the server, then we create a service – and implement a single method for it (GetServerTime). The service registers with the server on creation (as does the method with the service) so we don’t have to think about that. Also, on invocation the server automatically validates all parameters with the auto-generated schema, so most of the tedious work is taken care of.

Implementing a REST server should be fast, fun and easy!

  FServer := TIdRestServer.Create(self);
  FServer.Defaultport := 8090;

  FService := TidRESTService.Create(FServer);
  FService.ServiceName.Value:='sharebike';

  FMethod := TidRESTMethodGET.Create(FService);
  FMethod.RestMethodName.Value := 'getservertime';
  FMethod.Parameters.Add('id',TIdRestSchemaParameterType.rpText,'');
  FMethod.HandleWith( procedure (const Info:TidRestMethodCallData)
    var
      mData:  String;
      mError: TIdRestErrorMessage;
    begin
      // Make sure we can read the ID parameter
      if Info.Params.TryGetValue('id',mData) then
      begin
        with info.SocketInfo do
        begin
          Response.ContentText :='{ "result": ' + DQuoteStr(mData) + ' }';
          Response.WriteHeader;
          Response.WriteContent;
        end;
      end else
      begin
        mError := TIdRestErrorMessage.Create('syntax-error','Parameter ID could not be read error');
        try
          with info.SocketInfo do
          response.contentText:=mError.Serialize('Tag-value-here');
        finally
          mError.free;
        end;
      end;
    end);

That’s essentially all it takes to implement a fully multi-threaded REST server.

Internal project

If you are expecting source-code for this, then sadly I have to decline. This is an in-house project and naturally belongs to my employer. But writing your own REST server is not hard if you know you way around Delphi and Indy.

But my final verdict is that I’m getting sold on REST. I have been very WebSocket biased, but having spent a couple of weeks working on REST I see that it indeed is very useful and powerful. And our system in place we at least get to enjoy writing services 🙂

Pretty cool to work with

As a bonus, Smart Mobile Studio now supports REST calls, so make sure you check back soon for a demonstration!