Archive

Archive for the ‘Language research’ Category

Amibian.js under the hood

December 5, 2018 2 comments

Amibian.js is gaining momentum as more and more developers, embedded systems architects, gamers and retro computer enthusiasts discover the project. And I have to admit I’m pretty stoked about what we are building here myself!

intro

In a life-preserver no less ūüėÄ

But, with any new technology or invention there are two common traps that people can fall into: The first trap is to gravely underestimate a technology. JavaScript certainly invites this, because only a decade ago the language was little more than a toy. Since then JavaScript have evolved to become the most widely adopted programming language in the world, and runtime engines like Google’s V8 runs JavaScript almost as fast as compiled binary code (“native” means machine code, like that produced by a C/C++ compiler, Pascal compiler or anything else that produces programs that run under Linux or Windows).

It takes some adjustments, especially for traditional programmers that havent paid attention to where browsers have gone – but long gone are the days of interpreted JavaScript. Modern JavaScript is first parsed, tokenized and compiled to bytecodes. These bytecodes are then JIT compiled (“just in time”, which means the compilation takes place inside the browser) to real machine-code using state of the art techniques (LLVM). So the JavaScript of 2018 is by no means the JavaScript of 2008.

The second trap you can fall into – is to exaggerate what a new technology can do, and attach abilities and expectations to a product that simply cannot be delivered.¬†It is very important to me that people don’t fall into either trap, and that everyone is informed about what Amibian.js actually is and can deliver – but also what it wont deliver. Rome was not built-in a day, and it’s wise to study all the factors before passing judgement.

I have been truly fortunate that people support the project financially via Patreon, and as such I feel it’s my duty to document and explain as much as possible. I am a programmer and I often forget that not everyone understands what I’m talking about. We are all human and make mistakes.

Hopefully this post will paint a clearer picture of Amibian.js and what we are building here. The project is divided into two phases: first to finish Amibian.js itself, and secondly to write a Visual Studio clone that runs purely in the browser. Since it’s easy to mix these things up, I’m underlining this easy – just in case.

What the heck is Amibian.js?

Amibian.js is a group of services and libraries that combined creates a portable operating-system that renders to HTML5. A system that was written using readily available web technology, and designed to deliver advanced desktop functionality to web applications.

The services that make up Amibian.js was designed to piggyback on a thin Linux crust, where Linux deals with the hardware, drivers and the nitty-gritty we take for granted. There is no point trying to write a better kernel in 2018, because you are never going to catch up with Linus Torvalds. It’s must more interesting to push modern web technology to the absolute limits, and build a system that is truly portable and distributed.

smart_ass

Above: Amibian.js is created in Smart Pascal and compiled to JavaScript

The service layer is written purely in node.js (JavaScript) which guarantees the same behavior regardless of host platform. One of the benefits of using off-the-shelves web technology is that you can physically copy the whole system from one machine to the other without any changes. So if you have a running Amibian.js system on your x86 PC, and copy all the files to an ARM computer – you dont even have to recompile the system. Just fire up the services and you are back in the game.

Now before you dismiss this as “yet another web mockup” please remember what I said about JavaScript: the JavaScript in 2018 is not the JavaScript of 2008. No other language on the planet has seen as much development as JavaScript, and it has evolved from a “browser toy” – into the most important programming language of our time.

So Amibian.js is not some skin-deep mockup of a desktop (lord knows there are plenty of those online). It implements advanced technologies such as remote filesystem mapping, an object-oriented message protocol (Ragnarok), RPCS (remote procedure call invocation stack), video codec capabilities and much more — all of it done with JavaScript.

In fact, one of the demos that Amibian.js ships with is Quake III recompiled to JavaScript. It delivers 120 fps flawlessly (browser is limited to 60 fps) and makes full use of standard browser technologies (WebGL).

utube

Click on picture above to watch Amibian.js in action on YouTube

So indeed, the JavaScript we are talking about here is cutting edge. Most of Amibian.js is compiled as “Asm.js” which means that the V8 runtime (the code that runs JavaScript inside the browser, or as a program under node.js) will JIT compile it to highly efficient machine-code.

Which is why Amibian.js is able to do things that people imagine impossible!

Ok, but what does Amibian.js consist of?

Amibian.js consists of many parts, but we can divide it into two categories:

  • A HTML5 desktop client
  • A system server and various child processes

These two categories have the exact same relationship as the X desktop and the Linux kernel. The client connects to the server, invokes procedures to do some work, and then visually represent the response This is identical to how the X desktop calls functions in the kernel or one of the Linux libraries. The difference between the traditional, machine code based OS and our web variation, is that our version doesn’t have to care about the hardware. We can also assign many different roles to Ambian.js (more about that later).

smartdesk

Enjoying other cloud applications is easy with Amibian.js, here is Plex, a system very much based on the same ideas as Amibian.js

And for the record: I’m¬†trying to avoid a bare-metal OS, otherwise I would have written the system using a native programming language like C or Object-Pascal. So I am not using JavaScript because I lack skill in native languages, I am using JavaScript because native code is not relevant for the tasks Amibian.js solves. If I used a native back-end I could have finished this in a couple of months, but a native server would be unable to replicate itself between cloud instances because chipset and CPU would be determining factors.

The Amibian.js server is not a single program. The back-end for Amibian.js consists of several service applications (daemons on Linux) that each deliver specific features. The combined functionality of these services make up “the amibian kernel” in our analogy with Linux. You can think of these services as the library files in a traditional system, and programs that are written for Amibian.js can call on these to a wide range of tasks. It can be as simple as reading a file, or as complex as registering a new user or requesting admin rights.

The greatest strength of Amibian.js is that it’s designed to run clustered, using as many CPU cores as possible. It’s also designed to scale, meaning that it will replicate itself and divide the work between different instances. This is where things get’s interesting, because an Amibian.js cluster doesn’t need the latest and coolest hardware to deliver good performance. You can build a cluster of old PC’s in your office, or a handful of embedded boards (ODroid XU4, Raspberry PI’s and Tinkerboard are brilliant candidates).

But why Amibian.js? Why not just stick with Linux?

That is a fair question, and this is where the roles I mentioned above comes in.

As a software developer many of my customers work with embedded devices and kiosk systems. You have companies that produce routers and set-top boxes, NAS boxes of various complexity, ticket systems for trains and busses; and all of them end up having to solve the same needs.

What each of these manufacturers have in common, is the need for a web desktop system that can be adapted for a specific program. Any idiot can write a web application, but when you need safe access to the filesystem, unified API’s that can delegate signals to Amazon, Azure or your company server, things suddenly get’s more complicated. And even when you have all of that, you still need a rock solid application model suitable for distributed computing. You might have 1 ticket booth, or 10.000 nation wide. There are no systems available that is designed to deal with web-technology on that scale. Yet ūüėČ

Let’s look at a couple of real-life scenarios that I have encountered, I’m confident you will recognize a common need. So here are some roles that Amibian.js can assume and help deliver a solution rapidly. It also gives you some ideas of the economic possibilities.

Updated: Please note that we are talking javascript here, not native code. There are a lot of native solutions out there, but the whole point here is to forget about CPU, chipset and target and have a system floating on top of whatever is beneath.

  • When you want to change some settings on your router – you login to your router. It contains a small apache server (or something similar) and you do all your maintenance via that web interface. This web interface is typically skin-deep, annoying to work with and a pain for developers to update since it’s connected to a native apache module which is 100% dependent on the firmware. Each vendor end up re-inventing the wheel over and over again.
  • When you visit a large museum notice the displays. A museum needs to display multimedia, preferably on touch capable devices, throughout the different exhibits. The cost of having a developer create native applications that displays the media, plays the movies and gives visual feedback is astronomical. Which is why most museums adopt web technology to handle media presentation and interaction. Again they re-invent the wheel with varying degree of success.
  • Hotels have more or less the exact same need but on a smaller scale, especially the larger hotels where the lobby have information booths, and each room displays a web interface via the TV.
  • Shopping malls face the same challenge, and depending on the size they can need anything from a single to a hundred nodes.
  • Schools and education spend millions on training software and programming languages every year. Amibian.js can deliver both and the schools would only pay for maintenance and adaptation – the product itself is free. Kids get the benefit of learning traditional languages and enjoying instant visual feedback! They can learn Basic, Pascal, JavaScript and C. I firmly believe that the classical languages will help make them better programmers as they evolve.

You are probably starting to see the common denominator here?

They all need a web-based desktop system, one that can run complex HTML5 based media applications and give them the same depth as a native operating-system; Which is pretty hard to achieve with JavaScript alone.

Amibian.js provides a rich foundation of more than 4000 classes that developers can use to write large, complex and media rich applications (see Smart Mobile Studio below). Just like Linux and Windows provides a wealth of libraries and features for native application development – Amibian.js aims to provide the same for cloud and embedded systems.

And as the name implies, it has roots in the past with the machine that defined multimedia, namely the Commodore Amiga. So the relation is more than just visually, Amibian.js uses the same system architecture – because we believe it’s one of the best systems ever designed.

If JavaScript is so poor, why should we trust you to deliver so much?

First of all I’m not selling anything. It’s not like this project is something that is going to make me a ton of cash. I ask for support during the development period because I want to allocate proper time for it, but when done Amibian.js will be free for everyone (LGPL). And I’m also writing it because it’s something that I need and that I havent seen anywhere else. I think you have to write software for yourself, otherwise the quality wont be there.

Secondly, writing Amibian.js in raw JavaScript with the same amount of functions and depth would take years. The reason I am able to deliver so much functionality quickly, is because I use a compiler system called Smart Mobile Studio. This saves months and years of development time, and I can use all the benefits of OOP.

Prior to starting the Amibian.js project, I spent roughly 9 years creating Smart Mobile Studio. Smart is not a solo project, many individuals have been involved – and the product provides a compiler, IDE (editor and tools), and a vast run-time library of pre-made classes (roughly 4000 ready to use classes, or building-blocks).

amibian_shell

Writing large-scale node.js services in Smart is easy, fun and powerful!

Unlike other development systems, Smart Mobile Studio compiles to JavaScript rather than machine-code. We have spent a great deal of time making sure we could use proper OOP (object-oriented programming), and we have spent more than 3 years perfecting a visual application framework with the same depth as the VCL or FMX (the core visual frameworks for C++ builder and Delphi).

The result is that I can knock out a large application that a normal JavaScript coder would spend weeks on – in a single day.

Smart Mobile Studio uses the object-pascal language, a dialect which is roughly 70% compatible with Delphi. Delphi is exceptionally well suited for writing large, data driven applications. It also thrives for embedded systems and low-level system services. In short: it’s a lot easier to maintain 50.000 lines of object pascal code, than 500.000 lines of JavaScript code.

Amibian.js, both the service layer and the visual HTML5 client application, is written completely using Smart Mobile Studio. This gives me as the core developer of both systems a huge advantage (who knows it better than the designer right?). I also get to write code that is truly OOP (classes, inheritance, interfaces, virtual and abstract methods, partial classes etc), because our compiler crafts something called a VMT (virtual method table) in JavaScript.

Traditional JavaScript doesn’t have OOP, it has something called prototypes. With Smart Pascal I get to bring in code from the object-pascal community, components and libraries written in Delphi or Freepascal – which range in the hundreds of thousands. Delphi alone has a massive library of code to pick from, it’s been a popular toolkit for ages (C is 3 years older than pascal).

But how would I use Amibian.js? Do I install it or what?

Amibian.js can be setup and used in 4 different ways:

  • As a true desktop, booting straight into Amibian.js in full-screen
  • As a cloud service, accessing it through any modern browser
  • As a NAS or Kiosk front-end
  • As a local system on your existing OS, a batch script will fire it up and you can use your browser to access it on https://127.0.0.1:8090

So the short answer is yes, you install it. But it’s the same as installing Chrome OS. It’s not like an application you just install on your Linux, Windows or OSX box. The whole point of Amibian.js is to have a platform independent, chipset agnostic system. Something that doesn’t care if you using ARM, x86, PPC or Mips as your CPU of preference. Developers will no doubt install it on their existing machines, Amibian.js is non-intrusive and does not affect or touch files outside its own eco-system.

But the average non-programmer will most likely setup a dedicated machine (or several) or just deploy it on their home NAS.

The first way of enjoying Amibian.js is to install it on a PC or ARM device. A disk image will be provided for supporters so they can get up and running ASAP. This disk image will be based on a thin Linux setup, just enough to get all the drivers going (but no X desktop!). It will start all the node.js services and finally enter a full-screen web display (based on Chromium Embedded) that renders the desktop. This is the method most users will prefer to work with Amibian.js.

The second way is to use it as a cloud service. You install Amibian.js like mentioned above, but you do so on Amazon or Azure. That way you can login to your desktop using nothing but a web browser. This is a very cost-effective way of enjoying Amibian.js since renting a virtual instance is affordable and storage is abundant.

The third option is for developers. Amibian.js is a desktop system, which means it’s designed to host more elaborate applications. Where you would normally just embed an external website into an IFrame, but Amibian.js is not that primitive. Hosting external applications requires you to write a security manifest file, but more importantly: the application must interface with the desktop through the window’s message-port. This is a special object that is sent to the application as a hand-shake, and the only way for the application to access things like the file-system and server-side functionality, is via this message-port.

Calling “kernel” level functions from a hosted application is done purely via the message-port mentioned above. The actual message data is JSON and must conform to the Ragnarok client protocol specification. This is not as difficult as it might sound, but Amibian.js takes security very seriously – so applications trying to cause damage will be promptly shut down.

You mention hosted applications, do you mean websites?

Both yes and no: Amibian.js supports 3 types of applications:

  • Ordinary HTML5/JS based applications, or “websites” as many would call them. But like I talked about above they have to establish a dialog with the desktop before they can do anything useful.
  • Hybrid applications where half is installed as a node.js service, and the other half is served as a normal HTML5 app. This is the coolest program model, and developers essentially write both a server and a client – and then deploy it as a single package.
  • LDEF compiled bytecode applications, a 68k inspired assembly language that is JIT compiled by the browser (commonly called “asm.js”) and runs extremely fast. The LDEF virtual machine is a sub-project in Amibian.js

The latter option, bytecodes, is a bit like Java. A part of the Amibian.js project is a compiler and runtime system called LDEF.

patron_asm2

Above: The Amibian.js LDEF assembler, here listing opcodes + disassembling a method

The first part of the Amibian.js project is to establish the desktop and back-end services. The second part of the project is to create the worlds first cloud based development platform. A full Visual Studio clone if you like, that allows anyone to write cloud, mobile and native applications directly via the browser (!)

Several languages are supported by LDEF, and you can write programs in Object Pascal, Basic and C. The Basic dialect is especially fun to work with, since it’s a re-implementation of BlitzBasic (with a lot of added extras). Amiga developers will no doubt remember BlitzBasic, it was used to create some great games back in the 80s and 90s. It’s well suited for games and multimedia programming and above all – very easy to learn.

More advanced developers can enjoy Object Pascal (read: Delphi) or a sub-set of C/C++.

And please note: This IDE is designed for large-scale applications, not simple snippets. The ultimate goal of Amibian.js is to move the entire development cycle to the cloud and away from the desktop. With Amibian.js you can write a cool “app” in BlitzBasic, run it right in the browser — or compile it server-side and deploy it to your Android Phone as a real, natively compiled application.

So any notion of a “mock desktop for HTML” should be firmly put to the side. I am not playing around with this product and the stakes are very real.

But why don’t you just use ChromeOS?

There are many reasons, but the most important one is chipset independence. Chrome OS is a native system, meaning that it’s core services are written in C/C++ and compiled to machine code. The fundamental principle of Amibian.js is to be 100% platform agnostic, and “no native code allowed”. This is why the entire back-end and service layer is targeting node.js. This ensures the same behavior regardless of processor or host system (Linux being the default host).

Node.js has the benefit of being 100% platform independent. You will find node.js for ARM, x86, Mips and PPC. This means you can take advantage of whatever hardware is available. You can even recycle older computers that have lost mainstream support, and use them to run Amibian.js.

A second reason is: Chrome OS might be free, but it’s only as open as Google want it to be. ChromeOS is not just something you pick up and start altering. It’s dependence on native programming languages, compiler toolchains and a huge set of libraries makes it extremely niche. It also shields you utterly from the interesting parts, namely the back-end services. It’s quite frankly boring and too boxed in for any practical use; except for Google and it’s technology partners that is.

I wanted a system that I could move around, that could run in the cloud, on cheap SBC’s. A system that could scale from handling 10 users to 1000 users – a system that supports clustering and can be installed on multiple machines in a swarm.

A system that anyone with JavaScript knowledge can use to create new and exciting systems, that can be easily expanded and serve as a foundation for rich media applications.

What is this Amiga stuff, isn’t that an ancient machine?

In computing terms yes, but so is Unix. Old doesn’t automatically mean bad, it actually means that it’s adapted and survived challenges beyond its initial design. While most of us remember the Amiga for its games, I remember it mainly for its elegant and powerful operating-system. A system so flexible that it’s still in use around the world – 33 years after the machine hit the market. That is quite an achievement.

image2

The original Amiga OS, not bad for a 33-year-old OS! It was and continues to be way ahead of everyone else. A testament to the creativity of its authors

Amibian.js as the name implies, borrows architectural elements en-mass from Amiga OS. Quite simply because the way Amiga OS is organized and the way you approach computing on the Amiga is brilliant. Amiga OS is much more intuitive and easier to understand than Linux and Windows. It’s a system that you could learn how to use fully with just a couple of days exploring; and no manuals.

But the similarities are not just visual or architectural. Remember I wrote that hosted applications can access and use the Amibian.js services? These services implement as much of the original ROM Kernel functions as possible. Naturally I can’t port all of it, because it’s not really relevant for Amibian.js. Things like device-drivers serve little purpose for Amibian.js, because Amibian.js talks to node.js, and node talks to the actual system, which in turn handles hardware devices. But the way you would create windows, visual controls, bind events and create a modern, event-driven application has been preserved to the best of my ability.

But how does this thing boot? I thought you said server?

If you have setup a dedicated machine with Amibian.js then the boot sequence is the same as Linux, except that the node.js services are executed as background processes (daemons or services as they are called), the core server is initialized, and then a full-screen HTML5 view is set up that shows the desktop.

But that is just for starting the system. Your personal boot sequence which deals with your account, your preferences and adaptations – that boots when you login to the system.

When you login to your Amibian.js account, no matter if it’s just locally on a single PC, a distributed cluster, or via the browser into your cloud account — several things happen:

  1. The client (web-page if you like) connects to the server using WebSocket
  2. Login is validated by the server
  3. The client starts loading preferences files via the mapped filesystem, and then applies these to the desktop.
  4. A startup-sequence script file is loaded from your account, and then executed. The shell-script runtime engine is built into the client, as is REXX execution.
  5. The startup-script will setup configurations, create symbolic links (assigns), mount external devices (dropbox, google drive, ftp locations and so on)
  6. When finished the programs in the ~/WbStartup folder are started. These can be both visual and non-visual.

As you can see Amibian.js is not a mockup or “fake” desktop. It implements all the advanced features you expect from a “real” desktop. The filesystem mapping is especially advanced, where file-data is loaded via special drivers; drivers that act as a bridge between a storage service (a harddisk, a network share, a FTP host, Dropbox or whatever) and the desktop. Developers can add as many of these drivers as they want. If they have their own homebrew storage system on their existing servers, they can implement a driver for it. This ensures that Amibian.js can access any storage device, as long as the driver conforms to the driver standard.

In short, you can create, delete, move and copy files between these devices just like you do on Windows, OSX or the Linux desktop. And hosted applications that run inside their own window can likewise request access to these drivers and work with the filesystem (and much more!).

Wow this is bigger than I thought, but what is this emulation I hear about? Can Amibian.js really run actual programs?

Amibian.js has a JavaScript port of UAE (Unix Amiga Emulator). This is a fork of SAE (scripted Amiga Emulator) that has been heavily optimized for web. Not only is it written in JavaScript, it performs brilliantly and thus allows us to boot into a real Amiga system. So if you have some floppy-images with a game you love, that will run just fine in the browser. I even booted a 2 gigabyte harddisk image ūüôā

But Amiga emulation is just the beginning. More and more emulators are ported to JavaScript; you have Nes, SNes, N64, PSX I & II, Sega Megadrive and even a NEO GEO port. So playing your favorite console games right in the browser is pretty straight forward!

But the really interesting part is probably QEmu. This allows you to run x86 instances directly in the browser too. You can boot up in Windows 7 or Ubuntu inside an Amibian.js window if you like. Perhaps not practical (at this point) but it shows some of the potential of the system.

I have been experimenting with a distributed emulation system, where the emulation is executed server-side, and only the graphics and sound is streamed back to the Amibian.js client in real-time. This has been possible for years via Apache Guacamole, but doing it in raw JS is more fitting with our philosophy: no native code!

I heard something about clustering, what the heck is that?

Remember I wrote about the services that Amibian.js has? Those that act almost like libraries on a physical computer? Well, these services don’t have to be on the same machine — you can place them on separate machines and thus its able to work faster.

47470965_10155861938320906_4959664457727868928_n

Above: The official Amibian.js cluster, 4 x ODroid XU4s SBC’s in a micro-rack

A cluster is typically several computers connected together, with the sole purpose of having more CPU cores to divide the work on. The cool thing about Amibian.js is that it doesn’t care about the underlying CPU. As long as node.js is available it will happily run whatever service you like – with the same behavior and result.

The official Amibian.js cluster consists of 5 ODroid XU4/S SBC (single board computers). Four of these are so-called “headless” computers, meaning that they don’t have a HDMI port – and they are designed to be logged into and software setup via SSH or similar tools. The last machine is a ODroid XU4 with a HDMI out port, which serves as “the master”.

The architecture is quite simple: We allocate one whole SBC for a single service, and allow the service to copy itself to use all the CPU cores available (each SBC has 8 CPU cores). With this architecture the machine that deals with the desktop clients don’t have to do all the grunt work. It will accept tasks from the user and hosted applications, and then delegate the tasks between the 4 other machines.

Note: The number of SBC’s is not fixed. Depending on your use you might not need more than a single SBC in your home setup, or perhaps two. I have started with 5 because I want each part of the architecture to have as much CPU power as possible. So the first “official” Amibian.js setup is a 40 core monster shipping at around $250.

But like mentioned, you don’t have to buy this to use Amibian.js. You can install it on a single spare X86 PC you have, or daisy chain a couple of older PC’s on a switch for the same result.

Why Headless? Don’t you need a GPU?

The headless SBC’s in the initial design all have GPU (graphical processing unit) as well as audio capabilities. What they lack is GPIO pins and 3 additional USB ports. So each of the nodes on our cluster can handle graphics at blistering speed — but that is ultimately not their task. They serve more as compute modules that will be given tasks to finish quickly, while the main machine deals with users, sessions, traffic and security.

The 40 core cluster I use has more computing power than northern europe had in the early 80s, that’s something to think about. And the pricetag is under $300 (!). I dont know about you but I always wanted a proper mainframe, a distributed computing platform that you can login to and that can perform large tasks while I do something else. This is as close as I can get on a limited budget, yet I find the limitations thrilling and fun!

Part of the reason I have opted for a clustered design has to do with future development. While UAE.js is brilliant to emulate an Amiga directly in the browser – a more interesting design is to decouple the emulation from the output. In other words, run the emulation at full speed server-side, and just stream the display and sounds back to the Amibian.js display. This would ensure that emulation, of any platform, runs as fast as possible, makes use of multi-processing (read: multi threading) and fully utilize the network bandwidth within the design (the cluster runs on its own switch, separate from the outside world-wide-web).

I am also very interested in distributed computing, where we split up a program and run each part on different cores. This is a topic I want to investigate further when Amibian.js is completed. It would no doubt require a re-design of the LDEF bytecode system, but this something to research later.

Will Amibian.js replace my Windows box?

That depends completely on what you use Windows for. The goal is to create a self-sustaining system. For retro computing, emulation and writing cool applications Amibian.js will be awesome. But Rome was not built-in a day, so it’s wise to be patient and approach Amibian.js like you would Chrome OS. Some tasks are better suited for native systems like Linux, but more and more tasks will run just fine on a cloud desktop like Amibian.js.

Until the IDE and compilers are in place after phase two, the system will be more like an embedded OS. But when the LDEF compiler and IDE is in place, then people will start using it en-mass and produce applications for it. It’s always a bit of work to reach that point and create critical mass.

tomes

Object Pascal is awesome, but modern, native development systems are quite demanding

My personal need has to do with development. Some of the languages I use installs gigabytes onto my PC and you need a full laptop to access them. I love Amibian.js because I will be able to work anywhere in the world, as long as a browser and normal internet line is available. In my case I can install a native compiler on one of the nodes in the cluster, and have LDEF emit compatible code; voila, you can build app-store ready applications from within a browser environment.

 

I also love that I can set-up a dedicated platform that runs legacy applications, games – and that I can write new applications and services using modern, off the shelve languages. And should a node in the cluster break down, I can just copy the whole system over to a new, affordable SBC and keep going. No super expensive hardware to order, no absurd hosting fees, and finally a system that we all can shape and use in a plethora of systems. From a fully fledged desktop to a super advanced NAS or Router that use Amibian.js to give it’s customers a fantastic experience.

And yes, I get to re-create the wonderful reality of Amiga OS without the absurd egoism that dominates the Amiga owners to this day. I don’t even know where to begin with the present license holders – and I am so sick of the drama that rolling my own seemed the only reasonable path forward.

Well — I hope this helps clear up any misconceptions about Amibian.js, and that you find this as interesting as I do. As more and more services are pushed cloud-side, the more relevant Amibian.js will become. It is perfect as a foundation for large-scale applications, embedded systems — and indeed, as a solo platform running on embedded devices!

I cant wait to finish the services and cluster this sucker on the ODroid rack!

If you find this project interesting, head over to my Patreon website and get involved! I could really use your support, even if it’s just a $5 “high five”. Visit the project at:¬†http://www.patreon.com/quartexNow

Smart Mobile Studio presentation in Oslo

September 28, 2018 Leave a comment

Yesterday evening I traveled to Oslo and held a presentation on Smart Mobile Studio. The response was very positive and I hope that everyone who attended left with some new ideas regarding JavaScript, the direction the world of software is heading –¬†and how Smart Mobile Studio can be of service to Delphi.

Smart Pascal is especially exciting in concert with Rad-Server, where it opens the doors to Node based, platform independent services and sub clustering. With relatively little effort Rad-Server can absorb the wealth that node has to offer through Smart – but on your terms, and under Delphi’s control. The best of both worlds.

You get the stability and structure that makes Delphi so productive, and then infuse that with the flamboyance, flair and async brilliance that JavaScript represents.

More important than technology is the community! It’s been a few years since I took part in the Oslo Delphi Club’s meetups, so it was great to chat with Halvard Vassbotten,¬†Trond Gr√łntoft, Alf Christoffersen, Torgeir Amundsen and Robin Bakker face to face again. I also had the pleasure of meeting some new Delphi developers.

prespic

Presentation at¬†ABG Sundal Collier’s offices in Oslo

Thankfully the number of attendees were a moderate 14, considering this was my first presentation ever. Last time I visited was when our late PaweŇā GŇāowacki presented FMX, and the turnout was in the ballpark of a hundred. So it was an easy-going, laid-back atmosphere throughout the evening.

Conflict of interest?

Some might wonder why a person working for Embarcadero will present Smart Mobile Studio, which some still regard as competition. Smart is not in competition with Delphi and never will be. It is written by Delphi developers for Delphi developers as a means to bridge two worlds. It’s a project of loyalty and passion. We continue because we love what it enables us to do.

The talks on Smart that I am holding now, including the november talk in London, were booked before I started at Embarcadero (so it’s not a case of me promoting Smart in leu of Embarcadero). I also made it perfectly clear when I accepted the job that my work on Smart will continue in my spare time. And Embarcadero is fine with that. So I am free to spend my after-work hours and weekend time as I see fit.

smart_desktop

The Smart Desktop, codename Amibian.js, is a solid foundation for building large-scale web front-ends. Importing Sencha’s JS API’s can be done via our TypeScript wizard

So, after my presentation in London in november Smart Mobile Studio presentations (at least hosted by me) can only take place during weekends. Which is fair and the way it should be.

Recording the English version

Since the presentation last evening was in Norwegian, there was little point in recording it. Norway have a healthy share of Delphi developers, but a programming language available internationally must be presented in English.

techA couple of months back, before I started working for Embarcadero I promised to do a video presentation that would be available on Delphi Developer and YouTube. I very much like to keep that promise. So I will re-do the presentation in English as soon as possible. I would have done it today after work, but buying tech from the US have changed quite dramatically in just a couple of years.

In short: I haven’t received the remaining equipment I ordered for professional video recording and audio podcasting (which is a part of my Patreon offering as well), as such there will be no live video-feed /slash/ webinar – and questions will be limited to either the comment-section on Delphi Developer; or perhaps more appropriate, the Smart Mobile Studio Forums.

I’m hoping to get the HD camera, mic-table-arm and various bits-and-bobs i ordered from the US sometime next week. I have no idea why FedEx have become so difficult lately, but the package is apparently at LaGuardia, and I have to send receipts that document that these items are paid for before they ship them abroad (so the package manifest listing me as the customer, my address, phone number and receipt from the seller is somehow not enough). This is a first for me.

Interestingly they also stopped a package from Embarcadero with giveaways for my upcoming Delphi presentation in Sweden – at which point I had to send them a copy of my work contract to prove that I indeed work for an American company.

But a promise is a promise, so come rain or shine it will be done. Worst case scenario we can put Samsung’s claims to the test and hook up a mic + photo lens and see if their commercials have any merit.

HexLicense, Patreon and all that

September 6, 2018 Comments off

Apparently using modern service like Patreon to maintain components has become a point of annoyance and confusion. I realize that I formulated the initial HexLicense post somewhat vague and confusing, in retrospect I will admit that and also take critique for not spending a little more time on preparations.

Having said that, I also corrected the mistake quickly and clarified the situation. I feel some of the comments have been excessively critical for something that, ultimately, is a service to the community. But I’ll roll with the punches and let’s just put this issue to bed.

From the top please

fromthetopI have several products and frameworks that naturally takes time to maintain and evolve. And having to maintain websites, pay for tax and invoicing services, pay for hosting (and so on), well it consumes a lot of hours. Hours that I can no longer afford to spend (my work at Embarcadero must come first, I have a family to support). So Patreon is a great way to optimize a very busy schedule.

Today developers solve a lot of the business strain by using Patreon. They make their products open source, but give those that support and help fund the development special perks, such as early access, special builds and a more direct line of control over where the different projects and sub-projects are heading.

The public repository that everyone has access to is maintained by pushing the code on interval, meaning that the public “free stuff” (LGPL v3 license) will be some months behind the early-access that patrons enjoy. This is common and the same approach both large and small teams go about things in 2018. Quite radical compared to what we “old-timers” are used to, but that’s how things work now. I just go with flow and try to do the most amount of good on the journey.

Benefits of Patreon

The benefits are many, but first and foremost it has to do with time. Developer don’t have to maintain 3-4 websites, pay for invoicing services on said products, pay hosting fees and rent support forums — instead focus is on getting things done. So instead of an hour here and there, you can (based on the level of support) allocate X hours within a week or weekend that are continuous.

4a128ea6852444fbfc89022be4132e9b

Patreon solves two things: time and cost

Everyone wins. Those that support and help fund the projects enjoy early access and special builds. The community at large wins because the public repository is likewise maintained, albeit somewhat behind the cutting edge code patrons enjoy. And the developers wins because he or she doesn’t have to run around like a mad chicken maintaining X number of websites -wasting more time doing maintenance than building cool new features.

 

And above all, pricing goes down. By spreading the cost over a larger base of interest, people get access to code that used to cost $200 for $35. The more people that helps out, the more the cost can be reduced per tier.

To make it crystal clear what the status of my frameworks and component packages are, here is a carbon copy from HexLicense.com

For immediate release

Effective immediately HexLicense is open-source, released under the GNU Lesser General Public License v3. You can read the details of that license by clicking here.

Patreon model

Patreon_logo.svgIn order to consolidate the various projects I maintain, I have established a Patreon account. This means that people can help fund further development on HexLicense, LDEF, Amibian and various Delphi libraries as a whole. This greatly simplifies things for everyone.

I will be able to allocate time based on a broader picture, I also don’t need to pay for invoicing¬†services, web hosting and more. This allows me to continue to evolve the components and code, but without so many separate product identities to maintain.

Patreon supporters will receive updates before anyone else and have direct access to the latest code at all times. The public bitbucket repository will be updated on interval, but will by consequence be behind the Patreon updates.

Further security

One of the core goals on Patreon is the evolution of a bytecode compiler. This should be of special interest to HexLicense users. Being able to compile modules that hackers will be unable to debug gives you a huge advantage. The engine is designed so that the instruction-set can be randomized for a particular build. Making it unique for your application.

patron_asm1

The LDEF assembler prototype running under Smart Mobile Studio

Well, I want to thank everyone involved. It has been a great journey to produce so many components, libraries and solutions over the years – but now it’s time for me to cut down on the number of projects and focus on core technology.

HexLicense with the update license files will be uploaded to BitBucket shortly.

Sincerly

Jon Lennart Aasenden

 

 

Support my work on Patreon, get awesome stuff

September 2, 2018 3 comments

For well over a decade now I have tried my best to be of service to the Delphi community. I run six pascal forums on Facebook, I teach Delphi for free in my spare time and I help people solve problems, find jobs and get inspired.

“to utterly re-write the traditional development toolchain and create
a desktop environment and development studio that is unbound
by chipset, cpu and platform”

I am about to embark on the biggest journey I have ever undertaken, namely to deliver a technological platforms that combined will give both users and developers unprecedented advantages.

patreon

Support my work by becoming a patron

The challenge with new and awesome technology, is that it can be difficult to convey. The full implications of something revolutionary needs a little bit of gestation, maturity and overview before the “OMG” factor hits home. But thankfully the Delphi and Smart Pascal community is amongst the most learned, creative and innovative I have ever seen. Not to mention the Amiga retro scene that also have supported me – a group made up of hardware wizards, FPGA programmers and hackers that eat assembly code for breakfast.

I won’t dazzle you with empty promises or quick fixes. Every part of what I present here is rooted in code I have running in my lab. I hope that the doors Smart Mobile Studio have opened, the work I have done on the RTL and the products I have made – that they at least have earned me your patience; and that you will read this and see if it’s worthy of your support.

Context

When we released Smart Mobile Studio 3.0 we made a live web desktop demo to showcase some of the potential the technology has to offer. What was not mentioned was that this in fact was not a mockup or slap-dash demo intended to impress you with Quake III or the Bassoon music tracker. It has deeper roots and is a re-formation on the Quartex Desktop API that has been an essential part of Smart Mobile Studio since the beginning.

The desktop, codename Amibian.js, is actually a platform that is a part of a larger, loftier goal. One that was outlined to investors as early as 2013. Sadly I was unable to secure funds for it, despite the fact that two companies are using the prototype for kiosk and embedded systems already (city kiosk terminals in Spain running on ODroid XU4 ARM boards, and also an educational platform for schools in New Zealand).

The goal, to cut it short, is quite simply:¬†to utterly re-write the traditional development toolchain and create a desktop environment and development studio that is unbound by chipset, cpu and platform. In other words, to re-implement and build a “visual studio” environment that lives completely in the cloud, that can be accessed by any modern browser, on any operating system, anywhere in the world.

I’m not talking about Notepad or Ace here, I am talking about a complete IDE with form designer, database designer, cloud endpoints, multi language support and above all – the ability to compile and deploy both virtual and native applications through established build services. All of it JavaScript, all of it running on Node.js, Electron or HTML5.

You wont be drag & dropping components, you will be dropping entire ecosystems.

Smart Mobile Studio, new tools for a new age

When I started some eight years ago, this would have been impossible. There were no compilers that could take a complex language like object pascal or C++ and successfully express that as JavaScript. JavaScript on its own, at least compared to C++ or Delphi, is quite poor. Things we take for granted like classes, linear inheritance, virtual and abstract methods (requires a VMT), interfaces (and more) simply does not exist. There have been some advances lately of course, but JavaScript is and will always be, a prototype based runtime system.

For eight years the Smart Mobile Studio team have worked to create the ecosystem needed to make large-scale application development for JSVM (Javascript virtual machine, the browser, Phonegap, NodeJS and more) a reality. We have forged the compiler, the support code and an RTL spanning thousands of classes from scratch.

If is now possible to write JS based applications that rival native applications both in scope and complexity. This has without a doubt been one of the hardest tasks I have ever been involved in.

With Smart Mobile Studio in place and the foundation stone set – we can finally get to work on the real product. Namely a cloud forge unlike any other.

The Amibian desktop environment

The desktop platform that forms the basis of my work Рwas nicknamed Amibian due to its visual inspiration from Amiga OS 4.1, a modern but somewhat obscure operating system for PPC computers. But while there are cunning visual similarities, Amibian.js is a very different beast under the hood.

First of all Amibian.js is written from scratch to be cloud oriented. The Ragnarok message server at the heart of the system, is capable of delegating hundreds of users each dispatching high data volume simultaneously. It is a server system that is designed from scratch to be clustered, scalable and distributed.

devkit

The Ragnarok message protocol performs brilliantly, here testing IO messages live

You can run it together with the client, forming an OS much like ChromeOS, on something as small as a Tinkerboard ($70 embedded board) or scale it to a 100 node Amazon cluster. If node.js can be installed, Amibian can run. CPU or chipset is quite frankly irrelevant.

This is the foundation that the next generation IDE and compiler toolchain will be built on. A toolchain that doesn’t care if you prefer Linux, Windows, OSX or Android.

If you have a HTML5 compliant browser, you can create full-scale applications with the same level of depth as Delphi, and target 8 operating systems and more than 50 embedded devices.

What does that mean for Delphi users

Like Smart Mobile Studio, Amibian is not meant to compete with Delphi. It is designed to complement and extend Delphi – allowing Delphi developers to reach avenues where native code might be impractical or less cost-effective.

The new compiler is based around the LDEF virtual machine specification that I drafted spring 2018. It is written in Smart Pascal and runs on every system that node.js supports (which as of writing is substantial). LDEF is a bytecode specification designed to make native code generation easy. Unlike .Net or Java, LDEF is a register based virtual machine. It is a cross-section of how ARM, x86 and MC68000¬†CPU’s work in real life. It has stacks, registers, condition flags, data control, program control, absolute and relative addressing; and of course instructions that all CPUs support.

patron_asm1

The LDEF assembler is implemented completely in Smart Pascal. The picture shows the testbed with a visual coding editor. The assembler is meant to run under node.js server-side but can also be hosted on a website or post compiled into a native executable

When executing this bytecode under JavaScript, the runtime uses the subset of JavaScript called “Asm.js” out of the box. AsmJS is more mature than WebAssembly and less restrictive (modules are not sandboxed from the DOM). So to make it short: the code runs close to native courtesy of JIT optimization.

LDef is modular, meaning that parser, compiler, assembler and codegen (the part that converts bytecodes it to something else) are separate modules. Writing a WebAssembly codegen, x86 codegen or ARM codegen can be done separately without breaking the existing tooling.

patron_asm2

Having assembled the code (see picture above) the list command dumps the bytecodes to the console in readable fashion. It is then disassembled using the “dasm” command.

The LDEF prototype has been completely written in Smart Pascal, but a port is underway for Delphi and C++ builder. This gives Delphi developers the benefit of using bytecode libraries in their code. If you install Delphi server-side, you can use Amibian as a pure web front end for Delphi (!)

Create applications anywhere, on anything

Since everything is JavaScript you are no longer bound to chipset or CPU. You can set up Amibian on Amazon or Azure, an office server or an affordable, off the shelves SBC (single board computer). You can daisy chain 10 older PC’s into a cluster and get 5 more years out of the hardware; the compiler is made in JS; it doesn’t care if the real CPU is outdated. It cares about bytes and endian-ness, that’s it.

Screenshot

Early implementation of the desktop, here running native 68k (Amiga) code directly. Both x86 and PPC runtimes are now possible – the days of cloud are here

You can be on holiday in Spain armed only with an iPad and a BlueTooth keyboard, and should inspiration strike, you can login and write your application without even installing an app on your iPad. You just need a modern browser to start writing applications.

Patreon Tiers

Depending on your level of support, you get access to different parts of my work. As of writing I have 4 frameworks that is being maintained and that I want to continue to maintain for those that support me:

  • $5: High five! Support the work as a nice gesture
  • $10: Access to and support for developing my tweening library for VCL
  • $25:¬†License management for VCL and FMX, full source code access to Hexlicense and support for porting Ironwood to Delphi + a new REST based registration server
  • $35:¬†Rage libraries, get full access to the ByteRage database framework, Pixelrage graphics library and support their evolution. The timeline includes SQL and condition parsing which will not be covered by the current running tutorial. Want a clean Delphi alternative to SQLite? Well, let me make it for you.
  • $45:¬†LDEF assembler and virtual machine. Get full source code access to the Smart Pascal assembler (runs on node.js) and the Delphi port as soon as it rolls off the assembly line (pun intended). Enjoy proper documentation for instructions, bytecode format and enjoy both the native and web assembler application!¬†As a bonus, this level gives you access to video tutorials and recordings dealing with LDEF, HexLicense, Tweening and everything else.
  • $50: Amibian and Ragnarok: Amibian.js client, server and development toolchain.
    This is the motherload and you get to enjoy all of it before anyone else.

    • Full access to beta builds, updates, new features – all of it before anyone else!
    • Explore the Ragnarok client / server message API
    • Follow my video tutorials and let me help you dig into Smart Pascal and node.js
    • Ask questions and get a deeper understanding of both Smart Mobile Studio, Amibian.js and LDEF.
    • Have a front seat reserved as we unleash the power of Delphi, Smart Pascal and JavaScript on the world.
  • $100:¬†Amibian Embedded Setup:¬†For the true Amibian.js supporters! You get all the perks of previous tiers, but with an added bonus of pre-made Amibian.js disk images for the ODroid XU4 and the Asus Tinkerboard once LDEF and the IDE has been implemented.These disk images starts the Ragnarok server as a daemon (Linux Service) during the boot sequence. The system then continues booting into a full-screen webview that renders the Amibian.js desktop. There is no Linux desktop involved.
    This is by far the most cost effective setup for Kiosk and Embedded work with either a touch display or keyboard access.

    As an extra perk this version of Amibian.js contains an optimized version of uae.js (Amiga emulation) and is capable of executing ADF disks and harddisk images directly in their own window.

    With the service layer now fully developed, combined with truly platform independent compiler technology Рwe have in fact created an interesting alternative to ChromeOS. One with a minimal footprint that is cost effective and easy to expand. A system that you have full control over and can change, rebrand, modify and enjoy!

    Congratulations! You have helped bring Amibian.js and a whole new development paradigm into this world!

If this wets your appetite then head over to my Patreon site and show your support! I start shipping code to those that support me next week, so get onboard and let’s make it happen!

Final words

26229892_10155095303530906_800744220432589611_nPatreon is not the same as a kickstarter or a formal investment, I think this is important to underline. I hope however that you find my work interesting and that you would like to see this realized.

LDEF is not just a fancy bytecode runtime, it is also a framework that other developers can use to make new languages. The whole point of this is to blow the old limitations away and to really push technology to the maximum.

Being able to write system services that work the same on all operating-systems, and then deploy entire ecosystems¬†– this used to be science fiction. Now it’s not.

I want to thank those that have become patrons – it really means so much! If enough support my work I can allocate more time for implementing the tools the community needs and be of greater service to everyone.

Thank you for your time

Jon Lennart Aasenden

Power for pennies, getting a server rack and preparing my ultimate coding environment

July 18, 2018 Leave a comment

One of the benefits of doing repairs on your house, is that during the cleanup process you come over stuff you had completely forgot about. Like two very powerful Apple blade servers (x86) I received as a present three years ago. I never got around to using them because I there was literally no room in my house for a rack cabinet.

Sure, a medium model rack cabinet isn’t that big (the size of a cabin refrigerator), but you also have to factor in that servers are a lot more noisy than desktop PCs; the older they are the more noise they make. So unless you have a good spot to place the cabinet, where the noise wont make it unbearable to be around,¬† I suggest you just rent a virtual instance at Amazon or something. It really depends on how much service coding you do, if you need to do dedicated server and protocol stress testing (the list goes on).

Power for pennies

serverrack

Sellers photo. It needs a good clean, but this kit would have set you back $5000 a decade ago; so picking this up for $400 is almost ridicules.

The price of such cabinets (when buying new ones) can be anything from $800 to $5000 depending on the capacity, features and materials. My needs for a personal server farm are more than covered by a medium cabinet. If it wasnt for my VMWare needs I would say it was overkill. But some of my work, especially with node.js and Delphi system services that should handle terabytes of raw data reliably 24/7, that demands a hard-core testing environment.

Having stumbled upon my blade servers I decided to check the local second-hand online forum; and I was lucky enough to find (drumroll) a second-hand cabinet holding a total of 10 blades for $400. So I’ll be picking up this beauty next weekend. It will be so good to finally get my blades organized. Not to mention all my SBC / Node.js cluster experiments centralized in one physical location. Far¬†away from my home office space (!)

Interestingly, it comes fitted with 3 older servers. There are two Dell web and file servers, and then a third, unmarked mystery box (i3 cpu + sata caddies so that sounds good).

It really is amazing how much cpu fire-power you can pick up for practically nothing these days. $50 buys you a SBC (single board computer) that will rival a Pentium. $400 buys you a 10 blade cabinet and 3 servers that once powered a national newspaper (!).

VMWare delights

All the blades I have mentioned so far are older models. They are still powerful machines, way more than $400 livingroom NAS would get you. So my node.js clustering will run like a dream and I will be able to host all my Delphi development environments via VMware. Which brings us neatly to the blade I am really looking forward to get into the rack.

I bought an empty server blade case back in 2015. It takes a PSU, motherboard, fans and everything else is there (even the six caddies for disks). Into this seemingly worthless metal box I put a second generation Intel i7 monster (Asus motherboard), with 32 gigabyte ram Рand fitted it with a sexy NVidia GEFORCE GTX 1080 TI.

37013365_10155541149775906_3122577366065348608_o

All my Delphi work, Smart work and various legacy projects I maintain, all in one neat rack

This little monster (actually it takes up 2 blade-spots) allows me to run VMWare server, which gives me at least 10 instances of Windows (or Linux, or OSX) at the same time. It will also be able to host and manage roughly 1000 active Smart Desktop users (the bottleneck will be the disk and network more than actual computation).

Being a coder in 2018 is just fantastic!

Things we could only dream about a decade ago can now be picked up for close to nothing (compared to the original cost). Just awesome!

 

Using Smart Mobile Studio under Linux

April 22, 2018 Leave a comment

Every now and then when I post something about Smart Mobile Studio, an individual or two wants to inform me how they cannot use Smart because it’s not available for Linux. While more rare, the same experience happens now and then with OS X.

linux

The Smart desktop demo running in Firefox on Ubuntu, with Quake 3 at 60 fps

While the request for Linux or OS X support is both valid and understandable (and something we take seriously), more often than not these questions can be a pre-cursor to a larger picture; one that touches on open-source, pricing and personal philosophical points of view; often with remarks on pricing.

Truth be told, the price we ask for Smart Mobile Studio is close to symbolic. Especially if you take the time to look at the body of work Smart delivers. We are talking hundreds of hand written units with thousands of classes, each spesifically adapted for HTML5, Phonegap and Node.js. Not to mention ports of popular JavaScript frameworks.

If you compare this to native object pascal development tools with similar functionality, they can set you back thousands of dollars. Our asking price of $149 for the pro edition, $399 for the enterprise edition, and a symbolic $42 for the basic edition, that is an affordable solution. Also keep in mind that this gives you access to updates for a duration of 12 months. When was the last time you bought a full development suite that allows you to write mobile applications, platform independent servers, platform independent system services and HTML5 web applications for less that $400 ?

prices

Our price model is more than reasonable considering what you get

With platform independent we mean that in the true sense of the word: once compiled, no changes are required. You can write a system service on Windows and it will run just fine under Linux or OS X. No re-compile needed. You can take a server and copy it to Amazon or Azure, run it in a cluster or scale it from a single instance to 100 instances without any change. That is unheard of for object pascal until now.

Smart Mobile Studio is the only object pascal development system that delivers a stand-alone IDE, stand-alone compiler, a wast object-oriented run-time library written from scratch to cover HTML5, Node.js and embedded systems that run JavaScript.

And yes, we know there are other systems in circulation, but none of them are even close to the functionality that we deliver. Functionality that has been polished for seven years now. And our RTL is growing every day to capture and expose more and more advanced functionality that you can use to enrich your products.

21272810_1585822718147745_4358597225084661216_o

The RTL class browser shows the depth of our RTL

As of writing we have a team of six people working on Smart Mobile Studio. We have things in our labs that is going to change the way people build applications forever. We were the first to venture into this new landscape. There were nobody we could imitate, draw inspiration from or learn from. We had to literally make the path as we moved forward.

And our vision and goal remains the same today as it was seven years ago: to empower object pascal developers – and to secure their investment in the language and methodology that is object pascal.

Discipline and purpose

There is so much I would like to work on right now. Things I want to add to Smart Mobile Studio because I find them interesting, powerful and I know people are going to love them. But that style of development, the “Garage days” as people call it, is over. It does wonders in the beginning of a project maybe, but eventually you reach a stage where a formal timeline and business plan must be carved in stone.

And once defined, you have to stick to it. It would be an insult to our customers if we pivoted left and right on a whim. Like every company we have room for research, even a couple of skunkwork projects, but our primary focus is to make our foundation rock solid before further growth.

j5

By tapping into established JavaScript frameworks you can cover more than 40+ embedded systems and micro-controllers. More and more hardware supports JS for automation

Our “garage days” ended around three years ago, and through hard work we defined our timeline, business model and investor program. In 2017 we secured enough capital to continue full-time development.

Our timeline has been published earlier, but we can re-visit some core points here:

The visual components that shipped with Smart Mobile Studio in the beginning, were meant more as examples than actual ready-to-use modules. This was common for other development platforms of the day, such as Xamarin’s C# / Mono toolchain, where you were expected to inherit from and implement aspects of a “partial class”. This is also why Smart Pascal has support for partial classes (neither Delphi or Freepascal supports this wonderful feature).

native

One of our skunkwork projects is a custom linux distro that runs your Smart applications directly in the Linux framebuffer. No X or desktop, just your code. Here running “the smart desktop” as the only visual front-end under x86 vmware

Since developers coming from Delphi had different expectations, there was only one thing to do. To completely re-write every single visual control (and add a few new controls) so that they matched our customers expectations. So the first stretch of our timeline has been 100% dedicated to the visual aspects of our RTL. While doing so we have made the RTL faster, more efficient, and added some powerful sub-systems:

  • A dedicated theme engine
  • Unified event delegation
  • Storage device classes
  • Focus and control tracking
  • Support for relative position modes
  • Support for all boxing models
  • Inline linking ( {$R “file.js”} will now statically link an external library)
  • And much, much more

So the past eight months has been all about visual components.

22814515_1630289797034370_9138255627706616601_n

Theming is important

The second stretch, which we are in right now, is dedicated to the non-visual infrastructure. This means in particular Node.js but also touches on non-visual components, TAction support and things that will appear in updates this year.

Node.js is especially important since it allows you to write platform and chipset independent web servers, system services and command-line applications. This is pioneering work and we are the first to take this road. We have successfully tamed the alien landscape of JavaScript, both for client, mobile and server Рand terraformed it into a familiar, safe and productive environment for object pascal developers.

I feel the results speak for themselves, and our next update brings Smart Mobile Studio to the next level: full stack cloud and web development. We now cover back-end, middle-ware and front-end technologies. And our RTL now stretches from micro-controllers to mobile application to clustered cloud services.

This is the same technology used by Netflix to process terabytes of data every second on a global scale. Which should tell you something about the potential involved.

Working on Linux

Since Smart Mobile Studio was designed to be a swiss army knife for Delphi and Lazarus developers, capable to reaching segments of the market where native code is unsuitable – our primary focus is Microsoft Windows. At least for now.

Delphi itself is a Windows-based development system, and even though it supports multiple targets, Windows is still the bread and butter of commercial software development.

Like I mentioned above, we have a timeline to follow, and until we have reached the end of that line, we are not prepared to refactor our IDE for Linux or OS X. This might change sooner than people think, but until our timeline for the RTL is concluded, we will not allocate time for making the IDE platform independent.

But, you can actually run Smart Mobile Studio on both Linux and OS X today.

Linux has a system called Wine. This is a system that implements the Windows API, but delegates all the calls to Linux. So when a Windows program calls a WinAPI through Wine, its delegated to Linux variation of the same call. This is a massive undertaking, but it has years of work behind it and functions extremely well.

So on linux you can install it by opening up a shell and typing:

sudo apt-get install wine

I take for granted here that your Linux flavour has aperture installed (I’m using Ubuntu since that is easy to work with), which is the package manager that gives you the “apt-get” command. If you have some other system then just google how to install a package.

With Wine and it’s dependencies installed, you can run the Smart Mobile Studio installer. Wine will create a virtual, sandboxed disk for you – so that all the files end up where they should. Once finished you punch in the license serial number as normal, and voila – you can now use Smart Mobile Studio on Linux.

Note: in some cases you have to right-click the SmartMS.exe and select “run with -> Wine”, but usually you can just double-click the exe file and it runs.

Smart Mobile Studio on OSX

Wine has been ported to OS X, but it’s more user friendly. You download a program called wine-bottler, which takes Smart and bundles it with wine + any dependencies it needs. You can then start Smart Mobile Studio like it was a normal OS X application.

Themes and look

The only problem with Wine is that it doesnt support Windows themes out of the box. It would be illegal for them to ship those files. But you can manually copy over the windows theme files and install them via the Wine config application. Once installed, Smart will look as it should.

By default the old Windows 95 look & feel is used by Wine. Personally I dont care too much about this, it’s being able to code, compile and run the applications that matters to me – but if you want a more modern look then just copy over the Windows theme files and you are all set.

 

 

Smart Mobile Studio 3.0 and beyond

March 20, 2018 Leave a comment

cascade_03With Smart Mobile Studio 3.0 entering its second beta, Smart Pascal developers are set for a boost in quality, creativity and power. We have worked extremely hard on the product this past year, including a complete rewrite of all our visual controls (and I mean all). We also introduced a completely new theme engine, one that completely de-couples visual appearance from structural architecture (it also allows scripting inside the CSS theme files).

All of that could be enough for a version bump, but we didn’t stop there. Much of the sub-strata in Smart has been¬†re-implemented. Focus has been on stability, speed and future growth. The system is now divided into a set of name-spaces (System, SmartCL, SmartNJ, Phonegap, and Espruino), making it easier to navigate between the units as well as expanding the codebase in the future.

To better understand the namespaces and why this is a good idea, let’s go through how our units are organized.

smart_namespace

The RTL is made to expand easily and preserve as much functionality as possible

  • The System namespace is the foundation. It contains clean, platform independent code. Meaning code that doesn’t rely on the DOM (browser) or runtime (node). Focus here is on universal code, and to establish common object-pascal classes.
  • Our SmartCL namespace contains visual code, meaning code and controls that targets the browser and the DOM. SmartCL rests on the System namespace and draws functionality from it. Through partial classes we also expand classes introduced in the system namespace. A good example is System.Time.pas and SmartCL.Time.pas. The latter expands the class TW3Dispatch with functionality that will only work in the DOM.
  • SmartNJ is our high-level nodejs namespace. Here you find classes with fairly complex behavior such as servers, memory buffers, processes and auxillary classes. SmartNJ draws from the system namespace just like SmartCL. This was done to avoid multiple implementations of streams, utility classes and common functions. Being able to enjoy the same functionality under all platforms is a very powerful thing.
  • Phonegap is our namespace for mobile devices. A mobile application is basically a normal visual application using SmartCL, but where you access extra functionality through phonegap. Things like access to a device’s photos, filesystem, dialogs and so on is all delegated via phonegap.
  • Espruino is a namespace for working with Espruino micro-controllers. This has been a very low-level affair so far, due to size limitation on these devices. But with our recent changes you can now, when you need to, tap into the system namespace for more demanding behavior.

As you can see there is a lot of cool stuff in Smart Mobile Studio, and our codebase is maturing nicely. With out new organization we are able to expand both horizontally and vertically without turning the codebase into a gigantic mess (the VCL being a prime example of how not to implement a multi-platform framework).

Common behavior

One of the coolest things we have added has to be the new storage device classes. As you probably know the browser has a somewhat “limited” storage mechanism. You are stuck with name-value pairs in the cache, or a filesystem that is profoundly frustrating to work with. To remedy this we took the time to implement a virtual filesystem (in memory filesystem) that emits data to the cache; we also implemented a virtual storage device stack on top of it, one for each target (!).

In short, if a target has IO capability, we have implemented a storage “driver” for it. So instead of you having to write 4-5 different storage mechanisms – you can now write the storage code once, and it works everywhere.

This is a pretty cool system because it doesn’t limit us to local device storage. We can have device classes that talk to Google-Storage, One-Drive, Dropbox and so on. It also opens up for custom storage solutions should you already have this pre-made on your server.

Database support, a quick overview

Databases have always been available in Smart Mobile Studio. We have units for WebSQL, IndexDB and SQLite. In fact, we even compiled SQLite3 from native C code to asm.js, meaning that the whole database engine is now pure JavaScript and no-longer dependant on W3C standards.

smart_db

Each DB engine is implemented according to a framework

Besides these we also have TW3Dataset which is a clean, Smart Pascal implementation of a single table dataset (somewhat inspired by Delphi’s TClientDataset). In our previous beta we upgraded TW3Dataset with a robust expression parser, meaning that you can now set filters just like Delphi does. And its all written in Smart Mobile Studio which means there are no dependencies.

 

And ofcourse, there is also direct connections to Embarcadero Datasnap servers, and Remobjects SDK servers. This is excellent if you have an already existing Delphi infrastructure.

A unified DB framework

If you were hoping for a universal DB framework in beta-2 of v3.0, sadly that will not be the case. The good news is that databases should make it into v3.2 at the latest.

Databases looks simple: table, rows and columns right? But since each database engine known to JavaScript is written different from the next, our model has to take height for these and be dynamic enough to deal with them.

The model we used with WebSQL is turning out to be the best way forward I feel, but its important to leave room for reflection and improvements.

So getting our DB framework established is a priority for us, and we have placed it on our timeline for (at the latest) v3.2. But im hoping to have it done by v3.1. So it’s a little ahead of us, but we need that time to properly evolve the framework.

Smart Desktop [a.k.a Amibian.js]

The feedback we have received on our Smart Desktop demos have been pretty overwhelming. It is also nice to know that our prototype is being used to deliver software to schools and educational centers. So our desktop is not going away!

smart_desktop

Fancy a game of Quake at 60+ fps? Web assembly rocks!

But we are not rushing into this without some thought first. The desktop will become a project type like I have written about many times before. So you will be able to create both the desktop and client applications for it. The desktop is suitable for software that requires a windowing environment (a bit like Sencha or similar frameworks). It is also brilliant for kiosk displays and as a remote application hub.

Our new storage device system came largely from Amibian, and with these now a part of our RTL we can clean up the prototype considerably!

Smart assembler

It may sound like an oxymoron, but a lab project we created while testing our parser framework (system.text.parser unit) turned into an exercise in compiler / assembler making. We implemented a virtual machine that runs instructions represented by bytecodes (fairly straight ahead stuff). It supports the most common assembler methods, vaguely inspired by the Motorolla 68k processor with a good dose of ARM thrown in for good measure.

smart_assembler

Yes that is a full parser, assembler and runtime model

If you ponder why on earth this would be interesting, consider the following: most web platforms allow for scripting by third-party developers. And by opening up for that these, the websites themselves become prone to attacks and security breaches. There is no denying that any JS based framework is very fragile when potentially hundreds of unknown developers are hacking away at it.

But what if you could offer third parties to write plugins using more traditional languages? Perhaps a dialect of pascal, a subset of basic or perhaps C#? Wouldnt that be much better? A language and (more importantly) runtime that you have 100% control over.

While our assembler, disassembler and runtime is still in its infancy (and meant as a demo and excercise), it has future potential. We also made the instructions in such a way that JIT compiling large chunks of it is possible – and the output (or codegen) can be replaced by for example web assembly.

Right now it’s just a curiosity that people can play with. But when we have more time I will implement high-level parsers and codegens that emit code via this assembler. Suddenly we have a language that runs under node.js, in the browser or any modern JS runtime engine – and its all done using nothing but Smart Mobile Studio.

Well, stay tuned for more!

LDef parser done

July 21, 2017 Leave a comment

Note: For a quick introduction to LDef click here: Introduction to LDef.

Great news guys! I finally finished the parser and model builder for LDef!

02237439ec5958f6ec7362f726a94696-cogwheels-red-circle-icon-by-vexelsThat means we just need to get the assembler ported. This is presently running fine under Smart Pascal (I like to prototype things there since its faster) – and it will be easy to port it over to Delphi and Freepascal after the model has gone through the steps.

I’m really excited about this project and while I sadly don’t have much free time – this is a project I truly enjoy working on. Perhaps not as much as Smart Pascal which is my baby, but still; its turning into a fantastic system.

Thoughts on the architecture

One of the things I added support for, and that I have hoped that Embarcadero would add to Delphi for a number of years now, is support for contract coding. This is a huge topic that I’m not jumping into here, but one of the features it requires is support for entry and exit sections. Essentially that you can define code that executes before the method body and directly after it has finished (before the result is returned if it’s a function).

This opens up for some very clever means of preventing errors, or at the very least give the user better information about what went wrong. Automated tests also benefits greatly from this.

For example,  a normal object pascal method looks, for example, like this:

procedure TForm1.MySpecialMethod;
begin
  writeln("You called my-special-method")
end;

The basis of contract design builds on the classical and expands it as such:

procedure TForm1.MySpecialMethod;
  Before()
  begin
    writeln("Before my-special-method");
  end;

  After()
  begin
    writeln("After my-special-method");
  end;

begin
  writeln("You called my-special-method")
end;

Note: contract design is a huge system and this is just a fragment of the full infrastructure.

What is cool about the before/after snippets, is that they allow you to verify parameters before the body is even executed, and likewise you get to work on the result before the value is returned (if any).

You mights ask, why not just write the tests directly like people do all the time? Well, that is true. But there will also be methods that you have no control over, like a wrapper method that calls a system library for instance. Being able to attach before/after code for externally defined procedures helps take the edge off error testing.

Secondly, if you are writing a remoting framework where variant data and multi-threaded invocation is involved – being able to check things as they are dispatched means catching potential errors faster – leading to better performance.

As always, coding techniques is a source of argument – so im not going into this now. I have added support for it and if people don’t need it then fine, just leave it be.

Under LDef assembly it looks like this:

public void main() {
  enter {
  }

  leave {
  }
}

Well I guess that’s all for now. Hopefully my next LDef post will be about the assembler being ready – leaving just the linker. I need to experiment a bit with the codegen and linker before the unit format is complete.

The bytecode-format needs to include enough information so that the linker can glue things together. So every class, member, field etc. must be emitted in a way that is easy and allows the linker to quickly look things up. It also needs to write the actual, resulting method offsets into the bytecode.

Have a happy weekend!

Smart Pascal: Amibian vs. FriendOS

July 20, 2017 Leave a comment

This is not a new question, and despite my earlier post I still get hammered with these on a weekly basis – so lets dig into this subject and clean it up.

I fully understand that for non-developers suddenly having two Amiga like web desktops can be a bit confusing; especially since they superficially at least do many of the same things. But there is actually a lot of co-incidence surrounding all this, as well as evolution of the general topic. People who work with a topic will naturally come up with the same ideas from time to time.

But ok, lets dig into this and clear away any confusion

You know about FriendOS right? It looks a lot like Amibian

20100925-Designer

Custom native web servers has been a part of Delphi for ages, so it’s not that exciting for us

“A lot” is probably stretching it. But ok: ¬†FriendOS is a custom server system with a sexy desktop front-end written in HTML5. So you have a server that is custom written to interact with the browser in a special way. This might sound like a revolution to non-developers but it’s actually an established technology; its been a part of Delphi and C++ builder for at least 12 years now (Intraweb¬†being the best example, Raudus another). So if you are wondering why im not dazzled, it’s because this has been there for a while.

The whole point of Amibian.js is to demonstrate a different path; to get away from the native back-end and to make the whole system portable and platform independent. So in that regard the systems are diametrically different.

maxresdefault

Custom web servers that talk to your web-app is old news. Delphi developers have done this for a decade at least and it’s not really interesting at this point. Node.js holds much greater promise.

What FriendOS has done that is unique, and that I think is super cool – is to couple their server with RDP (remote desktop protocol) and some nice video streaming for smooth video chat. Again these are off the shelves parts that anyone can add once you have a native back-end, it’s not really hard to code but time-consuming; especially when you are potentially dealing with large number of users spawning threads all over the place. I think Friend-Labs have done an exceptional good job here.

When you combine these features it creates the impression of an operating system like environment. And this is perfectly fine for ordinary users. It all depends on your needs and what exactly you use the computer for.

And just to set the war-mongers straight:¬†FriendOS is not going up against Amibian. it’s going up against ChromeOS, Nayu and and a ton of similar systems; all of them with deep pockets and an established software portfolio. We focus on software development. Not even in the same ballpark.

To be perfectly frank: I see no real purpose for a web desktop except when connected to a context. There has to be an advantage beyond isolating web functions in one place. You need something special that your system does better than others, or different than others. Amibian has been about UAE.js and to run retro games in a familiar environment. And thus create a base that Amiga lovers can build on and play with. Again based on our prefab for customers that make embedded systems and use our compiler and RTL for that.

If you have a hardware product like a NAS, a ticket system or a retro-game machine and want to have a nice web front-end for it: then it makes sense. But there is absolutely nothing in both our systems that you can’t whip-up using Intraweb or Raudus in a few weeks. If you have the luxury of a native back-end, then adding Active Directory support is a matter of dropping a component. You can even share printers and USB devices over the wire if you like, this has been available to Delphi and c++ developers for ages. The “new” factor here, which FriendOS does very well i might add, is connectivity.

This might sound like criticism but it’s really not. It’s honesty and facts. They are going to need some serious cash to take on Google, Samsung, LG and various other players that have been doing similar things for a long time (or about to jump on the same concepts) — Amibian.js is for Amiga fans and people who use Smart Pascal to write embedded applications. We don’t see anything to compete with because Amibian is a prefab connected to a programming language. FriendOS is a unification system.

A programming language doesnt have the aspirations of a communication company. So the whole “oh who is best” or “are you the same” is just wrong.

Ok you say it’s not competing, but why not?

To understand Amibian.js you first need to understand Smart Pascal (see Wikipedia article on Smart Pascal). Smart Pascal (smartmobilestudio.com) is a software development studio for writing software using web technology rather than native machine-code. It allows you to create whatever you like, from games to servers, or kiosk software to the next Facebook clone.

Our focus is on enabling our customers to quickly program robust mobile applications, servers, kiosk software, games or large JavaScript projects; products that would otherwise be hard to manage if all you have is vanilla JavaScript. I mean why spend 2 years coding something when you can do it in 2 months using Smart? So a web desktop is just ridicules when you understand how large our codebase is and the scope of the product.

smart

Under Smart Pascal what people know as Amibian.js is just a project type. There is no competition between FriendOS and Amibian because a web desktop represents a ridicules small piece of our examples; it’s literally mistaking the car for the factory. Amibian is not our product, it is a small demo and prefab (pre fabricated system that others can download and build on) project that people use to save time. So under Smart, creating your own web desktop is a piece of cake, it’s a click, and then you can brand it, expand it and do whatever you like with it. Just like you would any project you create in Visual Studio, Delphi or C++ builder.

So we are not in competition with FriendOS because we create and deliver development tools. Our customers use Smart Pascal to create web environments both large and small, and naturally we deliver what they need. You could easily create a FriendOS clone in Smart if you got the skill, but again – that is but a tiny particle in our codebase.

Really? Amibian.js is just a project under Smart Pascal?

Indeed. Our product delivers a full object-oriented pascal compiler, debugger and IDE. So you can write classes, use inheritance and enjoy all the perks of a high-level language — and then compile this to JavaScript.

You can target node.js, the browser and about 90+ embedded devices out of the box. The whole point of Smart Pascal is to avoid the PITA that is writing large applications in JavaScript. And we do this by giving you a classical programming language that was made especially for application authoring, and then compile that to JavaScript instead.

Screenshot

Amibian.js is just a tiny, tiny part of what Smart Pascal is all about

This is a massive undertaking that started back in 2009/2010 and involves a high-quality compiler, linker, debugger and code generator; a full IDE with a ton of capabilities and last but not least: a huge run-time library that allows you to work with the DOM (document object model, or HTML) and node.js from the vantage point of a programmer.

Most people approach web development as a designer. They write html and then style them using a stylesheet. They work with colors, aspects and pages. Which means people who traditionally write programs falls between two chairs: first they must learn about html and css, and secondly a language which is ill equipped for large scale applications (imagine writing adobe photoshop in nothing but JS. Sure it’s possible, but wouldnt you rather spend a month coding that than a year? In a language that actually makes sense?).

With Smart you approach web development like you do writing programs. You work with visual controls, change properties, write code in response to events. Even writing your own visual controls that you can re-use and inherit from later is both fun and easy. So rather than ending up with a huge was of spaghetti code, which sadly is the fate of most large-scale JavaScript projects — Smart lets you work like you are used to. In a language better suited for the task.

And yes, I was not kidding when I said this was a huge undertaking. The source code in our codebase is close to 2.5 gigabytes. And keep in mind that this is source-code and libraries. So it’s not something you slap together over the weekend.

20108543_10154652373945906_5493167218129195758_n

The Smart source-code is close to 2.5 gigabytes. It has taken years to complete

But why do Amibian and FriendOS both focus on the Amiga?

That is pure co-incidence. The guys over at Friend Labs started out on the Amiga just like we did. So when I updated our desktop project (previously called Quartex Media Desktop) the Amiga look and feel came natural to me.

commodoreI’m a huge retro-computing fan that loves the Amiga. When I sat down to rewrite our window manager I loved the way Amiga OS 4.x looked, so I decided to implement an UI inspired by that.

People have to remember that the Amiga was a huge success in Scandinavia, so finding developers that are in their late 30s or early 40s that didn’t own an Amiga is harder than you think.

So the fact that we all root our ideas back to the Amiga is both co-incidence and a mutual passion for a great platform. One that really should have survived the financial onslaught of fat CEO’s and thir minions in the board.

But Amibian does a lot of what FriendOS does?

Probably. JavaScript is multi-tasking by default so if loading external URL’s into window containers, doing live resize and other things is what you refer to then yes. But that is the nature of web programming. Its like creating a bucket if you want to carry water; it is a natural first step of an evolutionary pattern. It’s not like FriendOS is copying us I would imagine.

240_F_61497799_GnuUiuJliH9AyOJTeo6i3bS8JNN7wgr2

For the record Smart started back in 2010 and the media desktop came in with the first hotfix, so its been available years before Friend-Labs even existed. Creating a desktop has not been a huge part of what we do because mobile applications, building a rich and solid run-time-library with hundreds of classes for our customers – and making an IDE that is great to use, that is our primary job.

We didn’t even know FriendOS existed. Let alone that it was a Norwegian product.

But you posted that you worked for FriendOS earlier?

Yes I did, very briefly. I was offered a position and I worked there for a month. It was a chance to work side by side with legends like David John Pleasance, ex head of Commodore for europe; and also my childhood hero Francois Lionet, author of Amos Basic for the Amiga way back in the 80’s and 90s.

blastfromthepast

We never forget our childhood heroes

Sadly we had our wires crossed. I am an awesome object pascal developer, while the guys at Friend-Labs are awesome C developers. I work primarily on Windows while they work mostly on Linux. So in essence they hired a Delphi developer to work in a language he doesn’t know on a platform he havent used.

They simply took for granted that I worked in C/C++, while I took for granted that they used object pascal. Its an easy mistake to make and its not the first time; and probably not the last.

Needless to say the learning curve would be extremely high for any developer (learning a new operating-system and programming language at the same time as you are supposed to be productive).

When my girlfriend suddenly faced a life threatening illness the situation became worse. It was impossible for me to commute or leave her side for the unforeseeable future; so when you add the six months learning curve to this situation; six months of not being able to contribute on the level I am used to; well I am old enough to know how that ends. So I did what was best for everyone and resigned.

Besides, I am a damn good Delphi developer with standing invitation to many companies; so it made more sense to just take a step backwards. Which was not fun because I really enjoyed the short time I was there. But, it was not meant to be.

And that is basically all there is to it.

Ok. But if Smart is a development tool, will it support Friend-OS ?

This is something that I really want to do. But since The Smart Company is a proper company with stocks, shareholders and investors – it’s not a decision I can take on my own. It is something that must be debated by the board. But personally yeah, I would love that.

friend

As they grow, so does the need for proper development tools

One of the reasons I hope FriendOS succeeds is because it’s a win-win situation. The more they expand the more relevant Smart becomes. Say what you will about JavaScript but writing large and complex applications is not easy by any measure.

So the moment we introduce Smart Pascal for Friend, their users will be able to write large applications rapidly, with better time-to-market and consequent ROI. So it’s a win-win. If they succeed then we get a bigger market; If they don’t we havent lost anything.

This may sound extremely self-serving, but Friend-Labs have had the same chance as everyone else to invest in Smart; our investor plans have been available for quite some time, and we have to do what is best for our company.

But what about Amibian, was it just a short thing?

Not at all. It is put on hold for a few months while we release the next generation RTL. Which is probably the biggest update in the history of Smart Pascal. We have a very clear agenda ahead of us and Amibian.js is (as underlined) a very small part of what we do.

But Amibian is written using our next generation RTL, and without that our customers cant really do much with it. So it’s important to get the RTL out first and then work on the IDE to reflect its many new features. After that – Amibian.js development will continue.

The primary target for Amibian.js is embedded devices and kiosk systems, coupled with full-screen web applications and hardware front-ends (NAS and backup devices being great examples). So the desktop will run on affordable, off the shelves hardware starting at $40 and all the way up to the most powerful and expensive x86 boards on the market. Cheap solutions like Raspberry PI, ODroid XU4 and Tinkerboard will deliver what you today need a dedicated $120 x86 board to achieve.

kiosk-systems

Our desktop will run on many targets and is platform independent by design

This means that our deskop has a wildly different modus operandi. We will not require a constant connection to a remote server. Amibian will happily boot up on a single device, regardless of processor type.

Had we coded our backend using Delphi or C++ builder (native like FriendOS have done) we would have been finished months ago. And I could have caught up with FriendOS in a couple of months if I wanted to. But that is not in our agenda. We have written our server framework for node.js as we coded the desktop ¬†– which means it’s platform and OS agnostic by design. If node.js runs, Amibian will run. It wont care if you are running on a $40 embedded board or the latest Intel i9 cpu.

Last words

I really hope this has helped and that the confusion between Amibian.js and our agenda, versus what Friend-Labs is doing, is now clearer.

Amibian666

From Norway with love

I wish Friend-Labs the very best and hope they are successful in their endeavour. They have worked very hard on the product and deserve that. And while I might come over as arrogant at times, im really not.

Web desktops have been around for a long time now (Asustor is my favorite) through Delphi and C++ builder and that is just facts. But that doesn’t mean you can’t put things together in new and interesting ways! Smart itself was first put together by existing technology. It was said to be impossible by many because JavaScript and object pascal are unthinkable companions. But it turned out to be a perfect match.

As for the future – personally I don’t believe in the web-desktop outside a specific context, something to give it purpose if you like. I believe for instance that Amibian.js will be awesome for Amiga users when its running on a $99 ARM laptop. Where the system boots straight into a full-screen desktop and where UAE.js is fully integrated into the core, making retro-gaming and running old programs close to seamless. That I can believe in.

But it would make no sense running Amibian or FriendOS in a browser on top of a Windows desktop or a full Ubuntu X session. Unless the virtual desktop functions as your corporate window with access to company mail, documents and essentially what every web-based intranet already does. So once again we end up with the fact that this has already been done. And unless you create a unique context for it, it just wont have any appeal.¬†This is also why I havent pursued¬†the same tech Friend-Labs have, because that’s not where the exciting stuff is happening.

But I will happily be proven wrong, because that means an even bigger market for us should we decide to support the platform.

LDef and bytecodes

July 14, 2017 Leave a comment

LDef, short for Language Definition format, is a standard I have been formulating for a couple of years. I have taken my experience with writing various compilers and parsers, and also my experience of writing RTL’s and combined it all into a standard.

programming-languages-for-iot-e1467856370607LDef is a way for anyone to create their own programming language. Just like popular libraries and packages deals with the low-level stuff, like Gr32 which is an excellent graphics library — LDef deals with the hard stuff and leaves you with the pleasant job of defining what the language should look like.

The idea is to make a language construction kit if you like, where the underlying engine is flexible enough to express the languages we know and love today – and also powerful enough to express new ideas. For example: let’s say you want to create an awesome new game system (just as an example, it applies to any system that can be automated). You have the means and skill to create the actual engine – but how are you going to market it? You will be up against monoliths like Unity and simple “click and play” engines like ClickTeam Fusion, Game Maker and the likes.

Well, the only way to make good games is hard work. There is no two ways about it. You can fake your way only so far – so at the end of the day you want to give your users something solid.

In our example of publishing a game-engine, I think that you would stand a much better chance of attracting users if you hooked that engine up to a language. A language that is easy to use, easy to learn and with commands that are both specific and non-specific to your engine.

There are some flavours of Basic that has produced knock-out games for decades, like BlitzBasic. That language alone has produced hundreds of titles for both PC, XBox and even Nintendo. So it’s insanely fast and not a pushover.

And here is the cool part about LDEF: namely that it makes it easy for you to design your own languages. You can use one of the pre-defined languages, like object pascal or visual basic if that is what you like – but ultimately the fun begins when you start to experiment with new ideas and language features. And it’s fun when you get to that point, because all the nitty gritty is handled. You get to focus on the superficial stuff like syntax and high level functions. So you can shave off quite a bit of development time and make coding fun again!

The paradox of faster bytecodes

Bytecodes used to be to slow for anything substantial. On 16-bit machines bytecodes were used in maybe one language (that I know of) and that was the ‘E’ compiler. The E language was maybe 30 years ahead of its time and is probably the only language I can think of that fits cloud programming like hand in glove. But it was also an excellent system automation language (scripting) and really turned some heads back in the late 80s and early 90s. REXX was only recently added to OS X, some 28 years after the Amiga line of computers introduced it to the general public.

ldef_bytecodes

Bytecode dump of a program compiled with the node.js version of the compiler

In modern times bytecodes have resurfaced through Java and the .NET framework which for some reason caused a stir in the whole development community. I honestly never bought into the hype, but I am old enough to remember the whole story – so I’m probably not the Microsoft demographic anyways. Java hyped their virtual machine opcodes to the point of exhaustion. People really are stupid. Man did they do a number on CEO’s and heads of R&D around the world.

Anyways, end of the story was that Intel and AMD went with it and did some optimizations that could help bytecodes run faster. The stack was optimized with Java, because let’s face it – it is the proverbial assault on the hardware. And the cache was expanded on command from the emper.. eh, Microsoft. Also (if I remember correctly) the “jump to pointer” and various branch instructions were made to execute faster. I remember reading about this in Dr. Dobbs Journal and Microsoft Developer Magazine; granted it was a few years ago. What was interesting is the symbiotic relationship that exists between Intel and Microsoft, I really didn’t know just how closely knit these guys were.

Either way, bytecodes in 2017 is capable of a lot more than they ever were on 16-bit and early 32-bit systems. A cpu like Intel i5 or i7 will chew through bytecodes like a warm knife on butter. It depends on how you orchestrate the opcodes and how much work you delegate to the various instructions.

Modeled instructions

Bytecodes are cool but they have to be modeled right, or its all going to end up as a bloated, slow and limited system. You don’t want to be to low-level, otherwise what is the point of bytecodes? Bytecodes should be a part of a bigger picture, one that could some day be modeled using FPGA’s for instance.

The LDef format is very flexible. Each instruction is ultimately a single 32-bit longword (4 bytes) where each byte holds key information about the command, what data is forward in the cache and how it should be read.

The byte organization is:

  • 0 – Actual opcode
  • 1 – Instruction layout

Depending on the instruction layout, the next two bytes can hold different values. The instruction layout is a simple value that defines how the data for the instruction is passed.

  • Constant to register
  • Variable to register
  • Register to register
  • Register to variable
  • Register to stack
  • Stack to register
  • Variable to variable
  • Constant to variable
  • Stack to variable
  • Program counter (PC) to register
  • Register to Program counter
  • ED (exception data) to register
  • Register to exception-data

As you can probably work out from the information here, this list hints to some archetectual features. Variables are first class citizens in LDef, they are allocated, managed and released using instructions. Constants can be either automatically handled and references by id (a resource chunk is linked to the class binary) or “in place” and compiled directly into the assembly as part of the instruction. For example

load R[0], "this is a test"

This line of code will take the constant “this is a test” and move it into register #0. You can choose to have the text-data stored as a proper resource which is appended to the compiled bytecode (all classes and modules have a resource chunk) or just compile “as is” and have the data read directly. The first option is faster and something you can adjust with compiler optimization options. The second option is easier to work with when you debug since you can see the data directly as a part of the debug memory dump.

And last but not least there are registers. 32 of them in number (so for the low-level coders out there you should have few limitations with regards to register mapping). All operations (like divide, multiply etc) operate on registers only. So to multiply two variables they first have to be moved into registers and the multiplication is executed there – then you can move the result to a variable afterwards.

ldef_asm

LDef assembly code. Simple but extremely effective

The reason registers is used in my runtime system – is because you will not be able to model a FPGA with high-level concepts like “variables” should someone every try to implement this as hardware. Things like registers however is very easy to model and how actual processors work. You move things from memory into a cpu register, perform an action, and then move the result back into memory.

This is where Java made a terrible mistake. They move all data onto the stack and then call the operation. This simplifies execution of instructions since there is never any registers to keep track of, but it just murders stack-space and renders Java useless on mobile devices. The reason Google threw out classical Java (e.g Java as bytecodes) is due to this fact (and more). After the first android devices came out they quickly switched to a native compiler – because Java was too slow, to power-hungry and required too much memory (especially stack space) to function properly. Battery life was close to useless and the only way to save Java was to go native. Which is laughable because the entire point of Java was mobility, “compile once run everywhere” — yeah well, that didn’t turn out to well did it ūüėÄ

Dot net improved on this by adding a “load resource” type instruction, where each method will load in the constant data by number – and they are loaded into pre-defined slots (the variables you have used naturally). Then you can execute operations in typical “A + B to C” style (actually all of that is omitted since the compiler already knows both A, B and C). This is much more stack friendly and places performance penalty on the common language runtime (CLR).

Sadly Microsoft’s platform, like everything Microsoft does, requires a pretty large infrastructure. It’s not simple, elegant and fast – it’s more monolithic, massive and resource hungry. You don’t see .net being the first thing ported to a new platform. You typically see GCC followed by Freepascal.

LDef takes the bytecode architecture one step further. On assembly level you reference data using identifiers just like .net, and each instruction is naturally executed by the runtime-engine – but data handling is kept within the virtual realm. You are expected to use the registers as temporary holding slots for your information. And no operations are ever done directly on a variable.

The benefit of this is:

  • Better payload balancing
  • Easier to JIT since the architecture is closer to real assembly
  • Retains important aspects of how real hardware works (with FPGA in mind)

So there are good reasons for the standard, all of them very good.

C like intermediate language

With assembler so clearly defined you would expect ¬†assembly to be the way you work. In essence that what you do, but since OOP is built into the system and there are structures you are expected to populate — structures that would be tedious to do in raw unbridled assembler, I have opted for a C++ inspired intermediate language.

ldef_app

The LDEF assembler kitchen sink

You would half expect me to implement pascal, but truth be told pascal parsing is more complex than C parsing, and C allows you to recycle parsers more easily, so dealing with sub structures and nested regions is less maintainance and easier to write code for.

So there is no spesific reason why I picked C++ as a intermediate language. I would prefer pascal but I also think it would cause a lot of confusion since object pascal will be the prime citizen of LDef languages. My other language, N++ also used curley brackets so I’m honestly not strict about what syntax people prefer.

Intermediate language features supported are:

  • Class declarations
  • Struct declarations
  • Parameter to register mapping
  • Before mehod code (enter)
  • After method code (leave)
  • Alloc section for class fields
  • Alloc section for method variables

The before and after code for methods is very handy. They allow you to define code that should execute before the actual procedure. On a higher level when designing a new language, this is where you would implement custom allocation, parameter testing etc.

So if you call this method:

function testcode() {
    enter {
      writeln("this is called before the method entry");
    }
    leave { 
      writeln("this is called after the method exits");
    }
  writeln("this is the method body");
}

Results in the following output:

this is called before the method entry
this is the method body
this is called after the method exits

 

When you work with designing your language, you eventually.

Truly portable

Now I have no aspirations in going into competition with neither Oracle, Microsoft or anyone in between. Like most geeks I do things I find interesting and enjoy working within a field of computing that is stimulating and personally rewarding.

Programming languages is an area where things havent really changed that much since the golden 80s. Sure we have gotten a ton of fancy new software, and the way people use languages has changed – but at the end of the day the languages we use havent really changed that much.

JavaScript is probably the only language that came out of the blue and took the world by storm, but that is due to the central role the browser holds for the internet. I sincerely doubt JavaScript would even have made a dent in the market otherwise.

LDef is the type of toolkit that can change all this. It’s not just another language, and it’s not just another bytecode engine. A lot of thought has gone into its architecture, not just notions of “how can we do this or that”, but big ideas about the future of computing and how IOT will sculpt the market within 5-8 years. And the changes will be permanent and irrevocable.

Being able to define new languages will be utmost important in the decade ahead. We don’t even know the landscape yet but we can extrapolate some ideas based on where technology is going. All of it in broad strokes of course, but still – there are some fundamental facts about computers that the timeless and havent aged a day. It’s like mathematics, the Pythagorean theorem may be 2500 years old but it’s just as valid today as it was back then. Principles never die.

I took the example of a game engine at the start of this article. That might have been a poor choice for some, but hopefully the general reader got the message: the nature of control requires articulation. Regardless if you are coding an invoice system or a game engine, factors like time, portability and ease of use will be just as valid.

There is also automation to keep your eye on. While most of it is just media hype at this point, there will be some form of AI automation. The media always exaggerates things, so I think we can safely disregard a walking, self-aware Terminator type robot replacing you at work. In my view you can disregard as much as 80% of what the media talks about (regardless of topic). But some industries will see wast improvement from automation. The oil and gas sector are the most obvious. A the moment security is as good as humans can make them – which means it is flawed and something goes wrong every day around the globe. But smart pumping stations and clever pressure measurements and handling will make a huge difference for the people who work with oil. And safer oil pipelines means lives saved and better environmental control.

The question is, how do we describe programs 20 years from now? Is our current tools up for the reality of IOT and billions of connected devices? Do we even have a language that runs equally well as a 1000 instance server-cluster as it would as a stand alone program on your desktop? When you start to look into parallel computing and multi-cluster data processing farms – languages like C# and C++ makes little sense. Node.js is close, very close, but dealing with all the callbacks and odd limitations of JavaScript is tedious (which is why we created Smart Pascal to begin with).

The future needs new things. And for that to happen we first need tools to create them. Which is where my passion is.

Node, native and beyond

When people create compilers and programming languages they often do so for a reason. It could be that their own tools are lacking (which was my initial motivation), or that they have thought of a better way to achieve something; the reasons can be many. In Microsofts case it was revenge and spite, since they were unsuccessful in stealing Java away from Sun Microsystems (Oracle now owns Java).

LDEF

LDef binaries are fairly straight forward. The less fluff the better

Point is, you implement your idea using the language you know – on the platform you normally use. So for me that is object pascal on windows. I’m writing object pascal because while the native compiler and runtime is written in Delphi – it is made to compile under Freepascal for Linux and OS X.

But the primary work is done in Smart Pascal and compiled to JavaScript for node.js. So the native part is actually a back-port from Smart. And there is a good reason I’m doing it this way.

First of all I wanted a runtime and compiler system that would require very little to run. Node.js has grown fat in features over the past couple of years – but out of the box node.js is fast, portable and available almost anywhere these days. You can write some damn fast and scalable cloud servers with node (and with fast i mean FAST, as in handling thousands of online gamers all playing complex first person worlds) and you can also write some stable and rock solid system services.

Node is turning into a jack of all trades, capable of scaling and clustering way beyond what native software can do. Netflix actually re-wrote their entire service stack using node back in 2014. The old C++ and ASP approach was not able to handle the payload. And every time they did a small change it took 45 minutes to compile and get a binary to test. So yeah, node.js makes so much more sense when you start looking a big-data!

So I wanted to write LDef in a way that made it portable and easy to implement. Regardless of platform, language and features. Out of the box JavaScript is pretty naked stuff and the most advanced high-level feature LDef uses is buffers to deal with memory. everything else is forced to be simple and straight forward. No huge architecture or global system services, just a small and fast runtime and your binaries. And that’s all you need to run your compiled applications.

Ultimately, LDef will be written in LDef itself and compile itself. Needing only a small executable stub to be ported to a new platform. Most of mono C# for Linux is written in C# itself – again making it super easy to move mono between distros and operating systems. You can’t do that with the Visual Studio, at least not until Microsoft wants you to. Neither would you expect that from Apple XCode. Just saying.

The only way to achieve the same portability that mono, freepascal and C/C++ has to offer, is naturally to design the system as such from the beginning. Keep it simple, avoid (operatingsystem) globalization at all cost, and never-ever use platform bound APIs except in the runtime. Be Posix but for everything!

Current state of standard and licensing

The standard is currently being documented and a lot of work has been done in this department already. But it’s a huge project to document since it covers not only LDEF as a high-level toolkit, but stretches from the compiler to the source-code it is designed to compile to the very binary output. The standard documentation is close to a book at this stage, but that’s the way it has to be to ensure every part is understood correctly.

But the question most people have is often “how are you licensing this?”.

Well, I really want LDEF to be a free standard. However, to protect it against hijacking and abuse – a license must be obtained for financial entities (as in companies) using the LDEF toolkit and standard in commercial products.

I think the way Unreal software handles their open-source business is a great example of how things should be done. They never charge the little guy or the Indie developer – until they are successful enough to afford it. So once sales hits a defined sum, you are expected to pay a small percentage in royalties. Which is only fair since Unreal engine is central to the software to begin with.

So LDef is open source, free to use for all types of projects (with an obligation to pay a 3% royalty for commercial products that exceeds $4999 in revenue). Emphasis is on open source development. As long as the financial obligations by companies and developers using LDEF to create successful products is respected, only creativity sets the limit.

If you use LDEF to create a successful product where you make 50.000 NKR (roughly USD 5000) you are legally bound to pay 3% of your product revenue monthly for the duration of the product. Which is extremely little (3% of $5000 is $150 which is a lot less than you would pay for a Delphi license, the latter costing between upwards of USD 3000).

 

Smart-Pascal: A brave new world, 2022 is here

April 29, 2017 6 comments

Trying to explain what Smart Mobile Studio does and the impact it can have on your development cycle is very hard. The market is rampant with superficial frameworks that promises you the world, and investors have been taken for a ride by hyped up, one-click “app makers” more than once.

I can imagine that being an investor is a bit like panning for gold. Things that glitter the most often turn out to be worthless – yet fortunes may hide beneath unpolished and rugged surfaces.

Software will disrupt most traditional industries in the next 5-10 years.
Uber is just a software tool, they don’t own any cars, yet they are now the
biggest taxi company in the world. -Source: R.M.Goldman, Ph.d

So I had enough. Instead of trying to tell people what I can do, I decided I’m going to show them instead. As the american’s say: “talk is cheap”. And a working demonstration is worth a thousand words.

Care to back that up with something?

A couple of weeks ago I published a video on YouTube of our Smart Pascal based desktop booting up in VMWare. The Amiga forums went off the chart!

vmware

For those that havent followed my blog or know nothing about the desktop I’m talking about, here is a short¬†summary of the events so far:


Smart Mobile Studio is a compiler that takes pascal, like that made popular in Delphi or Lazarus, and compiles it JavaScript instead of machine-code.

This product has shipped with an example of a desktop for years (called “Quartex media desktop”). It was intended as an example of how you could write a front-end for kiosk machines and embedded devices. Systems that could use a touch screen as the interface between customer and software.

You have probably seen those info booths in museums, universities and libraries? Or the ticket machines in subways, train-stations or even your local car-wash? All of those are embedded systems. And up until recently these have been small and expensive computers for running Windows applications in full-screen. Applications which in turn talk to a server or local database.

Smart Mobile Studio is able to deliver the exact same (and more) for a fraction of the price. A company in Oslo replaced their $300 per-board unit – with off the shelves $35 Raspberry Pi mini-computers. They then used Smart Pascal to write their client software and ran it in a fullscreen-browser. The Linux distribution was changed to boot straight into Firefox in full-screen. No Linux desktop, just a web display.

The result? They were able to cut production cost by $265 per unit.


Right, back to the desktop. I mentioned the Amiga community. This is a community of coders and gamers that grew up with the old Commodore machines back in the 80s and 90s. A new Amiga is now on the way (just took 20+ years) – and the look and feel of the new operating-system, Amiga OS 4.1, is the look and feel I have used in The Smart Desktop environment. First of all because I grew up on these machines myself, and secondly because the architecture of that system was extremely cost-effective. We are talking about a system that delivered pre-emptive multitasking in as little as 512Kb of memory (!). So this is my “ode to OS 4” if you will.

And the desktop has caused quite a stir both in the Delphi community, cloud community and retro community alike. Why? Because it shows some of the potential cloud technology can give you. Potential that has been under their nose all this time.

And even more important: it demonstrate how productive you can be in Smart Pascal. The operating system itself, both visual and non-visual parts, was put together in my spare time over 3 weeks. Had I been able to work on it daily (as a normal job) I would have knocked it out in a week.

A desktop as a project type

All programming languages have project types. If you open up Delphi and click “new” you are greeted with a rich menu of different projects you can make. From low-level DLL files to desktop applications or database servers. Delphi has it all.

delphistuff

Delphi offers a wide range of projects types you can create

The same is true for visual studio. You click “new solution” and can pick from a wide range of different projects. Web projects, servers, desktop applications and services.

Smart Pascal is the only system where you click “new project” and there is a type called “Smart desktop” and “Smart desktop application”. In other words, the power to create a full desktop is now an integrated part of Smart Pascal.

And the desktop is unique to you. You get to shape it, brand it and make it your own!

Let us take a practical example

Imagine a developer given the task to move the company’s aging invoice and credit system from the Windows desktop – to a purely web-based environment.

legacy2The application itself is large and complex, littered with legacy code and “quick fixes” going back decades. Updating such a project is itself a monumental task – but having to first implement concepts like what a window is, tasks, user space, cloud storage, security endpoints, look and feel, back-end services and database connectivity; all of that before you even begin porting the invoice system itself ? The cost is astronomical.

And it happens every single day!

In Smart Pascal, the same developer would begin by clicking “new project” and selecting “Smart desktop”. This gives him a complete desktop environment that is unique to his project and company.

A desktop that he or she can shape, adjust, alter and adapt according to the need of his employer.¬†Things like file-type recognition, storage, getting that database ‚Äď all of these things are taken care of already. The developer can focus on his task, namely to deliver a modern implementation of their invoice and credit¬†software ‚Äď not waste months¬†trying to force JavaScript frameworks do things they simply lack the depth to deliver.

Once the desktop has the look and feel in order, he would have to make a simple choice:

  • Should the whole desktop represent the invoice system or ..
  • Should the invoice system be implemented as a secondary application running on the desktop?

If it’s a large and dedicated system where the users have no need for other programs running, then implementing the invoice system inside the desktop itself is the way to go.

If however the customer would like to expand the system later, perhaps add team management, third-party web-services or open-office like productivity (a unified intranet if you like) – then the second option makes more sense.

On the brink of a revolution

The developer of 2022 is not limited to the desktop. He is not restricted to a particular operating system or chip-set. Fact is, cloud has already reduced these to a matter of preference. There is no strategic advantage of using Windows over Linux when it comes to cloud software.

Where a traditional developer write and implement a solution for a particular system (for instance Microsoft Windows, Apple OS X or Linux) ‚Äď cloud developers deliver whole eco systems; constellations of software constructed from many parts, both micro-services developed in-house but also services from¬†others; like Amazon or Azure.

All these parts co-operate and can be combined through established end-point standards, much like how components are used in Delphi or Visual Studio today.

smartdesk

The Smart Desktop, codename “Amibian.js”

Access to products written in Smart is through the browser, or sometimes through a ‚Äúpaper thin‚ÄĚ native host¬†(Cordova Phonegap,¬†Delphi and C/C++) that expose system level functionality.¬†These hosts wrap your application in a native, executable container ready for Appstore or Google Play.

Now the visual content is typically the same, and is only adapted for a particular device. The real work is divided between the client (which is now very much capable) and your server back-end.

So people still write code in 2022, but the software behaves differently and is designed to function as a group (cluster). And this requires a shift in the way we think.

asmjs

Above: One of my asm.js prototype compilers. Lets just say it runs fast!

Scaling a solution from processing 100 invoices a minute to handling 100.000 invoices a minute – is no longer a matter of code, but of architecture. This is where the traditional, native only approach to software comes up short, while more flexible approaches like node.js is infinitely more capable.

What has emerged up until now is just the tip of the ice-berg.

Over the next five to eight years, everything is going to change. And the changes will be irrevocable and permanent.

Running your Smart Pascal server as a system-level daemon is very easy once you know what to look for :)

The Smart Desktop back-end running as a system service on a Raspberry PI

As the Americans say, talk is cheap – and I’m done talking. I will do this with you, or without you. Either way it’s happening.

Nightly-build of the desktop can be tested here: http://quartexhq.myasustor.com/

Smart Pascal, the next generation

April 15, 2017 1 comment

I want to take the time to talk a bit about the future, because like all production companies we are all working towards lesser and greater goals. If you don’t have a goal then you are in trouble; Thankfully our goals have been very clear from the beginning. Although I must admit that our way there has been.. “colorful” at times.

When we started back in 2010 we didn’t really know what would become of our plans. We only knew that this was important; there was a sense of urgency and “we have to build this” in the air; I think everyone involved felt that this was the case, without any rational explanation as to why. Like all products of passion it can consume you in a way – and you work day and night on turning an idea into something real. From the intangible to the tangible.

transitions_callback

It seems like yesterday, but it was 5 years ago!

By the end of 2011 / early 2012, Eric and myself had pretty much proven that this could be done. At the time there were more than enough nay-sayers and I think both of us got flamed quite often for daring to think different. People would scoff at me and say I was insane to even contemplate that object pascal could ever be implemented for something as insignificant and mediocre as JavaScript. This was my first meeting with a sub-culture of the Delphi and C++ community, a constellation I have gone head-to-head with on many occasions. But they have never managed to shake my resolve as much as inch.

 

 

When we released version 1.0 in 2012 some ideas about what could be possible started to form. J√łrn¬†defined plans for a system we later dubbed “Smart net”. In essence it would be something you logged onto from the IDE – allowing you to store your projects in the cloud, compile in the cloud (connected with Adobe build services) and essentially¬†move parts of your eco-system to the cloud. Keep in mind this was when people still associated cloud with “storage”; they had not yet seen things like Uber or Netflix¬†or played Quake 3 at 160 frames per second,¬†courtesy of asm.js in their browser.

The second part would be a website where you could do the same, including a live editor, access to the compiler and also the ability to buy and sell components, solutions and products. But for that we needed a desktop environment (which is where the Quartex Media Desktop came in later).

cool

The first version of the Media Desktop, small but powerful. Here running in touch-screen mode with classical mobile device layout (full screen forms).

Well, we have hit many bumps along the road since then. But I must be honest and say, some of our detours have also been the most valuable. Had it not been for the often absurd (to the person looking in) research and demo escapades I did, the RTL wouldn’t be half as powerful as it is today. It would deliver the basics, perhaps piggyback on Ext.js or some lame, run of the mill¬†framework – and that would be that. Boring, flat and limited.

What we really wanted to deliver was a platform. Not just a website, but a rich environment for creating, delivering and enjoying web and cloud based applications. And without much fanfare – that is ultimately what the Smart Desktop and it’s sexy node.js back-end is all about is all about.

We have many project types in the pipeline, but the Smart Desktop type is by far the most interesting and powerful. And its 100% under your control. You can create both the desktop itself as a project – and also applications that should run on that desktop as separate projects.

This is perfectly suited for NAS design (network active storage devices can usually be accessed through a web portal on the device), embedded boards, intranets and even intranets for that matter.

You get to enjoy all the perks of a multi-user desktop, one capable of both remote desktop access, telnet access, sharing files and media, playing music and video (we even compiled the mp4 codec from C to JavaScript so you can play mp4 movies without the need for a server backend).

The Smart Desktop

The Smart Desktop project is not just for fun and games. We have big plans for it. And once its solid and complete (we are closing in on 46% done), my next side project will not be more emulators or demos Рit will be to move our compiler(s) to Amazon, and write the IDE itself from scratch in Smart Pascal.

smart desktop

The Smart Desktop – A full desktop in the true sense of the word

And yeah, we have plans for EmScripten as well – which takes C/C++ and compiles it into asm.js. It will take a herculean effort to merge our RTL with their sandboxed infrastructure –¬†but the benefits are too great to ignore.

As a bonus you get to run native 68k applications (read: Amiga applications) via emulation. While I realize this will be mostly interesting for people that grew up with that machine – it is still a testament to the power of Smart and how much you can do if you really put your mind to it.

Naturally, the native IDE wont vanish. We have a few new directions we are investigating here – but native will absolutely not go anywhere. But cloud, the desktop system we are creating, is starting to become what we set out to make five years ago (has it been half a decade already? Tempus fugit!). As you all know it was initially designed as an example of how you could write full-screen applications for Raspberry PI and similar embedded devices. But now its a full platform in its own right – with a Linux core and node.js heart, there really is very little you cannot do here.

scsc

The Smart Pascal compiler is one of our tools that is ready for cloud-i-fication

Being able to login to the Smart company servers, fire up the IDE and just code – no matter if you are: be it Spain, Italy, Egypt, China or good old USA¬†— is a pretty awesome thing!

Clicking compile and the server does the grunt work and you can test your apps live in a virtual window; switch between device layouts and targets — then hit “publish” and it goes to Cordova (or Delphi) and voila – you get a message back when binaries for 9 mobile devices is ready for deployment. One click to publish your applications on Appstore, Google play and Microsoft marketplace.

Object pascal works

People may have brushed off object pascal (and from experience those people have a very narrow view of what object pascal is all about), but when they see what Smart delivers, which in itself is written in Delphi, powered by Delphi and should be in every Delphi developer’s toolbox — i think it should draw attention to both Delphi as a product, object pascal as a language – and¬†smart as a solution.

With Smart it doesn’t matter what computer you use. You can sit at home with the new A1222 PPC Amiga, or a kick-ass Intel i7 beast that chew¬†virtual machines for breakfast. If your computer can handle a modern website, then you can learn object pascal and work directly in the cloud.

desktop_embedded

The Smart Desktop running on cheap embedded hardware. The results are fantastic and the financial savings of using Smart Pascal on the kiosk client is $400 per unit in this case

Heck you can work off a $60 ODroid XU4, it has more than enough horsepower to drive the latest chrome or Firefox engines. All the compilation takes place on the server anyways. And if you want a Delphi vessel rather than phonegap (so that it’s a Delphi application that opens up a web-view in full-screen and expose features to your smart code) then you will be happy to know that this is being investigated.

More targets

There are a lot of systems out there in the world, some of which did not exist just a couple of years ago. FriendOS is a cloud based operating system we really want to support, so we are eager to get cracking on their SDK when that comes out. Being able to target FriendOS from Smart is valuable, because some of the stuff you can do in SMS with just a bit of code – would take weeks to hand write in JavaScript. So you get a productive edge unlike anything else – which is good to have when a new market opens.

As far as Delphi is concerned there are smaller systems that Embarcadero may not be interested in, for example the many embedded systems that have come out lately. If Embarcadero tried to target them all – it would be a never-ending cat and mouse game. It seems like almost every month there is a new board on the market. So I fully understand why Embarcadero sticks to the most established vendors.

ov-4f-img

Smart technology allows you to cover all your bases regardless of device

But for you, the programmer, these smaller boards can repsent thousands of dollars worth of saving. Perhaps you are building a kiosk system and need to have a good-looking user interface that is not carved in stone, touch capabilities, low-latency full-duplex communication with a server; not much you can do about that if Delphi doesnt target it. And Delphi is a work horse so it demands a lot more cpu than a low-budget ARM SoC can deliver. But web-tech can thrive in these low-end environments. So again we see that Smart can compliment and be a valuable addition to Delphi. It helps you as a Delphi developer to act on opportunities that would otherwise pass you by.

So in the above scenario you can double down. You can use Smart for the user-interface on a low power, low-cost SoC (system on a chip) kiosk — and Delphi on the server.

It all depends on what you are interfacing with. If you have a full Delphi backend (which I presume you have) then writing the interface server in Delphi obviously makes more sense.

If you don’t have any back-end then, depending on your needs or future plans, it could be wise to investigate if node.js is right for you. If it’s not – go with what you know. You can make use of Smart’s capabilities either way to deliver cost-effective, good-looking device front-ends of mobile apps. Its valuable tool in your Delphi toolbox.

Better infrastructure and rooting

So far our support for various systems has been in the form of APIs or “wrapper units”. This is good if you are a low-level coder like myself, but if you are coming directly from Delphi without any background in web technology – you wouldn’t even know where to start.

So starting with the next IDE update each platform we support will not just have low-level wrapper units, but project types and units written and adapted by human beings. This means extra work for us Рbut that is the way it has to be.

As of writing the following projects can be created:

  • HTML5 mobile applications
  • HTML5 mobile console applications
  • Node.js console applications
  • node.js server applications
  • node.js service applications (requires PM2)
  • Web worker project (deprecated, web-workers can now be created anywhere)

We also have support for the following operating systems:

  • Chrome OS
  • Mozilla OS
  • Samsung Tizen OS

The following API’s have shipped with Smart since version 1.2:

  • Khronos browser extensions
  • Firefox¬†spesific API
  • NodeWebkit
  • Phonegap
    • Phonegap provides native access to roughly 9 operating systems. It is however cumbersome to work with and beta-test if you are unfamiliar with the “tools of the trade” in the JavaScript world.
  • Whatwg
  • WAC Apis

Future goals

The first thing we need to do is to update and re-generate ALL header files (or pascal units that interface with the JavaScript libraries) and make what we already have polished, available, documented and ready for enterprise level use.

kiosk-systems

Why pay $400 to power your kiosk when $99 and Smart can do a better job?

Secondly, project types must be established where they make sense. Not all frameworks are suitable for full project isolation, but act more like utility libraries (like jQuery or similar training-wheels). And as much as possible of the RTL made platform independent and organized following our namespace scheme.

But there are also other operating systems we want to support:

  • Norwegian made Friend OS, which is a business oriented cloud desktop
  • Node.js OS is very exciting
  • LG WebOS,¬†and their Enyo application framework
  • Asustor DLM web operating system is also a highly attractive system to support
  • OpenNAS has a very powerful JavaScript application framework
  • Segate Nas OS 4 likewise use JavaScript for visual, universal applications
  • Microsoft Universal¬†Platform allows you to create truly portable, native speed JavaScript applications
  • QNap QTS web operating system [now at version 4.2]

All of these are separate from our own NAS and embedded device system: Smart Desktop, which uses node.js as a backend and will run on anything as long as node and a modern browser is present.

Final words

I hope you guys have enjoyed my little trip down memory lane, and also the plans we have for the future. Personally I am super excited about moving the IDE to the cloud and making Smart available 24/7 globally – so that everyone can use it to design, implement and build software for the future right now.

Smart Pascal Builder (or whatever nickname we give it) is probably the first of its kind in the world. There are a ton of “write code on the web” pages out there, but so far there is not a single hard-core development studio like I have in mind here.

So hold on, because the future is just around the corner ūüėČ

Delphi developer on its own server

April 4, 2017 Leave a comment

While the Facebook group will naturally continue exactly like it has these past years, we have set up a server active Delphi developers on my Quartex server.

This has a huge benefit: first of all those that want to test the Smart Desktop can do so from the same domain – and people who want to test Smart Mobile Studio can do so with myself just a PM away. Error reports etc. will still need to be sent to the standard e-mail, but now I can take a more active role in supervising the process and help clear up whatever missunderstanding could occur.

casebook

Always good with a hardcore Smart, Laz, amibian.js forum!

Besides that we are building a lively community of Delphi, Lazarus, Smart and Oxygene/Remobjects developers! Need a job? Have work you need done? Post an add — wont cost you a penny.

So why not sign up? Its free, it wont bite you and we can continue regardless of Facebook’s up-time this year..

You enter here and just fill out user/pass and that’s it:¬†http://quartexhq.myasustor.com/sharetronix/

Amiga revival, Smart Pascal and growing up

March 11, 2017 1 comment

Maybe its just me but the Amiga is kinda having a revival these days? Seems to me like the number of people going back to the Amiga has just exploded the past couple of years. Much of that is no doubt thanks to my buddy Gunnar Kristjannsen’s excellent work on the Amibian distro for Raspberry PI. Making a high-end Amiga experience that would have cost you thousands of dollars available at around $35.

amifuture

Looking forward to some cosy reading

While Gunnar’s great distro is no doubt a huge factor in this, I believe its more than just easy access. I think a lot of us that grew up with the system, who lived the Amiga daily from elementary school all the way to college – have come full circle. We spend our days coding on PC’s, Mac’s or making mobile software – but deep down inside, I think all of us are still in love with that magical machine; The Commodore Amiga.

I am honestly at a loss for words on this (and that’s a first, most days you can’t get me to shut the hell up). Why should a 30-year-old system attract me more, and still cause so much joy in my life – compared to the latest stuff? I mean, I got a fat ass I7 that growls when you start it with 64 gigabyte ram, SSD and all the extras; I got macs all over the house, the latest consoles – and enough embedded boards to start my own arcade if I so desired.

Yet at the end of the day, when the kids are in bed and GF firmly planted in front of her favorite tv show, fathers are down in basements all around europe. Not watching porn, not practising black magic or trying to transform led into gold, nope: coding in assembler on a mc68k processor running at a whopping 7Mhz and loving every minute of it!

Today the madness held no bounds and forced me, out of sheer perverted joy, to order 4 copies of Amiga Future magazine (yes there are still magazines for the Amiga, believe it or not), a few posters, a mousemat and (drumroll) the ever sexy A1222. Actually that was a lie, I ordered that weeks ago, Trevor Dickenson¬†over at A-EON hooked me up so im getting it as soon as it comes off the assembly line. And for those that don’t know, the A1222 is the new affordable Amiga that is released today. It’s not a remake of the older models, but a brand new thing. I havent been this giddy about a piece of silicon since I fell into a double-d cup at a beach in Spain last year.

Smart Pascal

It made sense to unite my two great computing passions, namely the object pascal language and Amiga into one package. So whenever I have some spare time I work my ass off on the update for Smart Mobile Studio. And it’s getting probably the biggest “demo” ever shipped with a programming language.

What? Well, a remake of the Amiga operating system. But not just a simple css-styled shallow lookalike. You know me, I just had to go all the way. So I married the system with something called uae.js. Which is essentially the JavaScript version of the Amiga emulator. Its compiled with EmScripten – a post processor that takes LLVM compiled bitcode compiled with C/C++ and spits out Asm.js optimized code.

amidesk

You just cant kill it, Amiga is 4ever

So, Smart Pascal in one hand – C/C++ in the right hand. Its like being back in college all over again. Only thing missing now is that Wacom suddenly returns and Borland rise from the grave with another Turbo product. But yes, JavaScript is something I really enjoy. And being able to compile object pascal to JavaScript is even better.

The end result? Well since I don’t have too much time on my hands it’s roughly 31-32% done, and when we hit 50% is when UAE.js will be activated. So right now its a sexy cloud front end. It has a virtual filesystem that runs fine over localstorage, but it can also talk to node.js and access the real filesystem on your server.

But when UAE.js kicks in you will be able to run your favorite Amiga demos, applications and games in your browser. I am actually very excited about seeing the performance. It runs most demos OK (using the Aros rom-files). I imagine running things like blitzbasic, Amos basic and SAS-C/C++ should work fine. Or at least be within the “usable” range if you got a powerful PC to play with.

The V8 JavaScript engine in webkit is due for an overhaul next year – and while I can only speculate I’m guessing real-life compilation will be the addition. They already do some heavy JIT’ing but once you throw LLVM based actual compilation into the picture – large JS applications is going to fly side by side with native stuff. And that’s when cloud front-ends like ChromeOS and other FriendOS is going to take off.

My little remake is not that ambitious, but I do intend to make this an absolute kick-ass system as far as Amiga is concerned. And for Smart Pascal developers? Well, lets just say that this demo project has pushed the RTL for all it’s worth and helped fix bugs and expand the RTL in a way that makes it a real power-house!

Growing up

Do we ever really grow up? I’m not sure any more. I look at others and see some that have adopted this role, this image of how an adult should be like — but its more often than not tied into the whole A4¬†family thing or some superficial work profile. And since most Amiga fanatics are in their 40’s and 50’s (same age as Delphi hooligans, Turbo was released in 1983 same year as the Amiga came out), I guess this is when kids have grown up enough for people to go “wait a minute, what .. where is my Amiga!“.

But good things come to those who wait. If someone told me that I would one day work side by side with giants like David John Pleasance, Francois Lionet and the crew at FriendUp systems – I would never have believed them. A member of quartex in meetings with the head of Commodore? My teenage self would¬†never have believed it. Both of these men, including all the tech guys at Commodore,¬†Mark Sibly the guy behind BlitzBasic — these were my teenage heroes. And now I get to work with¬†two of them. That is priceless.

As for growing up – if that means losing that spark, that trigger that when lost would render us incapable of enjoying things like the Amiga, reduced to a suit in a grey world of PCs – you know, then I’m happy to be exactly where I am. If you can go to work wearing an Amiga T-Shirt, tracker music on your iPod, a family you love at home, cool people to work with – I would call that a wrap.

And looking at the hundreds and thousands of people returning to the Amiga after 30 years in the desert – something tells me I wont be alone .. ūüėČ

 

Smart Pascal: Download streams

January 20, 2017 Leave a comment

Real, binary streams has been a part of the Smart Pascal RTL for quite some time now. As a Delphi developer you probably take that for granted, but truth be told – no other JavaScript framework even comes close to our implementation. So this is unique to Smart Pascal, believe it or not.

The same can be said about the ability to allocate, move and work with memory buffers. Sure you can write similar code by hand in pure JavaScript, but the amount of code you have to write will quickly remind you why object orientation is so important.

Binary data counts

So you got streams, what of it? I hear you say. But you are missing the point here. If there is one thing JavaScript sucks at, it’s dealing with binary data. It has no concept really of bytes versus 32 bit integers, or 64bit integers. There is no such thing as a pointer in JavaScript. So while we have enjoyed pointers, memory allocations, being able to manipulate memory directly and use streams to abstract from our binary data for decades in Delphi — all of this is brand new under JavaScript.

672274_cd11_2

And binary data counts. The moment you want to write code that does something on any substancial level – the capacity for dealing with binary data in a uniform way is imperative. If you are into HTML5 game coding you will sooner or later get in contact with map editors (or level editors) that work best in binary format. If you plan on making a sound app that runs in the cloud, again being able to read binary files (just like we do in Delphi) is really, really important. Just stop and think for a few seconds how poor Delphi and C++ builder would be without streams.

Making data available

One developer asked me an important question earlier: how do you get data out? And he meant that quite literally. What if my Smart app is the producer of binary data? What if I use SMS to create the fancy map editor, or the waveform generator code or whatever – what then?

Indeed that is a good question, but thankfully an easy one.

Most JavaScript objects, or data objects in general, inside a browser can be exported. What this means is that the browser can tag a spesific object with an ID, and then make the data available as a normal link.

For instance, if you have a block of memory like an uint8Array and you want that data exported, you would call url.createObjectURL() and it will create an URL you can use to get that data. Let’s have a look at the code you need first:

function BinaryStreamToURLObject(Stream: TStream):String;
var
  mBlob:  THandle;
begin
  if stream<>NIL then
  begin
    stream.position:=0;
    var mTemp := TDatatype.BytesToTypedArray(Stream.read(stream.size));
    asm
      var encdec = window.URL || window.webkitURL;
      @mBlob = new Blob([@mTemp],{ type: "application/octet-binary" } );
      @result = encdec.createObjectURL(@mBlob);
      console.log(@result);
    end;
  end;
end;

procedure ForceDownloadOf(FileName: string; Stream: TStream);
var
  LARef:  TControlHandle;
begin
  if Stream <> nil then
  begin
    if Stream.Size > 0 then
    begin
      // Create node
      asm
        @LARef = document.createElement('a');
      end;

      // Setup values
      LARef.style := "display: none";
      LARef.href := BinaryStreamToURLObject(Stream);
      LARef.download := Filename;

      // Add to DOM
      asm
        document.body.appendChild(@LARef);
      end;

      // Wait for the obj to appear in the DOM
      LARef.readyExecute( procedure ()
        begin
          // Invoke click on link
          LARef.click();
        end);

    end;
  end;
end;

Note #1: Notice how I use a TControlHandle in the example above. Why? Because this handle has a helper class that gives us some perks, like readyExecute(), which fires when the element is safely in the DOM and is ready to be used.

Note #2: Since the built-in browser in Smart Mobile Studio doesnt have download functionality, nothing will happen when you run this inside the IDE. So click on the “Open in browser” and run your app there to see it.

The first function takes a stream and converts it into a blob object. The second function creates an anchor object, and then calls the click() method on that anchor. Essentially kick-starting the download. It is the exact same as you clicking on a download link, except we do it purely through code.

Let’s go through the steps

  • Grab all the data from the stream
  • Convert from TByteArray to a typed browser array
  • Fetch the browser’s URL object
  • Call createObjectURL, passing the data
  • Return the internal URL for the data, which is now kept safe
  • Create an anchor link object
  • Make sure the anchor is invisible
  • Set the URL to our blob above
  • Add the anchor to the DOM
  • Call the Click() method on the anchor

Voila! Not to hard was it ūüôā

So now you can just have a button and in the onClick event you just call ForceDownload() and bob’s your uncle ūüôā

Here is the internal link I got after saving a TW3Dataset to a stream and pushing it through the steps above: blob:http%3A//192.168.38.102%3A8090/9a351a97-5f6d-4a43-b23b-c81b77972e21

This link is relative to the content, so it will only work for as long as your Smart app is in memory (actually, only while the blob is managed, you can remove the blob as well).

LDef try/catch support

January 14, 2017 1 comment

Now this was a pickle: namely to support try/catch constructs in LDEF on assembly level. It may sound simple, but it all depends on how exactly the data-model stores individual instructions.

Since LDef has various blocks where code can be defined, abstracting the instruction buffers had to be done. With blocks I naturally mean code sections. A procedure for instance is such a block. A procedure contains instructions and calls to other code blocks. But Рit can also contain sub-blocks. Consider the following:

/* block 1 */
public void SomeProc() {
  a = 12;
  b = 24;
  c = a + b;
  if (c >= 27) {
    /* Block 2 */
  } else {
    /* block 3 */
  }
}

The code above, ridicules in it’s simplicity, demonstrates a fundamental principle that all compilers must support, namely to execute different blocks based on some value. In this example block #2 will execute if “c” is more or equal to 27, or block #3 if its not.

This is pretty straight forward right? Well not quite. It all depends on how you store bytecodes in the data model. The first question you should ask is: how do we execute block #2 and not block #3. Remember that in assembly language (or bytecode) this is all one big chunk. Had this been machine code, the compiler would have to calculate the offset of block #3, also where block #3 ends. If the condition was false a jump to block #3 must be performed (skipping over block #2). Well, you get the idea I think.

LDEF was first written in Smart Pascal and runs on node.js and in vanilla html5 applications

LDEF was first written in Smart Pascal and runs on node.js and in vanilla html5 applications

Since LDef is very low-level, I have to come up with something similar. But I also wanted a solution that made things easier. Doing in-place forward calculations etc. is not hard, boring perhaps but not a showstopper by any means. But could I come up with a more flexible solution

First stop was to fragment the instruction blocks. So instead of having a single list of instructions associated with a procedure or function, these can now have as many instruction lists associated with it as memory can hold. The idea is that they are all glued together into a final list when the model is emitted to disk. But the ability to organize and work with chunks of code like this is really a step up from barebone assembly.

type
  TLDefModelParamType =
    (
    ptRegister,   // Parameter is a register
    ptVariable,   // Parameter is a variable (index follows in bytecode)
    ptConst,      // Parameter is a constant (index follows in bytecode)
    ptValue,      // Parameter is a direct value, raw data follows in bytecode
    ptDC          // Parameter is the data-control register
    );

  TLDefModelParam = class
  strict private
    FType:  TLDefModelParamType;  // Param type
    FIndex: integer;              // index (register only!)
    FData:  string;               // data (const + variable only!)
  public
    property  ParamType: TLDefModelParamType read FType write FType;
    property  Index: integer read FIndex write FIndex;
    property  Data: string read FData write FData;
  end;
  TLDefModelParamList = TObjectList;

  TLDefModelInstruction = class(TLDefModelSymbol)
  strict private
    FInstr:     integer;          // Index of instruction in dictionary
    FParams:    TLDefModelParamList;   // Parsed parameters
  public
    property    Id: integer read FInstr write FInstr;
    property    Params: TLDefModelParamList read FParams;
    constructor Create(const AParent: TParserModelObject); override;
    destructor  Destroy; override;
  end;

  TLDefModelInstructionIfThen = class(TLDefModelInstruction)
  strict private
    FThen:      TLDefModelInstructionList;
  public
    property    ThenCode: TLDefModelInstructionList read FThen;
    constructor Create(const AParent: TParserModelObject); override;
    destructor  Destroy; override;
  end;

  TLDefModelInstructionIfThenElse = class(TLDefModelInstructionIfThen)
  strict private
    FElse:      TLDefModelInstructionList;
  public
    property    ElseCode: TLDefModelInstructionList read FElse;
    constructor Create(const AParent: TParserModelObject); override;
    destructor  Destroy; override;
  end;

  TLDefModelInstructionTryCatch = class(TLDefModelInstruction)
  strict private
    FTryCode:   TLDefModelInstructionList;
    FCatchCode: TLDefModelInstructionList;
  public
    property  TryCode: TLDefModelInstructionList read FTryCode;
    property  CatchCode: TLDefModelInstructionList read FCatchCode;
    constructor Create(const AParent: TParserModelObject); override;
    destructor  Destroy; override;
  end;

  TLDefModelInstructionList = class(TLDefModelSymbol)
  strict protected
    function  GetItem(index: integer): TLDefModelInstruction;
  public
    property  Count: integer read ChildGetCount;
    property  Item[index: integer]: TLDefModelInstruction read GetItem;
    function  Add: TLDefModelInstruction; overload;
    function  Add(const NewInstance: TLDefModelInstruction): TLDefModelInstruction; overload;

    function  AddIfThen: TLDefModelInstructionIfThen;
    function  AddIfThenElse: TLDefModelInstructionIfThenElse;
    function  AddTryExcept: TLDefModelInstructionTryCatch;
  end;

  TLDefModelByteCodeChunk = class(TLDefCollectionSymbol)
  strict protected
    function  GetSegment(index: integer): TLDefModelInstructionList; virtual;
  public
    property  Count: integer read ChildGetCount;
    property  Segment[index: integer]: TLDefModelInstructionList read GetSegment;

    function  Add: TLDefModelInstructionList;
  end;

By splitting up TLDefMOdelInstructionList into these parts, especially the if/then, if/then/else and so on classes, working with conditional execution is no longer problematic. A list will always know it’s own size and length, so it’s not really that much work involved in emitting the jump instructions and test stuff.

Exceptions

Exceptions is an intricate part of the virtual machine. How to deal with them however is something I have thought long and hard about. I finally ended up with a system that is easy to use. The ES register will be 0 (zero) if no except has occured, otherwise it will contain the exception identifier.

When an exception occurs, the type and message is pushed on the stack by the virtual machine. A catch block then have to read them out and deal with them. You can also re-throw the exception via “rethrow;” or just throw a new one via “throw ”

    try {
      /* calc longs */
      move r1, count;
      mod r1, 8;
      move r2, count;
      move _longs, r1;
    } catch {
      /* The ES register contains the exception state,
         but the message will be on the stack */
      pop r0; /* get type */
      pop r1; /* get message */
      swap r0, r1; /* Syntax for showmessage wants text in r0 */
      syscall -rtl_showmessage;
    }

Well, fun times ahead! Cant wait to finish the emitters and get this puppy running ūüôā

 

LDef Intermediate Language

January 13, 2017 Leave a comment

The LDEF bytecode engine is some time away from completion, but the IL source format that the assembler reads and turns into bytecode is getting there. At the moment there are only a few tidbits left to explore, like interfaces and generics, but those will be added.

It’s a real brain teaser because – some of the stuff that makes up a compileris not really related to code. When you think about generics you often make the mistake of thinking this is a code feature, like inheritance or virtual methods; it’s something that the code-emitter has to deal with or runtime engine to take height for. But generics is actually implemented¬†higher up. It exists between the parser and code-emitter.

Interfaces is another mirage or technological illusion. When you work with classes and interfaces you get a sense that it’s a solid thing, you know – you write a class and create objects and people imagine these objects as independent, material objects in a virtual space.¬†I tend to think of instances as molecules.

But objects is ultimately an illusion. Im not going to cover the intricate details of object orientation here, but OOP is actually about separating the data (fields) from the code acting on that data (members). So the only objects that really exist in the memory of a computer when you create instances, are buffers representing the fields of your class – combined with references and a list mapping the entrypoints for the members that makes up that instance (VMT). Compilers actually mimic object orientation by adding a secret parameter to all your methods, namely the “self” parameter. This “self” is a pointer to a record that contains all the information¬†pertaining to that instance.

class_makeup

Which is really cool because then you can create as many instances as you like – and they all use the same code. Which ofcourse object orientation is all about. But there is no such thing as an independent instance floating around computer memory. That is an optical illusion of sorts.

LDEF virtual machine

Virtual machines get’s to have all the fun. I mean, writing a compiler that emits real machine code is in itself not that hard, but generating a scheme that makes object orientation work and that keeps track of everything is. The x86 cpu architecture may be powerful but it’s an absolute bitch to work with. It has few registers (compared to ARM and PPC), several instruction sets and is known to be a somewhat unfriendly place. This is the reason that compiler makers tend to stick to a golden mean of instructions. Compilers like C++ builder and Delphi could actually generate faster and more efficient code if they knew exactly what cpu and architecture you used. But since that’s not how PC’s work and there are some 20 different cpu models on the market at any given time – with huge differences between AMD and Intel, it makes sense to use a safe collection of instructions.

LDEF is a bytecode runtime engine. And one of the perks of bytecode is that it’s very much abstracted from the whole issue of real-life assembly. Most programmers think that – if you make bytecodes then you dont have to think about low level stuff, which is cheating. That may be the case for other languages and engines, but not LDEF. In fact the whole point of writing a new engine is because I wanted a bytecode format that was closer to machine code. This is very important because at some point myself or someone else will write a JIT compiler for this, and if the runtime is to high-level or abstract, that is going to be very hard.

LDEF testbed

LDEF testbed

The LDEF instruction-set represents a golden mean of the average processor instruction set. I have taken instructions that most processors have, typical stuff like add, subtract, divide, modulus, multiply (and so on). Testing is where I have given LDEF some advantages, for instance when testing a list of parameters for conditions. Instead of keeping that as two instructions (test, branch [condition]) I have isolated that in a single instruction.

Let’s look at the instruction set so far:

  • move
  • blit – move memory block
  • valloc – allocate variable
  • vfree – free variable
  • malloc – allocate memory
  • mfree – release memory
  • add
  • sub
  • mul
  • muldiv
  • div
  • mod
  • moddiv
  • lsr – logical bit-shift right
  • lsl – logical bit-shift left
  • cmp – compare
  • tst -test
  • bne – branch not equal
  • beq – branch equal
  • ble – branch less
  • bgt – branch greater
  • jsr – jump sub routine
  • jmp – jump absolute
  • push – push to stack
  • pop – pop from stack
  • IFnb – test and branch if not equal
  • IFtb – test and branch if equal
  • throw – cause exception
  • syscall – invoke engine spesific, delphi code

One of the benefits of a virtual machine, is that you can do some magic with variables. In a real machinecode compiler, variable allocation and doing read / write operations can be very complex. But in a virtual machine you thankfully get to do something about that.

So LDEF allows you to move data seamlessly between variables and registers, which really is a massive time saver. The code standard also supports two levels of resources, global and local. The latter meaning class in this case. So there is ample room for high-level languages to store their data and implement classical features (like “resourcestring” in Delphi).

You also have support for constants, these are stored separately in the bytecode and loaded into a lookup-table associated with a class. Constants are different from resources in that they are designed to be quickly referenced. Constants has more infrastructure in the form of lookup-tables Рbecause they are meant to be used with the instructions. Resources are more like resource-strings in Delphi, they are stored in their own place in the bytecode and are loaded on demand. Constants are not. They are loaded into memory and are instantly available.

Having said that, the bytecode compiler also supports in-place data. Meaning that you can chose to write constant data where they are used. So instead of having to load a string constant (for example) before working with it, you can compile the string directly into the move instruction (or any other instruction).So this is actually perfectly valid:

move r0, "Hello world";

Other virtual machines, like the Java engine, force you to do this in two steps, which is slower:

lda r0, cost_id; // reference const by id
ldc r1, (r0); // get actual value into r1

You can also assign variables directly, you dont need to load the effective address first. So there is no extract cost involved in moving data between a register and variable, variable and variable, or resource / cost to a variable. This makes life much easier:

move V[$00F6], "String assignment";
move V[$01F0], 19875.32;
move V[$196], V[$197];

Registers

Another thing that is beneficial is that LDEF has 32 registers to work with (you can actually change that, so if you need 64 that’s no problem). How you use these is up to you, but it gives a lot of room for OOP schemes. For instance, if a language has a maximum limit of 6 parameters per method – you actually dont need to pass values on the stack at all (like Java does) but you can map parameters directly to registers.

Delphi, C++ builder and other native solutions tend to use stack-schemes to store information. So a pointer to some list is stored as the fourth item on the stack, and some other entry on the third (and so on). Which is perfectly fine and very effective on a real CPU (you really have no choice on a real x86). In LDEF you can now use spesific registers instead, which is a lot easier. What scheme you chose to use if naturally up to you – but at least the option is there.

Here are the registers LDEF presently supports

  • R0 – R31: general purpose registers
  • SP: stack pointer
  • PC: program control register
  • BC: branch control register
  • ES: exception state register

Optimization

If you are wondering why a virtual machine would support so much similar stuff, constants and resources, in place data or not, this all have to do with optimalization.

For example, if your compiler decides to collect all assignment values as constants, the¬†codesize of your program might be smaller; it all depends on what your code looks like. You will naturally consolidate identical strings, integer values and so on. So even if you have a program that writes “Hello world!” 10.000 times to the console, only one “Hello world!” will be stored in the bytecode file. So constants gives you smaller code, but at a cost of speed – since constants needs a lookup whenever they are used.

Now constants here are not “cost” declarations. Constants on this level is actually the values you write in plain-text in your program, stuff like this:

FMyText := 'Welcome to my program';
FCount := 100;

Both “welcome to my program” and “100” are constants. These are values that will never change, and compilers usually have to deal with these values in one of two ways. Both of which I have explained above (in place or not).

Source format

The LDEF intermediate format looks pretty much like C++. C is fast and easier to parse than other languages and it made sense to pick that.But this similarity is paper thin, in that only the constructs of C++ is used. Methods only constains the LDEF assembly language code, and variables are isolated in Alloc sections. The virtual machine takes care of the OOP layer for you so you dont have to worry about that.

Here is an example to give you a feel for it:

#include <stdio>;
#include <oop>;

struct CustomType
{
  uint32 ctMagic;
  uint32 ctSize;
  uint8  ctData[4096];
}

resources
{
  dc.s #welcome, "welcome to LDef";
}

class TBaseObject: object
{
  /* class-fields */
  alloc {
    uint8 temp;
    uint32 counter;
    CustomType handle;
  }

  /* Parser now handles register mapping */
  public void main(r0 as count, r1 as text) {
    enter { }
    leave { }

    alloc {
      /* method variables */
      uint32 _longs;
      uint32 _singles;
    }
    /* calc longs */
    move r1, count;
    mod r1, 8;
    move r2, count;
    move _longs, r1;

    /* calc singles */
    move r3, r1;
    mul  r3, 8;
    sub  r2, r3;
    move _singles, r2
  }

  /* test multi push to stack */
  private void _cleanup() {
     push [r0, r1, r2];
  }
}

Keep in mind that the format is not 100% finished yet, there are still a few tidbits that needs to be worked out. But the general system is firmly in place.

One of the cool things is that I get to add a few missing features from my favorite language, Delphi (object pascal). If you look at the first method (main), you will notice that it has two extra sections: enter and leave.

These sections can contain code that execute directly before and after the body of the method. In pascal it would look something like this:

procedure TMyClass.SomeProc;
  before
    writeln('about to execute someproc');
  end;

  after
    writeln('Someproc finished');
  end;
begin
end;

The purpose of these, especially the before() section, is to make sure the parameters are valid. So before is used to check that the parameters are within a legal range. After is naturally used to ensure that the result (for functions) is valid.

It also opens up for some interesting logging and debugging aspects.

More to come

This is turning into a long rant, but hopefully you found it interesting. I’ll keep you posted as the engine progress. There is still some way to go, but we are getting there. Once LDEF is implemented the fun starts, thats when I’ll code high-level parsers that targets the engine. First stop is naturally and object pascal compiler ūüôā

Delphi bytecode compiler

January 10, 2017 1 comment

This is a pet project I have been playing with on/off for a couple of years now. Since I’m very busy with Smart Mobile Studio these days I havent really had time to write much about this in a while. Well, today I needed a break for JavaScript – and what is more fun than relaxing with a cup of coco and bytecodes?

The LDEF bytecode standard

Do you remember Quartex Pascal? It was an alternative pascal compiler I wrote a couple of years back. The plan was initially to get Smart Mobile Studio into that codebase, and essentially kill 3 birds with one stone. I already have parsers for various languages that I have written: Visual Basic .NET, Typescript, C# and C++. Adding support for Smart Pascal to that library of parsers would naturally give us a huge advantage.

In essence you would just pick what language you wanted to write your Smart apps in, then use the same classes and RTL across the board. You could also mix and match, perhaps write some in typescript (which is handy when you face a class that is difficult to “pascalify” via the import tool).

LDEF test environment

LDEF test environment

But parsing code and building an AST (abstract symbol tree, this is a hierarchy of objects the parser builds from your source-code. Basically your program in object form) is only half the job. People often start coding languages and script engines without really thinking it through (I have done that myself a few times. It’s a great way to learn the ropes, but you waste a lot of time), after a while they give up because it dawns on them that the hard part is still ahead: namely to generate executable code or at least run the code straight off the AST. There is a reason components like dwScript have taken years to polish and perfect (same can be said of PAX script which is probably one of the most powerful engines available to Delphi developers).

The idea behind Quartex Pascal was naturally that you could have more than one language, but they would all build the same AST and all of them emit the same bytecode. Not exactly a novel idea, this is how most scripting engines work – and also what Java and .Net have been doing since day one.

But the hard part was still waiting for me, namely to generate bytecode. This may sound easy but it’s not just a matter of dumping out a longword with the index of a method. You really need to think it through because if you make it to high-level, it will cripple your language. If you make it to low-level – the code will execute slow and the amount of bytecodes generated can become huge compared to normal assembly-code.

And no, I’m not making it compatible with CIL or Java, this is a pure object pascal solution for object pascal developers (and C++ builder naturally).

LDEF source language

I decided to do something a bit novel. Rather than just creating some classes to help you generate bytecode, I decided to create a language around the assembly language. Since C++ is easy to parse and looks that way because of its close connection to assembler, I decided to use C++ as the basic dialect.

So instead of you emitting bytecodes, your program only needs to emit LDEF source-code. Then you can just call the assembler program (command line) and it will parse, compile and turn it into assemblies for you.

Here is an example snippet that compiles without problem:

struct CustomType
{
  uint32 ctMagic;
  uint32 ctSize;
  uint8  ctData[4096];
}

class TBaseObject: object
{
  /* class-fields */
  alloc {
    uint8 temp;
    uint32 counter;
  }

  /* Parser now handles register mapping */
  public void main(r0 as count, r1 as text) {
    alloc {
      /* method variables */
      uint32 _longs;
      uint32 _singles;
    }
    move r1, count;
    jsr @_cleanup;
    push [text];
  }

  /* test multi push to stack */
  private void _cleanup() {
     push [r0, r1, r2];
  }
}

The LDEF virtual machine takes care of things like instance management, VMT (virtual method table), inheritance and everything else. All you have to do is to generate the LDEF assembly code that goes into the methods Рand voila, the assembler will parse, compile and emit the whole shabam as assemblies (the -L option in the command-line assembler controls how you want to link the assemblies, so you can emit a single file).

The runtime engine is written in vanilla object pascal. It uses generics and lookup tables to achieve pretty good performance, and there is ample room for a JIT engine in the future. What was important for me was that i had a solution that was portable, require little maintenance, with an instruction set that could easily be converted to actual machine-code, LLVM or another high level language (like C++ that can piggyback on GCC regardless of platform).

LDEF instructions

The instruction set is a good mean of the most common instructions that real processors have. Conceptually the VM is inspired by the Motorola 68020 chip, in combination with the good old Acord Risc cpu. Some of the features are:

  • 32bit (4 byte) opcode size
  • 32 registers
  • 1024 byte cache (can be adjusted)
  • Stack
  • Built in variable management
  • Built in const and resource management
  • Move data seamlessly between registers and variables
  • Support for records (struct) and class-types
  • Support for source-maps (both from assembler to high-level and reverse)
  • Component based, languages and emitters can be changed

There are also a few neat language features that I have been missing from Delphi, like code criteria. Basically it allows you to define code that should be executed before the body of a procedure, and directly after. This allows you to check that parameter values are within range or valid before the procedure is allowed to execute.

Here is the pascal version:

function TMyClass.CalcValues(a,b,c: integer): integer;
begin
  before
    if (a<1) or (b<1) or (c<1) then
      Fail('Invalid values, %classname.%methodname never executed');
  end;

  result := a + b div c;

  after
    if result <23 then
      Fail('%classname.%methodname failed, input below lowest threshold error');
  end;
end;

I decided to add support for this in LDEF itself, including the C++ style intermediate language. Here it looks like this:

  /* Enter + Exit is directly supported */
  public void main(r0 as count, r1 as text) {
    enter {
    }

    leave {
    }

    alloc {
      /* method variables */
      uint32 _longs;
      uint32 _singles;
    }
    move r1, count;
    jsr @_cleanup;
    push [text];
  }

Why Borland/Embarcadero didn’t add this when they gave the compiler a full overhaul and support for generics – is beyond me. C++ have had this for many, many years now. C# have supported it since version 4, and Java I am usure about – but its been there for many years, thats for sure.

Attributes and RTTI

Attributes will be added but support for user-defined attributes will only appear later. Right now the only attributes I have plans for controls how memory is allocated for variables, if registers should be pushed to the stack on entry automatically, if the VMT should be flattened and the inheritance-chain reduced to a single table. More attributes will no doubt appear as I move forward, but right now I’ll stick to the basics.

RTTI is fairly good and presently linked into every assembly. You cant really omit that much from a bytecode system. To reduce the use of pure variables I introduced register mapping to make it easier for people to use the registers for as much as possible (much faster than variables):

  public void main(r0 as count, r1 as text) {
  }

You may have noticed the strange parameters on this method? Thats because it’s not parameters, but definitions that link registers to names (register mapping). That way you can write code that uses the original names of variables or parameters, and avoid allocating variables for everything. It has no effect except making it easier to write code.

LDEF, why should I care?

You should care because, with a bit of work we should be able to compile fairly modern Delphi source-code to portable bytecodes. The runtime or engine that executes these bytecodes can be compiled using freepascal, which means you can execute the program anywhere FPC is supported. So more or less every operative system on the planet.

You should care because you can now write programs for Ultibo, the pascal operative system for Raspberry PI. It can only execute 1 real program (your embedded program), but since you can now run bytecodes, you will be able to run as many programs as you like. Just run each bytecode program in a thread or play it safe and call next() on interval from a TTimer.

You should care because once LDEF is finished, being able to transcode from object pascal to virtually any language will be a lot easier. JavaScript? C++? Python? Take your pick. The point here is standards and a library that is written to last decades rather than years. So my code is very clean and no silly pointer tricks just to get a few extra cycles. Stability, portability and maintainance are the values here.

You should care because in the future, components like HexLicense will implement its security code in LDEF using a randomized instruction-set, making it very, very hard for hackers to break your product.

Ghost of xmas past, or better known as the folder where old projects and abandoned ideas end up

December 8, 2016 Leave a comment

I think everyone has a folder where they stuff old units, test projects and well – all those ideas that seemed awesome at the time, but you either discovered that it was crap, not enough time, or just forgot all about it. It happens. What can I say.

Yeah, I have one of those as well. And I also have those duplicate folders. You know, that time when you decided to make backups for everything Рbut then you got a new PC and now you dont have a clue which version to keep? And when you finally sit down with beyondcompare to get this sorted, you have managed to add to both copies.

Well, needless to say I had a day I just went apeshit about a year ago and sorted everything. And whatever did not fit into my neat new code-folder(s) was mercilessly stuffed into the dark, obscure folder known only as “stuff” or “misc”.

My first basic compiler, awwwww how cute

Well its not all rubbish! I did find a special version of my kitchen-sink research IDE, the IDE I use when trying out new code for DWScript, PaxCompiler and FreePascal. I think it has 4 compilers in it or something, yet its under 30 megabytes in size! Pretty neat ūüôā

Visual Basic to Smart Pascal, awww.... just want to snuggle it :)

Visual Basic to Smart Pascal, awww…. just want to snuggle it ūüôā

It also features a unified file-system! You can mount packages, FTP folders, network paths or local folders – the IDE could not care less, it is completely abstracted from the actual filesystem and relates only to symbols and identifiers. The actual storage is dealt with by the filesource classes.

Package, ftp, socket, local folder - the IDE could not care less

Package, ftp, socket, local folder – the IDE could not care less

The Basic dialect here is essentially classical Visual Basic. The way you would write classes and deal with instances before dot net came along and forced all languages to look the same. I kinda like old visual basic, it had soul. It was completely useless except for scripting, and Delphi made it look like a joke – but as far as basic dialects go, it’s probably one of the cleanest and easiest to read.

DWScript, QTX Pascal (a custom fork of DWScript with a lot of cool features, some that Eric has added afterwards), Basic - and even freepascal! Thats the mother of all kitchen sinks!

DWScript, QTX Pascal (a custom fork of DWScript with a lot of cool features, some that Eric has added afterwards), Basic – and even freepascal! Thats the mother of all kitchen sinks!

The loss of QTX Pascal is a bit sad. I spent a couple of months doing my own fork of DWScript. Most of the features I added have now been included (although in another form) by Eric. But the codegen produced faster javascript. One of the things I spent some time on was optimization. Like getting rid of “variable = 0” type initialization if an assignment followed. I also added BeforeDestruction() and AfterConstruction() calls. This made an RTL a lot easier to write, but also more overhead. I was about to do conditional testing (so these would only be called if you actually used them) when I had to stop and work on Smart Mobile Studio again.

N++, exotic and functional!

This was one of my favorite language research projects. And dare I say, it is way ahead of it’s time. The idea is simple: with the future of computing being distributed, cloud based and powered by a multitude of computing modules from the US to South Africa¬†— what would a programming language look like if built for 100% asyncronous, distributed execution?

Consider for instance, the execution of a procedure. Simple right? Well, in a truly distributed system – that procedure could execute anywhere. The whole point here is to utilize the combined power of each module (pc); which means a tasks could execute on different computers. A procedure could also be split up by the runtime environment and once again, be executed all over the place – with the dispatcher keeping track of when it’s done and whatever the result was (if any).

Only a spesific and generated on-demand “runtime context” would follow the task. This would contain things like variables it needs to execute, public symbols, basicaly just enough for the code to run successfully. This also includes resource forks (a term i used for accessing resources regardless of where they may be on the network).

The language design was actually done using Smart Mobile Studio (you can read more about the initial ideas here), and the first test modules ran on node.js (so no fancy graphics available sadly).

But it was incredibly fun to play with! But also time consuming to design a language for an execution model that havent really been invented yet. You can read more about some of my ideas for the execution model here.

I dont even think this execution model is out of the MIT labs yet — but it will once quantum compute models become commercially available (click here for a Berkley University¬†published introduction to the basic computational models used in quantum information theory).

An N++ procedure (actually a microservice that consumed a WSDL service and calls it) looks like this:

program("service_test") {

  handshake {

    input {
      /* Consume WSDL Web-Service Endpoint */
      service1 @ service[URI:"http://www.test.com/SOAP/myService/WSDL/",
                 serviceType:SoapService];
    }

    output {
        myProcess @ process("self");
        stdio @ pipe("stdout");
      }
  }

  /* Execute RPC call */
  execute (stdio,service1)  {
    stdio:writeln("Calling webservice");
    execute (*)  {
      var int32 result = 0;
      set result = service1:getUserId("quartex","secret");
      stdio:writelnF("UserID on server={0}", result);
    } fail (e) {
      stdio.writelnF("Calling soap service failed: {0}",e);
      proceed;
    }
  }

  /* Exit code for process */
  set myProcess:exitCode = 0;
}

RML, Reduced markup language parser and runtime

Dont you just hate it when you lose the source-file for a programming language you made in a day, so now you dont have a clue what the syntax really was? I know, I know, but I honestly dont have the syntax or examples. It does with a disk ages ago.

RML was written to simplify generating HTML documents. There are a few things that you can optimize when it comes to HTML. First of all you know that the browser will read the document from top and downwards. This might not mean much at first glance, but it actually means that a language doesnt need the abillity to call emitter-code that has already executed (above). Think of it as being able to omit calls to procedure behind you in the source. That would be a mess for a language like delphi – but it doesnt impact a language that execute linear without any forms of jumps!

I seem to remember it looked something like this:

createobject(TRMLObject)
{
   writestr("testing");
   if(a=12)
   {
       readstr("mer testing");
   }
}
createobject(TRMLHeading)
{
   settitle("testing second object");
   if(a=12)
   {
       writestr("heading object successfully done");
   }
}
createobject(TRMLObject)
{
   writestr("testing third object");
   if(a=12)
   {
       readstr("mer testing from da third object");
   }
}

The idea is ofcourse that you have some pre-defined objects, a bit like custom-controls in Smart Pascal, and you create and populate these in a top-down fashion.
Once defined, generating quite advanced HTML documents can be done extremely fast. Much faster than ASP, DWS or php since no jumps are involved.

Here is how you compile and run a small debug session (drop a memo and button on a form first):

  var
    FContext: TRMLParseContext;
    FText:  String;
  begin
    FText:= 'createobject(TRMLObject)'
    +       '{'
    +       '   writestr("testing");'
    +       '   if(a=12)'
    +       '   {'
    +       '       readstr("mer testing");'
    +       '   }'
    +       '}'
    +       'createobject(TRMLHeading)'
    +       '{'
    +       '   settitle("testing second object");'
    +       '   if(a=12)'
    +       '   {'
    +       '       writestr("heading object successfully done");'
    +       '   }'
    +       '}'
    +       'createobject(TRMLObject)'
    +       '{'
    +       '   writestr("testing third object");'
    +       '   if(a=12)'
    +       '   {'
    +       '       readstr("mer testing from da third object");'
    +       '   }'
    +       '}';

    // double the code, just to get some payload
    FText := FText + FText;

    memo1.text:='';

    If RMLAllocContext(FContext,FText,[TRMLObject,TRMLHeading]) then
    Begin
      try
        if RMLCompile(FContext) then
        Begin
          caption:='Compilation was a success';
          If RMLExecute(FContext) then
          Begin
            Caption:='Executed successfully';
            memo1.text:=FContext.pcOutput;
          end;
        end else
        caption:=Format('compilation failed [%s]',[FContext.pcError]);
      finally
        RMLDisposeContext(FContext);
      end;
    end;

And here is the full code for the parser and runtime (still a few tidbits to work on, but its childs play). If you can make sense of it, knock yourself out ūüôā

  unit rml;

  interface

  uses sysutils, classes, dialogs;

  type

  TRMLObject  = Class;

  TRMLClass     = Class of TRMLObject;
  TRMLOperator  = (opLess,opMore,opEquals,opNotEquals);
  TRMLDataType  = (daString,daNumber,daBoolean,daCustom);
  TRMLEntryType = (etAssignment,etMethod,etRepeat,etCreate,etIf);
  TRMLObjProc   = Function (Const EntryData:Pointer):Boolean of Object;

  (* represents a single condition in an IF statement *)
  PRMLCondition = ^TRMLCondition;
  TRMLCondition = Packed Record
    coSource:   String;
    coTarget:   String;
    coOperator: TRMLOperator;
  End;

  (* represents a parsed code declaration *)
  PRMLEntryDeclaration = ^TRMLEntryDeclaration;
  TRMLEntryDeclaration = Packed Record
    cxToken:      String;
    cxCondition:  String;
  End;

  (* represents a compiled parameter *)
  PRMLParameter = ^TRMLParameter;
  TRMLParameter = Packed Record
    prType:   TRMLDataType;
    prValue:  Pointer;
    prSize:   Integer;
  End;

  (* represents a compiled code entry *)
  PRMLEntryData = ^TRMLEntryData;
  TRMLEntryData = Packed Record
    edtype:         TRMLEntryType;
    edDeclaration:  TRMLEntryDeclaration;
    edObject:       TRMLObject;
    edParent:       PRMLEntryData;
    edMethod:       TRMLObjProc;
    edContext:      Pointer;
    edConditions:   Array of PRMLCondition;
    edParameters:   Array of PRMLParameter;
    edSubEntries:   Array of PRMLEntryData;
  End;

  PRMLParseContext = ^TRMLParseContext;
  TRMLParseContext = Packed Record
    pcSignature:  Integer;
    pcCol:        Integer;
    pcRow:        Integer;
    pcPos:        Integer;
    pcLen:        Integer;
    pcData:       String;
    pcError:      String;
    pcOutput:     String;
    pcRoot:       TRMLEntryData;
    pcClasses:    Array of TRMLClass;
  End;

  TRMLReadProc  = Function  (Var OutData;var Bytes:Integer):Boolean of Object;
  TRMLWriteProc = Function  (Var InData;Const Bytes:Integer):Boolean of Object;
  TRMLProcEntry = Function  (Const Entry:PRMLEntryData):Boolean of Object;

  PRMLObjectIndex = ^TRMLObjectIndex;
  TRMLObjectIndex = Packed Record
    oiMethods:    Array of record
                    omName:   String;
                    omSyntax: Array of TRMLDataType;
                    omEntry:  TRMLProcEntry;
                  end;
    oiProperties: Array of record
                    oiName:   String;
                    oiRead:   TRMLReadProc;
                    oiWrite:  TRMLWriteProc;
                    oiType:   TRMLDataType;
                  end;
  End;

  TRMLObject = Class(TObject)
  Private
    FIndexData: TRMLObjectIndex;
    FIndexPTR:  PRMLObjectIndex;
  Private
    Function    DoWriteStr(Const Entry:PRMLEntryData):Boolean;
    Function    DoReadStr(Const Entry:PRMLEntryData):Boolean;
  protected
    Procedure   Output(Const Context:PRMLParseContext;Const Value:String);
    Procedure   RegisterProperty(Const Name:String;Const DataType:TRMLDataType;
                Const _Read:TRMLReadProc;Const _Write:TRMLWriteProc);
    Procedure   RegisterMethod(Const Name:String;
                Const Syntax: Array of TRMLDataType;
                Const Entry: TRMLProcEntry);
  Public
    Property    ObjectIndex:PRMLObjectIndex read FIndexPTR;
    Constructor Create;virtual;
  End;

  TRMLHeading = Class(TRMLObject)
  Private
    FTitle:     String;
    Function    DoSetTitle(Const Entry:PRMLEntryData):Boolean;
    Function    DoReadTitle(Var OutData;var Bytes:Integer):Boolean;
    Function    DoWriteTitle(Var InData;Const Bytes:Integer):Boolean;
  Public
    property    Title:String read FTitle write FTitle;
    Constructor Create;override;
  End;

  Function  RMLAllocContext(var Context:TRMLParseContext;
            Const Source:String;Const ClsBase:Array of TRMLClass):Boolean;
  Function  RMLDisposeContext(Var Context:TRMLParseContext):Boolean;
  Function  RMLCompile(Var Context:TRMLParseContext):Boolean;
  Function  RMLExecute(Const Context:TRMLParseContext):Boolean;
  Function  RMLParseEntry(Value:String;
            var Declaration:TRMLEntryDeclaration;var Error:String):Boolean;

  implementation

  //###########################################################################
  // TRMLHeading
  //###########################################################################

  Constructor TRMLHeading.Create;
  Begin
    inherited;
    RegisterMethod('settitle',[daString],DoSetTitle);
    RegisterProperty('title',daString,DoReadTitle,DoWriteTitle);
  end;

  Function TRMLHeading.DoSetTitle(Const Entry:PRMLEntryData):Boolean;
  var
    FTemp:  String;
  Begin
    result:=length(Entry^.edParameters)>0;
    If result and (Entry^.edParameters[0].prType=daString) then
    DoWriteTitle(Entry^.edParameters[0].prValue^,
    Entry^.edParameters[0].prSize);
  end;

  Function TRMLHeading.DoReadTitle(Var OutData;var Bytes:Integer):Boolean;
  Begin
    Bytes:=Length(FTitle);
    If Bytes>0 then
    move(FTitle[1],outData,Bytes);
  end;

  Function TRMLHeading.DoWriteTitle(Var InData;Const Bytes:Integer):Boolean;
  Begin
    SetLength(FTitle,bytes);
    if Bytes>0 then
    move(inData,FTitle[1],Bytes);
  end;

  //###########################################################################
  // TRMLObject
  //###########################################################################

  Constructor TRMLObject.Create;
  begin
    inherited;
    FIndexPTR:=@FIndexData;
    RegisterMethod('writestr',[daString],DoWriteStr);
    RegisterMethod('readstr',[daString],DoReadStr);
  end;

  Procedure TRMLObject.Output(Const Context:PRMLParseContext;
            Const Value:String);
  Begin
    Context^.pcOutput:=Context^.pcOutput + Value;
  end;

  Function TRMLObject.DoReadStr(Const Entry:PRMLEntryData):Boolean;
  var
    FText:  String;
  Begin
    result:=True;
    With Entry^ do
    Begin
      FText:='Token: ' + edDeclaration.cxToken + #13#10;
      FText:=FText + 'Condition: ' + edDeclaration.cxCondition + #13#10;
      FText:=FText + #13#10;
      Output(edContext,FText);
    end;
  end;

  Function TRMLObject.DoWriteStr(Const Entry:PRMLEntryData):Boolean;
  var
    FText:  String;
  Begin
    result:=True;
    With Entry^ do
    Begin
      FText:='Token: ' + edDeclaration.cxToken + #13#10;
      FText:=FText + 'Condition: ' + edDeclaration.cxCondition + #13#10;
      FText:=FText + #13#10;
      Output(edContext,FText);
    end;
  end;

  Procedure   TRMLObject.RegisterProperty(Const Name:String;
              Const DataType:TRMLDataType;
              Const _Read:TRMLReadProc;Const _Write:TRMLWriteProc);
  var
    FCount: Integer;
  Begin
    //FCount:=high(FIndexData.oiProperties) - Low(FIndexData.oiProperties) + 1;
    FCount:=Length(FIndexData.oiProperties);
    SetLength(FIndexData.oiProperties,FCount+1);
    FIndexData.oiProperties[FCount].oiName:=Name;
    FIndexData.oiProperties[FCount].oiRead:=_Read;
    FIndexData.oiProperties[FCount].oiWrite:=_Write;
    FIndexData.oiProperties[FCount].oiType:=DataType;
  end;

  Procedure   TRMLObject.RegisterMethod(Const Name:String;
              Const Syntax: Array of TRMLDataType;
              Const Entry: TRMLProcEntry);
  var
    FCount: Integer;
    FTemp:  Integer;
  Begin
    //FCount:=high(FIndexData.oiMethods) - Low(FIndexData.oiMethods) + 1;
    FCount:=Length(FIndexData.oiMethods);
    SetLength(FIndexData.oiMethods,FCount+1);
    FIndexData.oiMethods[FCount].omName:=Name;
    FIndexData.oiMethods[FCount].omEntry:=Entry;

    //FTemp:=high(Syntax) - Low(Syntax) + 1;
    FTemp:=Length(Syntax);
    If FTemp>0 then
    Begin
      SetLength(FIndexData.oiMethods[FCount].omSyntax,FTemp);
      for FTemp:=low(syntax) to high(syntax) do
      FIndexData.oiMethods[FCount].omSyntax[FTemp]:=Syntax[FTemp];
    end;
  end;

  //###########################################################################
  // RML util methods
  //###########################################################################

  Function RMLContainChars(Const Value:String;const Chars:String):Boolean;
  var
    x:  Integer;
  Begin
    result:=True;
    for x:=1 to length(chars) do
    Begin
      if pos(chars[x],Value)<1 then
      Begin
        result:=False;
        Break;
      end;
    end;
  end;

  Function  RMLScanFor(const Value:String;Const Target:CHAR;
            Const Breakers:String;var Len:Integer):Boolean;
  var
    xpos: Integer;
  Begin
    result:=False;
    Len:=-1;
    xpos:=1;
    while xpos<=Length(Value) do     Begin       If Value[xpos]=Target then       Begin         Len:=xpos-1;         Result:=True;         Break;       end else       Begin         if pos(Value[xpos],Breakers)>0 then
        Break;
      end;
      inc(xpos);
    end;
  end;

  Function RMLisNumber(Const Value:String):Boolean;
  const
    CHARSET = '0123456789';
  var
    x:  Integer;
  Begin
    Result:=True;
    for x:=1 to length(Value) do
    Begin
      if pos(Value[x],CHARSET)<1 then
      Begin
        result:=False;
        Break;
      end;
    end;
  end;

  Function RMLIsBoolean(Const Value:String):Boolean;
  var
    FTemp:  String;
  Begin
    FTemp:=lowercase(trim(Value));
    result:=(FTemp='false') or (FTemp='true');
  end;

  Function RMLisString(Const Value:String):Boolean;
  var
    x:      Integer;
    FLeft:  Integer;
  Begin
    result:=False;
    FLeft:=0;
    (* check left side *)
    for x:=1 to length(Value) do
    Begin
      if Value[x]='"' then
      Begin
        FLeft:=x;
        Result:=True;
        Break;
      end else
      if Value[x]<>#32 then
      Break;
    end;
    (* check right side *)
    If result then
    Begin
      for x:=Length(Value) downto 1 do
      Begin
        if Value[x]='"' then
        Begin
          If x>FLeft then
          Break else
          Begin
            Result:=False;
            Break;
          end;
        end else
        if Value[x]<>#32 then
        Break;
      end;
    end;
  end;

  Function  RMLParseEntry(Value:String;
            var Declaration:TRMLEntryDeclaration;
            var Error:String):Boolean;
  var
    xpos: Integer;
  Begin
    fillchar(Declaration,SizeOf(Declaration),0);
    Result:=RMLContainChars(value,'()');
    if Result then
    Begin
      Result:=RMLScanFor(value,'(',')',xpos);
      if result then
      Begin
        Declaration.cxToken:=trim(copy(value,1,xpos));
        delete(value,1,xpos+1);
        Result:=RMLScanFor(value,')','(',xpos);
        if result then
        Begin
          Value:=TrimRight(Value);
          Result:=xpos=(length(value)-1);
          if result then
          Declaration.cxCondition:=trim(Copy(Value,1,xpos));
        end;
      end;
    end;
    If not Result then
    Error:='Invalid entry <' + value + '>';
  end;

  Function  RMLAllocContext(var Context:TRMLParseContext;
            Const Source:String;Const ClsBase:Array of TRMLClass):Boolean;
  var
    FCount: Integer;
  Begin
    If Context.pcSignature=SizeOf(Context) then
    RMLDisposeContext(Context);

    fillchar(Context,SizeOf(Context),#0);
    Context.pcSignature:=SizeOf(Context);
    Context.pcLen:=Length(Source);
    Context.pcData:=Source;

    FCount:=High(clsBase) - low(clsBase)+1;
    If FCount>0 then
    Begin
      SetLength(Context.pcClasses,FCount);
      for FCount:=Low(clsBase) to high(clsBase) do
      Context.pcClasses[FCount]:=clsBase[FCount];
    end;

    result:=True;
  end;

  Procedure RMLDisposeEntryData(Const Data:PRMLEntryData);
  var
    FTemp:  Integer;
    x:      Integer;
  Begin
    (* dispose of condition data *)
    FTemp:=length(Data^.edConditions);
    While FTemp>0 do
    Begin
      Dispose(Data^.edConditions[FTemp-1]);
      dec(FTemp);
    end;
    SetLength(Data^.edConditions,0);

    (* dispose of parameter data *)
    FTemp:=length(Data^.edParameters);
    While FTemp>0 do
    Begin
      If length(Data^.edParameters)>0 then
      Begin
        for x:=Low(Data^.edParameters) to high(Data^.edParameters) do
        If Data^.edParameters[x]^.prSize>0 then
        FreeMem(Data^.edParameters[x]^.prValue);
      end;
      Dispose(Data^.edParameters[FTemp-1]);
      dec(FTemp);
    end;
    SetLength(Data^.edParameters,0);

    (* dispose of sub entries *)
    //Ftemp:=High(Data^.edSubEntries)-Low(Data^.edSubEntries)+1;
    FTemp:=Length(Data^.edSubEntries);
    While FTemp>0 do
    Begin
      RMLDisposeEntryData(Data^.edSubEntries[FTemp-1]);
      Dec(FTemp);
    end;
    SetLength(Data^.edSubEntries,0);

    (* dispose of script object *)
    If  not (Data^.edtype in [etIf,etRepeat,etMethod])
    and (Data^.edObject<>NIL) then
    Data^.edObject.free;

    (* dispose of entry *)
    Dispose(Data);
  end;

  Function  RMLDisposeContext(Var Context:TRMLParseContext):Boolean;
  var
    FCount: Integer;
  Begin
    Result:=Context.pcSignature=SizeOf(Context);
    If Result then
    Begin
      Context.pcSignature:=0;
      Context.pcData:='';
      Context.pcError:='';
      FCount:=Length(Context.pcRoot.edSubEntries);
      //FCount:=High(Context.pcRoot.edSubEntries)
      //- Low(Context.pcRoot.edSubEntries) + 1;
      While FCount>0 do
      Begin
        RMLDisposeEntryData(Context.pcRoot.edSubEntries[FCount-1]);
        dec(FCount);
      end;
      SetLength(Context.pcRoot.edSubEntries,0);
    end;
  end;

  //###########################################################################
  // RML core methods
  //###########################################################################

  Function  RMLImplements(Const MethodName:String;
            Const obj:TRMLObject):Boolean;
  var
    FTable: PRMLObjectIndex;
    FCount: Integer;
  Begin
    Result:=Obj<>NIL;
    If result then
    Begin
      (* get object inex *)
      FTable:=Obj.ObjectIndex;
      //FCount:=High(FTable^.oiMethods) - low(FTable^.oiMethods) + 1;
      FCount:=Length(FTable^.oiMethods);
      Result:=FCount>0;
      If Result then
      Begin
        for FCount:=low(FTable^.oiMethods) to high(FTable^.oiMethods) do
        Begin
          Result:=FTable^.oiMethods[FCount].omName=MethodName;
          If Result then
          Break;
        end;
      end;
    end;
  end;

  Function  RMLGetMethodEntry(Const MethodName:String;
            Const obj:TRMLObject;var outEntry:TRMLObjProc):Boolean;
  var
    FTable: PRMLObjectIndex;
    FCount: Integer;
  Begin
    Result:=Obj<>NIL;
    If result then
    Begin
      (* get object inex *)
      FTable:=Obj.ObjectIndex;
      //FCount:=High(FTable^.oiMethods) - low(FTable^.oiMethods) + 1;
      FCount:=Length(FTable^.oiMethods);
      Result:=FCount>0;
      If Result then
      Begin
        for FCount:=low(FTable^.oiMethods) to high(FTable^.oiMethods) do
        Begin
          Result:=FTable^.oiMethods[FCount].omName=MethodName;
          If Result then
          Begin
            outEntry:=TRMLObjProc(FTable^.oiMethods[FCount].omEntry);
            Break;
          end;
        end;
      end;
    end;
  end;

  Function  RMLCreateObject(var Context:TRMLParseContext;
            Const Objname:String;var outObject:TRMLObject;
            var Error:String):Boolean;
  var
    FCount: Integer;
  Begin
    FCount:=High(Context.pcClasses) - Low(Context.pcClasses) + 1;
    Result:=FCount>0;
    if Result then
    Begin
      For FCount:=Low(Context.pcClasses) to high(Context.pcClasses) do
      Begin
        Result:=lowercase(Context.pcClasses[FCount].ClassName)=lowercase(objName);
        If result then
        Begin
          outObject:=Context.pcClasses[FCount].Create;
          Break;
        end;
      end;
    end;
    If not Result then
    Error:='Unknown class <' + Objname + '>';
  end;

  Function  RMLAddEntry(var Context:TRMLParseContext;
            Var Declaration:TRMLEntryDeclaration;
            Root:PRMLEntryData;var NewEntry:PRMLEntryData;
            var Error:String):Boolean;
  var
    FCount: Integer;
    x:      Integer;
    FTemp:  String;
    FLen:   Integer;
    FPar:   PRMLParameter;
  Begin
    Result:=Root<>NIL;
    If result then
    Begin
      (* create new entry *)
      new(NewEntry);

      (* Reset entry record *)
      NewEntry^.edType:=etAssignment;
      NewEntry^.edObject:=NIL;
      NewEntry^.edMethod:=NIL;
      SetLength(NewEntry^.edConditions,0);
      SetLength(NewEntry^.edParameters,0);
      SetLength(NewEntry^.edSubEntries,0);

      (* Set basic values *)
      NewEntry^.edParent:=Root;
      NewEntry^.edContext:=@Context;
      newEntry^.edDeclaration:=Declaration;

      (* insert entry into parent *)
      FCount:=Length(Root^.edSubEntries);
      SetLength(Root^.edSubEntries,FCount+1);
      Root^.edSubEntries[FCount]:=NewEntry;

      (* tokenize *)
      If declaration.cxToken='createobject' then
      Begin
        NewEntry^.edtype:=etCreate;
        Result:=RMLCreateObject
          (
          Context,declaration.cxCondition,
          NewEntry^.edObject,Error
          );
      end else

      if declaration.cxToken='if' then
      Begin
        NewEntry^.edtype:=etIF;
        NewEntry^.edObject:=NewEntry^.edParent.edObject;
      end else

      if declaration.cxToken='repeat' then
      NewEntry^.edtype:=etRepeat else

      Begin
        (* method call? Make sure entry object supports this *)
        Result:=NewEntry^.edParent.edObject<>NIL;
        If Result then
        Begin
          (* check if object supports the method name *)
          Result:=RMLImplements(declaration.cxToken,NewEntry^.edParent.edObject);
          If Result then
          Begin
            (* Query object for method entry *)
            Result:=RMLGetMethodEntry
              (
              declaration.cxToken,
              NewEntry^.edParent.edObject,
              NewEntry^.edMethod
              );

            If result then
            Begin
              NewEntry^.edtype:=etMethod;
              NewEntry^.edObject:=NewEntry^.edParent.edObject;

              (* now parse the parameter conditions *)
              x:=0;
              While x<Length(declaration.cxCondition) do               Begin                 inc(x);                 If (declaration.cxCondition[x]=',')                 or (x=Length(declaration.cxCondition)) then                 Begin                   If x=Length(declaration.cxCondition) then                   FTemp:=FTemp + declaration.cxCondition[x];                   FTemp:=trim(FTemp);                   If length(FTemp)>0 then
                  Begin
                    (* create a new parameter *)
                    FLen:=length(NewEntry^.edParameters);
                    setlength(NewEntry^.edParameters,FLen+1);

                    New(FPar);

                    If RMLIsString(FTemp) then
                    FPar^.prType:=daString else
                    if RMLIsNumber(FTemp) then
                    FPar^.prType:=daNumber else
                    if RMLIsBoolean(FTemp) then
                    FPar^.prType:=daBoolean else
                    FPar^.prType:=daCustom;

                    Case FPar^.prType of
                    daString:
                      Begin
                        Delete(FTemp,1,1);
                        Delete(FTemp,length(FTemp),1);
                        FPar^.prSize:=Length(FTemp);
                        FPar^.prValue:=AllocMem(FPar^.prSize);
                        move(FTemp[1],FPar^.prValue^,FPar^.prSize);
                      end;
                    daNumber:
                      Begin
                        FPar^.prSize:=SizeOf(Integer);
                        FPar^.prValue:=AllocMem(FPar^.prSize);
                        PInteger(FPar^.prValue)^:=StrToInt(FTemp);
                      end;
                    daBoolean:
                      Begin
                      end;
                    daCustom:
                      Begin
                      end;
                    end;

                    NewEntry^.edParameters[FLen]:=FPar;
                    FTemp:='';
                  end else
                  Begin
                    //Invalid parameter error
                  end;
                end else
                FTemp:=FTemp + declaration.cxCondition[x]
              end;

              {
              Validate parameter datatypes here!
              If  (Length(NewEntry^.edParameters)>0) then
              Begin
                for x:=Low(NewEntry^.edParameters) to
                high(NewEntry^.edParameters) do
                Begin
                  newEntry^.edObject.
                end;
              end;  }

            end;

          end else
          Begin
            // property assignment test here
          end;
        end;
      end;

      (* Failed to tokenize? *)
      If not Result then
      Begin
        (* dispose of entry data *)
        Dispose(NewEntry);
        NewEntry:=NIL;
        SetLength(Root^.edSubEntries,FCount);
        Context.pcError:=Format('Invalid token "%s"',[declaration.cxToken]);
      end;

    end else
    Error:='AddEntry failed, root can not be NIL';
  end;

  Function  RMLParseObject(Var Context:TRMLParseContext;
            Const Root:PRMLEntryData):Boolean;
  var
    FChar:        Char;
    FTemp:        String;
    FDeclaration: TRMLEntryDeclaration;
    FNewEntry:    PRMLEntryData;
  Begin
    Result:=Context.pcSignature=SizeOf(Context);
    If result then
    Begin
      (* update cursor *)
      inc(Context.pcPos);
      inc(Context.pcCol);

      while Result and (Context.pcPos<Context.pcLen) do
      Begin
        FChar:=Context.pcData[Context.pcPos];
        Case FCHAR of
        #13:
          Begin
            inc(Context.pcRow);
            inc(Context.pcPos);
            Context.pcCol:=0;
            Continue;
          end;
        ';':
          Begin
            Result:=RMLParseEntry(trim(FTemp),FDeclaration,Context.pcError);
            If result then
            Result:=RMLAddEntry
              (
                Context,
                FDeclaration,
                root,
                FNewEntry,
                Context.pcError
              );

            If Result then
            Begin
              inc(Context.pcPos);
              inc(Context.pcCol);

              If FNewEntry^.edtype=etIF then
              Result:=RMLParseObject(Context,FNewEntry);
            end;
            FTemp:='';
          end;
        '{':
          Begin
            Result:=RMLParseEntry(FTemp,FDeclaration,Context.pcError);
            If Result then
            Begin
              Result:=FDeclaration.cxToken='if';
              If result then
              Begin
                Result:=RMLAddEntry
                  (
                  Context,
                  FDeclaration,
                  Root,
                  FNewEntry,
                  Context.pcError
                  );
                If Result then
                Result:=RMLParseObject(Context,FNewEntry);
              end;
            end;
            FTemp:='';
          end;
        '}':
          Begin
            inc(Context.pcCol);
            inc(Context.pcPos);
            Break;
          end;
        else
          Begin
            FTemp:=FTemp + FChar;
            inc(Context.pcCol);
            inc(Context.pcPos);
          end;
        end;
      end;
    end;
  end;

  Function RMLExecute(Const Context:TRMLParseContext):Boolean;

    Function RunEntry(Const Item:PRMLEntryData):Boolean;
    var
      FSubCount:  Integer;
      x:          Integer;
    Begin
      result:=True;

      Case Item^.edtype of
      etCreate,
      etAssignment:
        Begin

          FSubCount:=Length(Item^.edSubEntries);
          for x:=1 to FSubCount do
          Begin
            result:=RunEntry(Item^.edSubEntries[x-1]);
            If not result then
            break;
          end;

        end;
      etMethod:
        Begin
          {result:=RMLGetMethodEntry(Item^.edDeclaration.cxToken,
          Item^.edObject,FEntry);
          If result then
          result:=FEntry(Item); }
          Result:=TRMLProcEntry(Item^.edMethod)(Item);
        end;
      etRepeat:
        Begin
          //FSubCount:=Length(Item^.edSubEntries);
        end;
      etIf:
        Begin
          FSubCount:=Length(Item^.edSubEntries);
          for x:=1 to FSubCount do
          RunEntry(Item^.edSubEntries[x-1]);
        end;
      end;
    End;

  Begin
    Result:=Context.pcSignature=SizeOf(Context);
    If result then
    Begin
      result:=length(Context.pcError)<1;
      If result then
      result:=RunEntry(@Context.pcRoot);
    end;
  end;

  Function RMLCompile(Var Context:TRMLParseContext):Boolean;
  var
    FChar:        Char;
    FTemp:        String;
    FDeclaration: TRMLEntryDeclaration;
    FNewEntry:    PRMLEntryData;
  Begin
    Result:=Context.pcSignature=SizeOf(Context);
    If result then
    Begin
      Context.pcCol:=0;
      Context.pcRow:=0;
      Context.pcPos:=1;
      Context.pcError:='';

      while Result and (Context.pcPos<Context.pcLen) do
      Begin
        FChar:=Context.pcData[Context.pcPos];
        Case FCHAR of
        #13:
          Begin
            inc(Context.pcRow);
            inc(Context.pcPos);
            Context.pcCol:=0;
            Continue;
          end;
        '{':
          Begin
            Result:=RMLParseEntry(FTemp,FDeclaration,Context.pcError);
            If Result then
            Begin
              Result:=FDeclaration.cxToken='createobject';
              If result then
              Begin
                Result:=RMLAddEntry
                  (
                  Context,
                  FDeclaration,
                  @Context.pcRoot,
                  FNewEntry,
                  Context.pcError
                  );
                If Result then
                Result:=RMLParseObject(Context,FNewEntry);
              end;
            end;
            FTemp:='';
          end;
        '}':
          Begin
          end;
        else
          Begin
            FTemp:=FTemp + FChar;
            inc(Context.pcCol);
            inc(Context.pcPos);
          end;
        end;
      end;
    end;
  end;

  end.

Why C# coders should shut up about Delphi

October 18, 2016 Comments off
13507225_10209662785317165_99874043994628045_n

When in Rome

Those that follow my blog or know me personally – also know that I don’t go out of my way to be confrontational or disrespectful. I try my best to follow the middle-path, to see positive in all things. But sometimes you face a level of stupid that begs a response.¬†A verbal one. And this is one of those cases.

Lately I see more and more Delphi developers getting into debates with¬†C# developers, and they are confronted with an attitude and belief-system that quite frankly is utter nonsense. It’s not based on history or facts but disputes rooted in childish “isms”. A mix of old habits and an unhealthy obsession with the notion¬†that new means better.

It is nothing short of an intellectual emergency.

Old is bad?

If that is the case then C/C++ should be utter shit, because C is 3 years older than Pascal. Pascal was created to replace C and get away from (amongst other things) the absurd and time-consuming header file practice, not to mention the recursive hell that still occur when headers reference other headers and conditional symbols pile up.

Do you know how curly brackets came into being? Jump into your¬†nearest Tardis and set the destination to the mid 1960’s. Not 1969 like Wikipedia will tell you, but a few years earlier. Back when mainframes were¬†the size of Portugal¬†and 1024 bytes of memory made professors all girly inside. You see, back then memory was sparse and pretty much all the languages (or toolkits) needed to save space. So the { } curly’s were used as replacements for words like “begin” and “end”. Why? Because¬†one ascii byte is better than five ascii bytes when there’s only 1024 of them around.

Ps: when I use the word toolkit here I really mean a random soup of asm snippets dipped in venom. The bones collected to make C was actually a mix¬†of homebrew toolkits made by different engineering teams at the time. And each of these toolkits had their own fanclub¬†/slash/ legal representation; because all of them, miraculously, had invented the exact same thing at the same time before everyone else. These guys kept on fighting over who owned what¬†well into the Watcom era. It’s probably the longest standing demonstration of a grown-up tantrum¬†since the roman emperor caligula outlawed clothes.

Fast forward to 1970¬†and the amount of memory¬†on mainframes had already doubled a few times just like Moore had predicted five years earlier.¬†The C standard that Dennis Richie had painstakingly wrangled from the cold, greedy¬†hands of grumpy old technicians¬†was due for an upgrade.¬†Tests were done by very dull, very serious men in grey clothing – and in their horror¬†they discovered that human beings would interact with¬†english words and phrases faster than¬†symbols and glyphs. So a word like “begin” would be picked up by the brain faster than {. It is just knockout stuff isn’t it.

You cannot praise Anders as a genius architect and at the same time throw his life work in the thrash. Perhaps you should ask yourself why Anders loved object pascal so much that he dedicated half his career making sure universities had access to it.

Living on the edge with a red tie, Nicolaus Wirth rocks the scene!

Living on the edge with a red tie, Nicolaus Wirth rocks the scene!

Enter¬†Niklaus Wirth, a man so clever and academic that he doesn’t catch exceptions; He is the exception. But next¬†to his colleagues over at Berkley our dear Niklaus is¬†the proverbial rock-star. He decided that in order to make programming more accessible for students, easier to learn and above all, establishing a standard for safer code, that they should take science to heart and include that in a programming language design.¬†So he put on his gravest lab coat and began to implement “a better C”.¬†Nine months later pascal was born,¬†and to celebrate Niklaus used colored socks that day. A risky bit of business that¬†could get him expelled from the university and cost¬†him his seat¬†at the strangely brown sorority club, in which he was master of ceremonies. But¬†hey, Nikolaus is¬†a rebel and this¬†was the swinging 70’s after all.

My point? This is why pascal uses “begin” and “end” as opposed to { and }. Science. Real science.

Whimsical satire aside: with the knowledge that C is actually older than pascal, not to mention that it is identical in features, depth and complexity–¬†what is it going to be? Do you still cling to the notion that older have to be of less value than new? Or could it in fact be that you havent really looked closer at object pascal to begin with? Have you even installed a modern version of Delphi and given it¬†a worthy test drive? Or do you just speak out based on what you like and know, as opposed to what you don’t know and havent even tried.

My 8-year-old daughter is like that. She makes up her mind about food before she has even tasted it. Surely a trained engineer can do better?

And I say that because —¬†if we follow your line of reasoning, C/C++ should be incapable of producing C#. How can something old and outdated possibly produce something new and exciting? You do know that C# is written in C¬†and assembler¬†right? And that Java and all the other high level languages out there didn’t spontaneously self assemble. They are all written in C, assembler or pascal. And there is a reason for that.

Just like nature have¬†different levels, from particles to minerals, from minerals to bacteria, from bacteria¬†to plants, from plants¬†to insects (and so on) — so does a computer. Each of these levels have laws and no matter how much you want to introduce something from a higher¬†level into a lower level – that’s just not how reality works.

foodchain

The fact that I have to explain this in 2016 demonstrates how education and academia has degraded after C and Pascal was replaced with Java.

The law is very simple: each level can only emit complexity upwards. So assembler is used to produce C and pascal, C and pascal is used to produce C++ and object pascal, C++ and object pascal is used to bring about every other single piece of tech you take for granted.

So when you criticize Delphi but use C#, you are actually saying “I want a T-Rex¬†to inhabit the mineral world”.¬†It’s just utterly divorced from reality.

It’s like saying Visual Basic 5 is cooler than assembler. You don’t debate with people like that, you just look at them with pity. I don’t know how many years a software engineering¬†¬†degree is elsewhere, but¬†if you just spent 3 years in college and 2 years at a university followed by one or two¬†additional years getting your diploma¬†–¬†and this is the level of insight you graduate with then I pity you. And I pity your employer because sooner or later you will bring about the apocalypse.

If we apply rational logical thought to the proposition at hand, we cannot but conclude that¬†the notion of “new is always better” is false.¬†It is to mistake marketing for facts and capitalism for science.

Your teacher should have taught you this: “In a computer, like nature, the past is always alive“. Look closely at your brand new pc: Bios, Vesa screen modes, non linear memory, hardware interrupt vectors and a cpu that supports no less than 3 instruction sets. Do you think Windows or the Linux kernel can just ignore this?¬†Why not call up Linus Torvalds and ask him why he’s not using C# in his line of work; or NVidia why their GPU pipeline only ships with C headers and not C++ classes. I sure as hell wouldn’t want to be on the receiving end¬†of that call.

C# is better than Delphi?

Anders Hejlsberg, the father of many languages

Anders Hejlsberg, the father of many languages

Actually, it’s not.¬†What today is known as .net, from its¬†CIL intermediate language right down to the global assembly cache¬†is Delphi. Yes you read that correctly the entire .net product family is Delphi. It was described on Borland’s news-servers years before it magically appeared as Microsoft’s flagship product.

Anders Hejlsberg created and prototyped these technologies while he was working at Borland. And indeed, the father of C# and entire dot net stack is none other than the father of Delphi.

At Borland¬†(in the borland.language newsfeed¬†I vividly remember) these technologies went mentioned under¬†the codename “portable Delphi”. I followed the thread on the¬†news servers for quite some time. It was no secret that Borland was tinkering away on the next big thing, and I believe it was ultimately Borland’s response to Java. Because at the time Java had eaten quite a chunk of Borlands educational seat licenses. And had Anders continued working for Borland, Delphi and C++ builder would¬†perhaps be bytecode based today.

So when you, the C# fanboy, propose that Delphi is somehow inferior to dot net, or that object pascal is outdated and technically behind C# you are not just shooting yourself in one foot, but both feet at the same time. Keep on reading and you will understand why.

But just to make this crystal clear: Anders Hejlsberg is not just the father of C# and dot net: he is also the father of Delphi. And before that he gave birth to Turbo Pascal.

Borland, land of the free

Borland, the company Anders worked for, was put out to pasture by Microsoft. Bill Gates and his thugs launched a financial onslaught that few companies on the planet could have endured. The reason was that Microsoft really had the worst developer tools on the market, while Borland represented the very best. So for a long time Borland was a direct threat to Microsoft. And they were winning every single time.

When Borland later began to flirt with Linux (Delphi Kylix) it must have made the R&D department at Microsoft piss themselves. Delphi is responsible for thousands of desktop applications synonymous with Windows – and if Linux users got their hands on RAD tools like Delphi, they were in a real position to hurt¬†Microsoft. You have to remember that this was before cloud computing.¬†Microsoft didn’t have¬†a fallback strategy and depended on desktop and server sales. Linux could do everything Windows could, but it lacked the desktop applications, the user friendliness and features Borland made simple. But that is a whole different story so let’s not deviate from our timeline.

Weird Science, a fun but completely fictional movie from 1985. I just posted this to liven up an otherwise boring trip down memory lane

Weird Science, a fun but completely fictional movie from 1985. My point? People who don’t know how computers worked were actually afraid this pseudo-garbage could one day be real. Just like C# and Java programmers believe they are top-dog, they are in fact standing on the shoulders of giants. C/C++ and object pascal is holding you up.

What Microsoft did was, basically, to buy out Anders from Borland and at the same time attack the company from every angle. Drowning them in bullshit lawsuits, patent claims and all the nasty, cowardly practices Microsoft is renowned for.

Bill Gates called Anders up personally¬†and gave him “an offer I could not refuse” as Anders dubs it in various interviews over the years.

So to put this myth to rest and be done with it: C# as a language and dot net as a technology did not originate with Microsoft. It has Borland written all over it and its architecture is intimately joined at the hip with Delphi and C++ builder. Companies like Microsoft dont innovate any more, they buy competence and re-brands it as their own.

Dot Net framework, you mean the VCL?

If that is not enough, take a long,¬†hard look at how the .net framework is organized. Pay special attention to the features and methods of the base-classes and RTTI. What you are looking at is the VCL, the visual component library, Delphi and C++ builder’s run time library.

There was never any secret where this came from, but it’s not advertised either so I don’t blame people for not recognizing it. But when people start to behave nasty over what should be either a passionate hobby or a professional discipline, then truth must¬†be told.¬†The very foundation of the dot net framework is taken straight out of the VCL.

And why shouldnt it? The whole platform was whispered about under the codename “portable Delphi” to begin with, it was even designed by the same author – so why should it come as a surprise that it’s basically Delphi reincarnated with a different syntax parser? Either way a few bells should start ringing in your head – telling you that you may have been wrong about Delphi and object pascal in general.¬†Probably because you know very little about Delphi to begin with.

So, what you perhaps believe sets C# apart from Delphi, what you imagine gives dot net an advantage over Delphi Рis in reality the opposite: it is Delphi. It originated in Delphi, was inspired by Delphi and flows straight from the inventor of Delphi himself. Anders is responsible for two distinct object pascal development platforms prior to working for Microsoft. Take some time to reflect on that.

You cannot praise Anders as a genius architect and at the same time throw his life work in the thrash. Perhaps you should ask yourself why Anders loved object pascal so much that he dedicated half his career making sure universities had access to it.

Strength in diversity

The fact that C/C++ is alive and well today serves to prove my point: age has no meaning when dealing with fundamental technology. These languages embody the fundamental principles of computing. It’s like a force of nature. Electricity was never invented, it was there all along as a part of nature – we just had to learn the principles in order to tame it. And archetypical programming languages is almost the same thing. They don’t grow old because they represent¬†a fundamental level that can never be replaced.

Things are the way they are for a reason. And programmers before you established standards that address things you might not have to deal with, but that’s because they dealt with it before you. Just because you don’t see the plumbing doesn’t mean it’s not there.

These are the compilers responsible for the foundation on which all other languages, operative systems, applications and services rest. Remove those, and the whole house of cards come crashing down.

So C/C++ and object pascal continue to exist, thrive even, precisely because they interface with the hardware itself, not a virtual machine or bytecode format. Delphi as a product is naturally tailored for operative systems, as is C/C++. But that is because we use programming languages to create applications. There is nothing in the way of someone using compilers like freepascal and writing an operative system from scratch.

Ultibo is an ARM operative system written purely in object pascal

Ultibo is an ARM operative system written purely in object pascal

By contrast,¬†C#, Java and Visual Basic are context dependent languages. They exist purely in context with an operative system or run-time environment. They can’t exist without these and lack the depth C and pascal brings to the table. The depth to span many levels, from the lowest all the way up to cloud and domain driven programming.

Just look at Android,¬†an operative system written in Java right? Well¬†that’s not really true. The Android operative system depends on a native bootstrap, a piece of code that establishes the run-time environment for Java applications. So when you fire up your Android phone, tablet or set-top box, the first thing to boot is a loading mechanism written in vanilla C. This in turn mounts a filesystem and continues to loads in bunch of native drivers. Drivers for the display, storage, wi-fi and all the hardware¬†components that ultimately make up your SoC (system on a chip).

Its only after this long process that the environment is capable of hosting the Java virtual machine. Then and only then is the device capable of executing Java.

7709-f-1

Notice the “C” part of the Java NDK? Wasnt that exactly what Java was supposed to solve? All that hype and here you are, back at where you started!

So Java, which for the past decade or more have bombarded us with advertising like “platform independence” is a total flop!¬†Its like communism and looks good on paper, but once you start testing¬†these ideas in real life – they just don’t work.

Turned out that the java virtual machine when running on small devices like phones and tablets, choked the CPU. Which in turn used more electricity to keep up, which in turn rendered the battery life of these devices null and void. Talk about irony. All that hype and Java couldn’t even deliver¬†its most basic promise: platform independence. And you removed C and pascal from our universities for this? It’s a disaster en-par with the burning of the great library of Alexandria.

There is more than a drop of ego and arrogance in this material. Young people especially like to believe they are smarter than those who lived before them. It’s a very selfish idea that really need to be addressed. Take someone like Alan Turing for example, the inventor of what we call computers today. This man assembled and created the first computer by hand. He even hand carved the wheels that made the mechanism run (which was mechanical during the second world war).¬†I would happily bet¬†$1000 that Alan could out program¬†the lot of us had he gotten the chance. He has been dead for many years, but to imagine yourself smarter or more capable than him would be the definition of arrogance.

PS: This is where we come into the realm of wisdom versus knowledge.¬†The difference may be subtle, but it’s so important to distinguish between them.

C# would have been a sad little language without the work of this man, Miguel de icaza

C# would have been a sad little language without the work of this man, Miguel de icaza

Either way without the native context to support it Java turned out to be a blundering behemoth. And its the exact same with C#. This is why Miguel de Icaza work on Mono has been so instrumental to the success of dot net and C# in particular. It was Miguel and his team that pushed C# into native compilation versus JIT for mobile devices. C# would never have been allowed on IOS or Android otherwise.

So that is the difference between archetypical languages and context based languages. Hopefully you now see why this negative attitude towards Delphi and object pascal doesn’t make sense. It really is just as stupid as¬†these cults that believe in dinosaurs and that human beings once lived side by side with a T-rex.¬†Anyone with a highschool diploma¬†should instinctively know better.

The problem is when people in great enough numbers start to believe in this nonsense, because then they can actually damage important mechanisms of our eco-system. Insane beliefs really is dangerous both socially and technically.

Grow up!

I have no problem using C# and consider it a valuable tool in my toolbox. Same with Delphi. But when it comes to kick-ass desktop applications for Windows and OS X, Delphi has no rival. It gives you a productive edge that I havent found in any other language. And I don’t base that on preference. I base that on engineering and the years of development invested in Delphi by both Borland and Embarcadero.

I just find it odd that C# developers seem to lack the ability to speak well about Delphi, because it’s a tell-tell sign of immaturity to be incapable of that. Object pascal and C++ is identical in every way (except multiple inheritance, which even C++ programmers shun like the plague so no loss there). Why then should educated programmers find it reasonable to praise C but thrash¬†pascal? It’s like saying that one car is faster than another – but when you pop the hood they have the same engine. The exact same machine code generator even. ¬†It is psychological masturbation. We are coders, we shouldnt do things like that. We should examine the system carefully and then make up our minds.

And what is a programming language anyway? It is the ability to describe the solution to a problem. The ability to define and control how a series of activities execute. Each language represents a methodology for solving problems. A mindset. Nothing more.

Languages like C, C++ and object pascal will always exist, they will always thrive, because they¬†are instrumental to everything else (desktop, operative system, services, everything that makes a computer capable of what it does). It doesn’t mean that Delphi is the solution to all problems, far from it. And¬†that is ok.

Context based languages are more often than not designed for a purpose. Java was initially designed as a network language. This makes Java infinitely more suitable for dealing with that line of work. You can do the exact same in C but you would have to add a few libraries and establish a platform. Which is exactly what C and pascal is good at, implementing fundamental technology. Object pascal is a language you would use when creating other languages. C is more popular for virtual machine development, no doubt there, but there is nothing in C/C++ that you wont find in object pascal.

Interestingly, languages that many¬†believe is long since extinct, like Cobol, are actually alive. Cobol was created to deal with large stacks of numbers and to push associated data through formulas. That is its context and no other language is better suited for the banking sector than Cobol. Which incidentally is the one place where Cobol is used¬†to this day. And why not? Why waste millions trying to hammer in a nail with a hacksaw? Why not use a language especially designed for that purpose? SQL is likewise a language especially tailored for its tasks. I don’t hear anyone bitching about how old and outdated that¬†is. It wont grow old because it embodies the principle of relational data.

And just what exactly is old about Delphi? Generics? Posix support? Weak and strong references? Garbage collection? Attributes? class, record and enum helpers? Dependency injection? Model view controller frameworks? The ability to write assembler side by side with pascal? The ability to link with C/C++ objects? Cross compilation for Windows, OS X and mobile platforms? What manner of logic are you using where you praise other languages for these features, but for Delphi the same features makes it old? Stop repeating what someone once told you like a mindless parrot and investigate. You are an engineer, a field of science; start acting like one.

In my work I use several languages: JavaScript, Delphi, Smart Pascal, Mono C# and even Python. I have also fallen in love with LUA believe it or not.

For web-pages I prefer Smart pascal which compiles to JavaScript (node.js is fantastic). Then you have shell scripting, a touch of x86 assembly code, a dash of bash on my Linux box and .. well, there is strength in diversity.

It all depends on what is best for the customer and what language yields the most robust solution. Sometimes C# is better suited for my clients infrastructure, other times Delphi makes more sense. But if I was to pick a single language to cover all of them, it would either be object pascal or C++ (gcc). These really have no rivals and will never go away.

These are the compilers responsible for the foundation on which all other languages, operative systems, applications and services rest. Remove those, and the whole house of cards come crashing down.

So using many languages is actually an enriching experience. It brings more depth to our way of thinking and (consequently) affect the quality of our products.

Anyways, I believe¬†congratulations are in order, because if your favorite language is C#, you have in fact been using Delphi without knowing it. There is so much Delphi in C# and they both originate from the same author. So why not pay a visit to C#’s older brother?

So go download Delphi, Smart Mobile Studio, Lazarus and Remobjects Oxygene Рenjoy the decades of research and excellence these dialects represent. It wont hurt you, it will probably do you a great deal of good.